You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by Qian Zhang <zh...@gmail.com> on 2017/10/09 01:03:51 UTC
Review Request 62836: Ignored the tasks already being killed when
killing the task group.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62836/
-----------------------------------------------------------
Review request for mesos and Vinod Kone.
Bugs: MESOS-8051
https://issues.apache.org/jira/browse/MESOS-8051
Repository: mesos
Description
-------
When the scheduler tries to kill multiple tasks in the task group
simultaneously, the default executor will kill the tasks one by
one. When the first task is killed, the default executor will kill
all the other tasks in the task group, however, we need to ignore
the tasks which are already being killed, otherwise, the check
`CHECK(!container->killing);` in `DefaultExecutor::kill()` will fail.
Diffs
-----
src/launcher/default_executor.cpp 136c000917b2c4fdaf68fd460764c3a15fdf3bf0
Diff: https://reviews.apache.org/r/62836/diff/1/
Testing
-------
Thanks,
Qian Zhang
Re: Review Request 62836: Ignored the tasks already being killed when
killing the task group.
Posted by Qian Zhang <zh...@gmail.com>.
> On Oct. 12, 2017, 4:10 p.m., Benjamin Bannier wrote:
> > Does it makes sense to add a test for this?
Yeah, I added a test here: https://reviews.apache.org/r/62837/
- Qian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62836/#review187752
-----------------------------------------------------------
On Oct. 9, 2017, 9:03 a.m., Qian Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62836/
> -----------------------------------------------------------
>
> (Updated Oct. 9, 2017, 9:03 a.m.)
>
>
> Review request for mesos and Vinod Kone.
>
>
> Bugs: MESOS-8051
> https://issues.apache.org/jira/browse/MESOS-8051
>
>
> Repository: mesos
>
>
> Description
> -------
>
> When the scheduler tries to kill multiple tasks in the task group
> simultaneously, the default executor will kill the tasks one by
> one. When the first task is killed, the default executor will kill
> all the other tasks in the task group, however, we need to ignore
> the tasks which are already being killed, otherwise, the check
> `CHECK(!container->killing);` in `DefaultExecutor::kill()` will fail.
>
>
> Diffs
> -----
>
> src/launcher/default_executor.cpp 136c000917b2c4fdaf68fd460764c3a15fdf3bf0
>
>
> Diff: https://reviews.apache.org/r/62836/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Qian Zhang
>
>
Re: Review Request 62836: Ignored the tasks already being killed when
killing the task group.
Posted by Benjamin Bannier <bb...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62836/#review187752
-----------------------------------------------------------
Does it makes sense to add a test for this?
- Benjamin Bannier
On Oct. 9, 2017, 3:03 a.m., Qian Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62836/
> -----------------------------------------------------------
>
> (Updated Oct. 9, 2017, 3:03 a.m.)
>
>
> Review request for mesos and Vinod Kone.
>
>
> Bugs: MESOS-8051
> https://issues.apache.org/jira/browse/MESOS-8051
>
>
> Repository: mesos
>
>
> Description
> -------
>
> When the scheduler tries to kill multiple tasks in the task group
> simultaneously, the default executor will kill the tasks one by
> one. When the first task is killed, the default executor will kill
> all the other tasks in the task group, however, we need to ignore
> the tasks which are already being killed, otherwise, the check
> `CHECK(!container->killing);` in `DefaultExecutor::kill()` will fail.
>
>
> Diffs
> -----
>
> src/launcher/default_executor.cpp 136c000917b2c4fdaf68fd460764c3a15fdf3bf0
>
>
> Diff: https://reviews.apache.org/r/62836/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Qian Zhang
>
>
Re: Review Request 62836: Ignored the tasks already being killed when
killing the task group.
Posted by Vinod Kone <vi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62836/#review189670
-----------------------------------------------------------
Ship it!
Ship It!
- Vinod Kone
On Oct. 12, 2017, 12:57 p.m., Qian Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62836/
> -----------------------------------------------------------
>
> (Updated Oct. 12, 2017, 12:57 p.m.)
>
>
> Review request for mesos and Vinod Kone.
>
>
> Bugs: MESOS-8051
> https://issues.apache.org/jira/browse/MESOS-8051
>
>
> Repository: mesos
>
>
> Description
> -------
>
> When the scheduler tries to kill multiple tasks in the task group
> simultaneously, the default executor will kill the tasks one by
> one. When the first task is killed, the default executor will kill
> all the other tasks in the task group, however, we need to ignore
> the tasks which are already being killed, otherwise, the check
> `CHECK(!container->killing);` in `DefaultExecutor::kill()` will fail.
>
>
> Diffs
> -----
>
> src/launcher/default_executor.cpp 136c000917b2c4fdaf68fd460764c3a15fdf3bf0
>
>
> Diff: https://reviews.apache.org/r/62836/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Qian Zhang
>
>
Re: Review Request 62836: Ignored the tasks already being killed when
killing the task group.
Posted by Qian Zhang <zh...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62836/
-----------------------------------------------------------
(Updated Oct. 12, 2017, 8:57 p.m.)
Review request for mesos and Vinod Kone.
Changes
-------
Addressed Vinod's comments.
Bugs: MESOS-8051
https://issues.apache.org/jira/browse/MESOS-8051
Repository: mesos
Description
-------
When the scheduler tries to kill multiple tasks in the task group
simultaneously, the default executor will kill the tasks one by
one. When the first task is killed, the default executor will kill
all the other tasks in the task group, however, we need to ignore
the tasks which are already being killed, otherwise, the check
`CHECK(!container->killing);` in `DefaultExecutor::kill()` will fail.
Diffs (updated)
-----
src/launcher/default_executor.cpp 136c000917b2c4fdaf68fd460764c3a15fdf3bf0
Diff: https://reviews.apache.org/r/62836/diff/2/
Changes: https://reviews.apache.org/r/62836/diff/1-2/
Testing
-------
Thanks,
Qian Zhang
Re: Review Request 62836: Ignored the tasks already being killed when
killing the task group.
Posted by Mesos Reviewbot <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62836/#review187384
-----------------------------------------------------------
Patch looks great!
Reviews applied: [62836]
Passed command: export OS='ubuntu:14.04' BUILDTOOL='autotools' COMPILER='gcc' CONFIGURATION='--verbose --disable-libtool-wrappers' ENVIRONMENT='GLOG_v=1 MESOS_VERBOSE=1'; ./support/docker-build.sh
- Mesos Reviewbot
On Oct. 9, 2017, 1:03 a.m., Qian Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62836/
> -----------------------------------------------------------
>
> (Updated Oct. 9, 2017, 1:03 a.m.)
>
>
> Review request for mesos and Vinod Kone.
>
>
> Bugs: MESOS-8051
> https://issues.apache.org/jira/browse/MESOS-8051
>
>
> Repository: mesos
>
>
> Description
> -------
>
> When the scheduler tries to kill multiple tasks in the task group
> simultaneously, the default executor will kill the tasks one by
> one. When the first task is killed, the default executor will kill
> all the other tasks in the task group, however, we need to ignore
> the tasks which are already being killed, otherwise, the check
> `CHECK(!container->killing);` in `DefaultExecutor::kill()` will fail.
>
>
> Diffs
> -----
>
> src/launcher/default_executor.cpp 136c000917b2c4fdaf68fd460764c3a15fdf3bf0
>
>
> Diff: https://reviews.apache.org/r/62836/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Qian Zhang
>
>
Re: Review Request 62836: Ignored the tasks already being killed when
killing the task group.
Posted by Mesos Reviewbot Windows <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62836/#review187377
-----------------------------------------------------------
FAIL: The system cannot find the file specified.
Reviews applied: `['62806']`
All the build artifacts available at: http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/62806
- Mesos Reviewbot Windows
On Oct. 9, 2017, 1:03 a.m., Qian Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62836/
> -----------------------------------------------------------
>
> (Updated Oct. 9, 2017, 1:03 a.m.)
>
>
> Review request for mesos and Vinod Kone.
>
>
> Bugs: MESOS-8051
> https://issues.apache.org/jira/browse/MESOS-8051
>
>
> Repository: mesos
>
>
> Description
> -------
>
> When the scheduler tries to kill multiple tasks in the task group
> simultaneously, the default executor will kill the tasks one by
> one. When the first task is killed, the default executor will kill
> all the other tasks in the task group, however, we need to ignore
> the tasks which are already being killed, otherwise, the check
> `CHECK(!container->killing);` in `DefaultExecutor::kill()` will fail.
>
>
> Diffs
> -----
>
> src/launcher/default_executor.cpp 136c000917b2c4fdaf68fd460764c3a15fdf3bf0
>
>
> Diff: https://reviews.apache.org/r/62836/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Qian Zhang
>
>
Re: Review Request 62836: Ignored the tasks already being killed when
killing the task group.
Posted by Mesos Reviewbot Windows <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62836/#review187376
-----------------------------------------------------------
FAIL: The system cannot find the file specified.
Reviews applied: `['62806']`
Failed command: `C:\mesos\src\mesos-tests.exe --verbose --gtest_filter="-ContentType/MasterAPITest.EventAuthorizationFiltering/1"`
All the build artifacts available at: http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/62806
Relevant logs:
- [mesos-tests-stdout.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/62806/logs/mesos-tests-stdout.log):
```
[ RUN ] FaultToleranceTest.SchedulerReregisterAfterUnregistration
[ OK ] FaultToleranceTest.SchedulerReregisterAfterUnregistration (389 ms)
[ RUN ] FaultToleranceTest.SchedulerFailoverRetriedReregistration
[ OK ] FaultToleranceTest.SchedulerFailoverRetriedReregistration (389 ms)
[ RUN ] FaultToleranceTest.DisconnectedSchedulerLaunchLost
[ OK ] FaultToleranceTest.DisconnectedSchedulerLaunchLost (609 ms)
[ RUN ] FaultToleranceTest.DisconnectedSchedulerLaunchDropped
[ OK ] FaultToleranceTest.DisconnectedSchedulerLaunchDropped (596 ms)
[ RUN ] FaultToleranceTest.SchedulerFailoverStatusUpdate
[ OK ] FaultToleranceTest.SchedulerFailoverStatusUpdate (923 ms)
[ RUN ] FaultToleranceTest.ReregisterFrameworkExitedExecutor
[ OK ] FaultToleranceTest.ReregisterFrameworkExitedExecutor (1249 ms)
[ RUN ] FaultToleranceTest.ForwardStatusUpdateUnknownExecutor
[ OK ] FaultToleranceTest.ForwardStatusUpdateUnknownExecutor (822 ms)
[ RUN ] FaultToleranceTest.SchedulerFailoverExecutorToFrameworkMessage
[ OK ] FaultToleranceTest.SchedulerFailoverExecutorToFrameworkMessage (941 ms)
[ RUN ] FaultToleranceTest.SchedulerFailoverFrameworkToExecutorMessage
[ OK ] FaultToleranceTest.SchedulerFailoverFrameworkToExecutorMessage (1060 ms)
[ RUN ] FaultToleranceTest.IgnoreKillTaskFromUnregisteredFramework
[ OK ] FaultToleranceTest.IgnoreKillTaskFromUnregisteredFramework (939 ms)
[ RUN ] FaultToleranceTest.SchedulerExit
[ OK ] FaultToleranceTest.SchedulerExit (841 ms)
[ RUN ] FaultToleranceTest.SlaveReliableRegistration
[ OK ] FaultToleranceTest.SlaveReliableRegistration (788 ms)
[ RUN ] FaultToleranceTest.SlaveReregisterOnZKExpiration
[ OK ] FaultToleranceTest.SlaveReregisterOnZKExpiration (778 ms)
[ RUN ] FaultToleranceTest.FrameworkReregisterEmptyExecutor
[ OK ] FaultToleranceTest.FrameworkReregisterEmptyExecutor (1130 ms)
[ RUN ] FaultToleranceTest.SplitBrainMasters
```
- [mesos-tests-stderr.log](http://dcos-win.westus.cloudapp.azure.com/mesos-build/review/62806/logs/mesos-tests-stderr.log):
```
I1006 17:26:12.991288 43216 master.cpp:8459] Removing framework ede9e20a-4e06-46fb-a22c-7e8013f03fa7-0000 (default) at scheduler-3a38ff4a-a81c-4c5b-9c40-38d63a94afd2@10.3.1.5:61771
I1006 17:26:12.991288 43216 master.cpp:3299] Deactivating framework ede9e20a-4e06-46fb-a22c-7e8013f03fa7-0000 (default) at scheduler-3a38ff4a-a81c-4c5b-9c40-38d63a94afd2@10.3.1.5:61771
I1006 17:26:12.992297 46984 hierarchical.cpp:412] Deactivated framework ede9e20a-4e06-46fb-a22c-7e8013f03fa7-0000
I1006 17:26:12.995283 43216 master.cpp:9389] Removing offer 60f5153f-ffdc-4156-8383-39cf5a6b25f8-O0
I1006 17:26:12.998283 46248 slave.cpp:869] Agent terminating
I1006 17:26:13.004289 21464 hierarchical.cpp:355] Removed framework ede9e20a-4e06-46fb-a22c-7e8013f03fa7-0000
I1006 17:26:13.007899 39284 master.cpp:1303] Agent ede9e20a-4e06-46fb-a22c-7e8013f03fa7-S0 at slave(37)@10.3.1.5:61771 (mesos-bld-s1.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) disconnected
I1006 17:26:13.008906 39284 master.cpp:3336] Disconnecting agent ede9e20a-4e06-46fb-a22c-7e8013f03fa7-S0 at slave(37)@10.3.1.5:61771 (mesos-bld-s1.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1006 17:26:13.008906 39284 master.cpp:3355] Deactivating agent ede9e20a-4e06-46fb-a22c-7e8013f03fa7-S0 at slave(37)@10.3.1.5:61771 (mesos-bld-s1.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net)
I1006 17:26:13.009905 46484 hierarchical.cpp:690] Agent ede9e20a-4e06-46fb-a22c-7e8013f03fa7-S0 deactivated
I1006 17:26:13.038906 46248 master.cpp:1145] Master terminating
I1006 17:26:13.040907 21464 hierarchical.cpp:626] Removed agent ede9e20a-4e06-46fb-a22c-7e8013f03fa7-S0
I1006 17:26:13.151911 46248 cluster.cpp:162] Creating default 'local' authorizer
I1006 17:26:13.184912 44660 master.cpp:445] Master 8c8fa538-3ba3-4608-8e12-2ba9b5151087 (mesos-bld-s1.zq4gs31qjdiunm1ryi1452nvnh.dx.internal.cloudapp.net) started on 10.3.1.5:61771
I1006 17:26:13.184912 44660 master.cpp:447] Flags at startup: --acls="" --agent_ping_timeout="15secs" --agent_reregister_timeout="10mins" --allocation_interval="1secs" --allocator="HierarchicalDRF" --authenticate_agents="true" --authenticate_frameworks="true" --authenticate_http_frameworks="true" --authenticate_http_readonly="true" --authenticate_http_readwrite="true" --authenticators="crammd5" --authorizers="local" --credentials="C:\Users\mesos\AppData\Local\Temp\2\BVZ2va\credentials" --filter_gpu_resources="true" --framework_sorter="drf" --help="false" --hostname_lookup="true" --http_authenticators="basic" --http_framework_authenticators="basic" --initialize_driver_logging="true" --log_auto_initialize="true" --logbufsecs="0" --logging_level="INFO" --max_agent_ping_timeouts="5" --max_completed_frameworks="50" --max_completed_tasks_per_framework="1000" --max_unreachable_tasks_per_framework="1000" --port="5050" --quiet="false" --recovery_agent_removal_limit="100%" --registry="in_memor
y" --registry_fetch_timeout="1mins" --registry_gc_interval="15mins" --registry_max_agent_age="2weeks" --registry_max_agent_count="102400" --registry_store_timeout="100secs" --registry_strict="false" --root_submissions="true" --user_sorter="drf" --version="false" --webui_dir="/webui" --work_dir="C:\Users\mesos\AppData\Local\Temp\2\BVZ2va\master" --zk_session_timeout="10secs"
I1006 17:26:13.223922 44660 master.cpp:496] Master only allowing authenticated frameworks to register
I1006 17:26:13.223922 44660 master.cpp:502] Master only allowing authenticated agents to register
I1006 17:26:13.223922 44660 master.cpp:508] Master only allowing authenticated HTTP frameworks to register
I1006 17:26:13.223922 44660 credentials.hpp:37] Loading credentials for authentication from 'C:\Users\mesos\AppData\Local\Temp\2\BVZ2va\credentials'
I1006 17:26:13.235913 44660 master.cpp:552] Using default 'crammd5' authenticator
I1006 17:26:13.236914 44660 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-master-readonly'
I1006 17:26:13.237915 44660 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-master-readwrite'
I1006 17:26:13.238914 44660 http.cpp:1045] Creating default 'basic' HTTP authenticator for realm 'mesos-master-scheduler'
I1006 17:26:13.239914 44660 master.cpp:631] Authorization enabled
I1006 17:26:13.270947 45764 master.cpp:2198] Elected as the leading master!
I1006 17:26:13.270947 45764 master.cpp:1687] Recovering from registrar
I1006 17:26:13.273921 39284 registrar.cpp:391] Successfully fetched the registry (0B) in 1.97504ms
I1006 17:26:13.273921 39284 registrar.cpp:495] Applied 1 operations in 0ns; attempting to update the registry
I1006 17:26:13.275921 21464 registrar.cpp:552] Successfully updated the registry in 999936ns
I1006 17:26:13.276916 21464 registrar.cpp:424] Success```
- Mesos Reviewbot Windows
On Oct. 9, 2017, 1:03 a.m., Qian Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62836/
> -----------------------------------------------------------
>
> (Updated Oct. 9, 2017, 1:03 a.m.)
>
>
> Review request for mesos and Vinod Kone.
>
>
> Bugs: MESOS-8051
> https://issues.apache.org/jira/browse/MESOS-8051
>
>
> Repository: mesos
>
>
> Description
> -------
>
> When the scheduler tries to kill multiple tasks in the task group
> simultaneously, the default executor will kill the tasks one by
> one. When the first task is killed, the default executor will kill
> all the other tasks in the task group, however, we need to ignore
> the tasks which are already being killed, otherwise, the check
> `CHECK(!container->killing);` in `DefaultExecutor::kill()` will fail.
>
>
> Diffs
> -----
>
> src/launcher/default_executor.cpp 136c000917b2c4fdaf68fd460764c3a15fdf3bf0
>
>
> Diff: https://reviews.apache.org/r/62836/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Qian Zhang
>
>
Re: Review Request 62836: Ignored the tasks already being killed when
killing the task group.
Posted by Qian Zhang <zh...@gmail.com>.
> On Oct. 12, 2017, 7:49 a.m., Vinod Kone wrote:
> > src/launcher/default_executor.cpp
> > Line 894 (original), 903 (patched)
> > <https://reviews.apache.org/r/62836/diff/1/?file=1849445#file1849445line903>
> >
> > Should we set this true even if the container is already in `killing`?
Yes, we should do it, thanks for the comment.
- Qian
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62836/#review187716
-----------------------------------------------------------
On Oct. 12, 2017, 8:57 p.m., Qian Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62836/
> -----------------------------------------------------------
>
> (Updated Oct. 12, 2017, 8:57 p.m.)
>
>
> Review request for mesos and Vinod Kone.
>
>
> Bugs: MESOS-8051
> https://issues.apache.org/jira/browse/MESOS-8051
>
>
> Repository: mesos
>
>
> Description
> -------
>
> When the scheduler tries to kill multiple tasks in the task group
> simultaneously, the default executor will kill the tasks one by
> one. When the first task is killed, the default executor will kill
> all the other tasks in the task group, however, we need to ignore
> the tasks which are already being killed, otherwise, the check
> `CHECK(!container->killing);` in `DefaultExecutor::kill()` will fail.
>
>
> Diffs
> -----
>
> src/launcher/default_executor.cpp 136c000917b2c4fdaf68fd460764c3a15fdf3bf0
>
>
> Diff: https://reviews.apache.org/r/62836/diff/2/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Qian Zhang
>
>
Re: Review Request 62836: Ignored the tasks already being killed when
killing the task group.
Posted by Vinod Kone <vi...@apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62836/#review187716
-----------------------------------------------------------
src/launcher/default_executor.cpp
Line 894 (original), 903 (patched)
<https://reviews.apache.org/r/62836/#comment264765>
Should we set this true even if the container is already in `killing`?
- Vinod Kone
On Oct. 9, 2017, 1:03 a.m., Qian Zhang wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62836/
> -----------------------------------------------------------
>
> (Updated Oct. 9, 2017, 1:03 a.m.)
>
>
> Review request for mesos and Vinod Kone.
>
>
> Bugs: MESOS-8051
> https://issues.apache.org/jira/browse/MESOS-8051
>
>
> Repository: mesos
>
>
> Description
> -------
>
> When the scheduler tries to kill multiple tasks in the task group
> simultaneously, the default executor will kill the tasks one by
> one. When the first task is killed, the default executor will kill
> all the other tasks in the task group, however, we need to ignore
> the tasks which are already being killed, otherwise, the check
> `CHECK(!container->killing);` in `DefaultExecutor::kill()` will fail.
>
>
> Diffs
> -----
>
> src/launcher/default_executor.cpp 136c000917b2c4fdaf68fd460764c3a15fdf3bf0
>
>
> Diff: https://reviews.apache.org/r/62836/diff/1/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Qian Zhang
>
>