You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Niu Zhaojie <nz...@gmail.com> on 2017/06/13 11:08:34 UTC
Can I use ChannelTrafficShapingHandler to control the network
read/write speed in shuffle?
Hi All:
I am trying to control the network read/write speed with
ChannelTrafficShapingHandler provided by Netty.
In TransportContext.java
I modify it as below:
public TransportChannelHandler initializePipeline(
SocketChannel channel,
RpcHandler channelRpcHandler) {
try {
// added by zhaojie
logger.info("want to try control read bandwidth on host: " + host);
final ChannelTrafficShapingHandler channelShaping = new
ChannelTrafficShapingHandler(50, 50, 1000);
TransportChannelHandler channelHandler =
createChannelHandler(channel, channelRpcHandler);
channel.pipeline()
.addLast("encoder", ENCODER)
.addLast(TransportFrameDecoder.HANDLER_NAME,
NettyUtils.createFrameDecoder())
.addLast("decoder", DECODER)
.addLast("channelTrafficShaping", channelShaping)
.addLast("idleStateHandler", new IdleStateHandler(0, 0,
conf.connectionTimeoutMs() / 1000))
// NOTE: Chunks are currently guaranteed to be returned in
the order of request, but this
// would require more logic to guarantee if this were not
part of the same event loop.
.addLast("handler", channelHandler);
I create a ChannelTrafficShapingHandler and register it into the pipeline
of the channel. I set the write and read speed as 50kb/sec in the
constructor.
Except for it, what else do I need to do?
However, it does not work. Is this idea correct? Am I missing something?
Is there any better way ?
Thanks.
--
*Regards,*
*Zhaojie*