Spark Network Module behaviour

[sbiookag <sb...@asu.edu>]

Dear All,

I am trying to understand how exactly spark network module works. Looking at
Netty package, I would like to intercept every server response for block
fetch. As I understood the place which is responsible for sending remote
blocks is "TransportRequestHandler.processFetchRequest". Im trying to
distinguish these flows from other flows in a switch using the information I
get from "channel" object in this function. These information are source and
destination IP and source and destination PORT. I get all these four numbers
from the channel object. The problem rises when I'm looking at the switch
log and I see extensive traffic going back and forth, which do not belong to
these flows I'm capturing. Now my question is, I am missing other places,
which are responsible for huge network traffic? Is the port and IP I get
from channel is not the correct one?

Thank,
Saman



--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Spark-Network-Module-behaviour-tp14402.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org