You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@mesos.apache.org by pangbingqiang <pa...@huawei.com> on 2016/12/19 11:54:29 UTC

[MESOS-6240] Allow executor/agent communication over non-TCP/IP stream socket.

Hi all:
   What’s the latest information about MESOS-6240 https://issues.apache.org/jira/browse/MESOS-6240 ,have any demo or design achieve?
I see libprocess have support domain socket communication, does agent and executor have support communication by domain socket too?
If have any related imformation, please let me know, thanks~.

[cid:image001.png@01D0E8C5.8D08F440]


Bingqiang Pang(庞兵强)


Distributed and Parallel Software Lab
Huawei Technologies Co., Ltd.
Email:pangbingqiang@huawei.com<ma...@huawei.com>

Re: [MESOS-6240] Allow executor/agent communication over non-TCP/IP stream socket.

Posted by tommy xiao <xi...@gmail.com>.

haosdent, do you konw unified containerized howto handle the process.

2016-12-20 10:08 GMT+08:00 haosdent <ha...@gmail.com>:

> > what reason for executors need communication with agent
> Executors need to report task statuses to the agent. Agent needs to send
> launch task command to the executors.
> Suppose executors and agents locate in different network namespaces, they
> could not communicate with each other unless we support communication via
> domain socket.
>
> On Tue, Dec 20, 2016 at 7:22 AM, tommy xiao <xi...@gmail.com> wrote:
>
> > don't understand what reason for executors need communication with agent?
> >
> >
> > 2016-12-19 19:54 GMT+08:00 pangbingqiang <pa...@huawei.com>:
> >
> >> Hi all:
> >>
> >>    What’s the latest information about MESOS-6240
> >> https://issues.apache.org/jira/browse/MESOS-6240 ,have any demo or
> >> design achieve?
> >>
> >> I see libprocess have support domain socket communication, does agent
> and
> >> executor have support communication by domain socket too?
> >>
> >> If have any related imformation, please let me know, thanks~.
> >>
> >>
> >>
> >> [image: cid:image001.png@01D0E8C5.8D08F440]
> >>
> >>
> >>
> >> Bingqiang Pang(庞兵强)
> >>
> >>
> >>
> >> Distributed and Parallel Software Lab
> >>
> >> Huawei Technologies Co., Ltd.
> >>
> >> Email:pangbingqiang@huawei.com <su...@huawei.com>
> >>
> >>
> >>
> >>
> >>
> >
> >
> >
> > --
> > Deshi Xiao
> > Twitter: xds2000
> > E-mail: xiaods(AT)gmail.com
> >
>
>
>
> --
> Best Regards,
> Haosdent Huang
>



-- 
Deshi Xiao
Twitter: xds2000
E-mail: xiaods(AT)gmail.com

Re: [MESOS-6240] Allow executor/agent communication over non-TCP/IP stream socket.

Posted by tommy xiao <xi...@gmail.com>.

haosdent, do you konw unified containerized howto handle the process.

2016-12-20 10:08 GMT+08:00 haosdent <ha...@gmail.com>:

> > what reason for executors need communication with agent
> Executors need to report task statuses to the agent. Agent needs to send
> launch task command to the executors.
> Suppose executors and agents locate in different network namespaces, they
> could not communicate with each other unless we support communication via
> domain socket.
>
> On Tue, Dec 20, 2016 at 7:22 AM, tommy xiao <xi...@gmail.com> wrote:
>
> > don't understand what reason for executors need communication with agent?
> >
> >
> > 2016-12-19 19:54 GMT+08:00 pangbingqiang <pa...@huawei.com>:
> >
> >> Hi all:
> >>
> >>    What’s the latest information about MESOS-6240
> >> https://issues.apache.org/jira/browse/MESOS-6240 ,have any demo or
> >> design achieve?
> >>
> >> I see libprocess have support domain socket communication, does agent
> and
> >> executor have support communication by domain socket too?
> >>
> >> If have any related imformation, please let me know, thanks~.
> >>
> >>
> >>
> >> [image: cid:image001.png@01D0E8C5.8D08F440]
> >>
> >>
> >>
> >> Bingqiang Pang(庞兵强)
> >>
> >>
> >>
> >> Distributed and Parallel Software Lab
> >>
> >> Huawei Technologies Co., Ltd.
> >>
> >> Email:pangbingqiang@huawei.com <su...@huawei.com>
> >>
> >>
> >>
> >>
> >>
> >
> >
> >
> > --
> > Deshi Xiao
> > Twitter: xds2000
> > E-mail: xiaods(AT)gmail.com
> >
>
>
>
> --
> Best Regards,
> Haosdent Huang
>



-- 
Deshi Xiao
Twitter: xds2000
E-mail: xiaods(AT)gmail.com

Re: [MESOS-6240] Allow executor/agent communication over non-TCP/IP stream socket.

Posted by haosdent <ha...@gmail.com>.

> what reason for executors need communication with agent
Executors need to report task statuses to the agent. Agent needs to send
launch task command to the executors.
Suppose executors and agents locate in different network namespaces, they
could not communicate with each other unless we support communication via
domain socket.

On Tue, Dec 20, 2016 at 7:22 AM, tommy xiao <xi...@gmail.com> wrote:

> don't understand what reason for executors need communication with agent?
>
>
> 2016-12-19 19:54 GMT+08:00 pangbingqiang <pa...@huawei.com>:
>
>> Hi all:
>>
>>    What’s the latest information about MESOS-6240
>> https://issues.apache.org/jira/browse/MESOS-6240 ,have any demo or
>> design achieve?
>>
>> I see libprocess have support domain socket communication, does agent and
>> executor have support communication by domain socket too?
>>
>> If have any related imformation, please let me know, thanks~.
>>
>>
>>
>> [image: cid:image001.png@01D0E8C5.8D08F440]
>>
>>
>>
>> Bingqiang Pang(庞兵强)
>>
>>
>>
>> Distributed and Parallel Software Lab
>>
>> Huawei Technologies Co., Ltd.
>>
>> Email:pangbingqiang@huawei.com <su...@huawei.com>
>>
>>
>>
>>
>>
>
>
>
> --
> Deshi Xiao
> Twitter: xds2000
> E-mail: xiaods(AT)gmail.com
>



-- 
Best Regards,
Haosdent Huang

Re: [MESOS-6240] Allow executor/agent communication over non-TCP/IP stream socket.

Posted by haosdent <ha...@gmail.com>.

> what reason for executors need communication with agent
Executors need to report task statuses to the agent. Agent needs to send
launch task command to the executors.
Suppose executors and agents locate in different network namespaces, they
could not communicate with each other unless we support communication via
domain socket.

On Tue, Dec 20, 2016 at 7:22 AM, tommy xiao <xi...@gmail.com> wrote:

> don't understand what reason for executors need communication with agent?
>
>
> 2016-12-19 19:54 GMT+08:00 pangbingqiang <pa...@huawei.com>:
>
>> Hi all:
>>
>>    What’s the latest information about MESOS-6240
>> https://issues.apache.org/jira/browse/MESOS-6240 ,have any demo or
>> design achieve?
>>
>> I see libprocess have support domain socket communication, does agent and
>> executor have support communication by domain socket too?
>>
>> If have any related imformation, please let me know, thanks~.
>>
>>
>>
>> [image: cid:image001.png@01D0E8C5.8D08F440]
>>
>>
>>
>> Bingqiang Pang(庞兵强)
>>
>>
>>
>> Distributed and Parallel Software Lab
>>
>> Huawei Technologies Co., Ltd.
>>
>> Email:pangbingqiang@huawei.com <su...@huawei.com>
>>
>>
>>
>>
>>
>
>
>
> --
> Deshi Xiao
> Twitter: xds2000
> E-mail: xiaods(AT)gmail.com
>



-- 
Best Regards,
Haosdent Huang

Re: [MESOS-6240] Allow executor/agent communication over non-TCP/IP stream socket.

Posted by tommy xiao <xi...@gmail.com>.

don't understand what reason for executors need communication with agent?


2016-12-19 19:54 GMT+08:00 pangbingqiang <pa...@huawei.com>:

> Hi all:
>
>    What’s the latest information about MESOS-6240
> https://issues.apache.org/jira/browse/MESOS-6240 ,have any demo or design
> achieve?
>
> I see libprocess have support domain socket communication, does agent and
> executor have support communication by domain socket too?
>
> If have any related imformation, please let me know, thanks~.
>
>
>
> [image: cid:image001.png@01D0E8C5.8D08F440]
>
>
>
> Bingqiang Pang(庞兵强)
>
>
>
> Distributed and Parallel Software Lab
>
> Huawei Technologies Co., Ltd.
>
> Email:pangbingqiang@huawei.com <su...@huawei.com>
>
>
>
>
>



-- 
Deshi Xiao
Twitter: xds2000
E-mail: xiaods(AT)gmail.com

Re: [MESOS-6240] Allow executor/agent communication over non-TCP/IP stream socket.

Posted by Greg Mann <gr...@mesosphere.io>.

Hi Bingqiang,
I've had some recent discussions with folks about this feature; it's
something we are interested in doing but I'm not sure what its priority is
in relation to other tickets.

While the AF_UNIX address type has been added to libprocess, libprocess
does not currently accept domain socket connections. Domain sockets are
used by the I/O switchboard in the Mesos agent to communicate with running
containers, but this is mostly application-level code. Libprocess currently
exposes a single server-side TCP socket [1] for incoming connections. To
support executor communication via domain sockets, we would need to add a
new server-side domain socket to libprocess.

Are there any particular goals or use cases you have in mind for this
feature?

Cheers,
Greg

[1]
https://github.com/apache/mesos/blob/be127e6eca1312bf8c2b039646f6909fa42cd342/3rdparty/libprocess/src/process.cpp#L571

On Mon, Dec 19, 2016 at 3:54 AM, pangbingqiang <pa...@huawei.com>
wrote:

> Hi all:
>
>    What’s the latest information about MESOS-6240
> https://issues.apache.org/jira/browse/MESOS-6240 ,have any demo or design
> achieve?
>
> I see libprocess have support domain socket communication, does agent and
> executor have support communication by domain socket too?
>
> If have any related imformation, please let me know, thanks~.
>
>
>
> [image: cid:image001.png@01D0E8C5.8D08F440]
>
>
>
> Bingqiang Pang(庞兵强)
>
>
>
> Distributed and Parallel Software Lab
>
> Huawei Technologies Co., Ltd.
>
> Email:pangbingqiang@huawei.com <su...@huawei.com>
>
>
>
>
>

Re: [MESOS-6240] Allow executor/agent communication over non-TCP/IP stream socket.

Posted by tommy xiao <xi...@gmail.com>.

don't understand what reason for executors need communication with agent?


2016-12-19 19:54 GMT+08:00 pangbingqiang <pa...@huawei.com>:

> Hi all:
>
>    What’s the latest information about MESOS-6240
> https://issues.apache.org/jira/browse/MESOS-6240 ,have any demo or design
> achieve?
>
> I see libprocess have support domain socket communication, does agent and
> executor have support communication by domain socket too?
>
> If have any related imformation, please let me know, thanks~.
>
>
>
> [image: cid:image001.png@01D0E8C5.8D08F440]
>
>
>
> Bingqiang Pang(庞兵强)
>
>
>
> Distributed and Parallel Software Lab
>
> Huawei Technologies Co., Ltd.
>
> Email:pangbingqiang@huawei.com <su...@huawei.com>
>
>
>
>
>



-- 
Deshi Xiao
Twitter: xds2000
E-mail: xiaods(AT)gmail.com

Re: [MESOS-6240] Allow executor/agent communication over non-TCP/IP stream socket.

Posted by Greg Mann <gr...@mesosphere.io>.

Hi Bingqiang,
I've had some recent discussions with folks about this feature; it's
something we are interested in doing but I'm not sure what its priority is
in relation to other tickets.

While the AF_UNIX address type has been added to libprocess, libprocess
does not currently accept domain socket connections. Domain sockets are
used by the I/O switchboard in the Mesos agent to communicate with running
containers, but this is mostly application-level code. Libprocess currently
exposes a single server-side TCP socket [1] for incoming connections. To
support executor communication via domain sockets, we would need to add a
new server-side domain socket to libprocess.

Are there any particular goals or use cases you have in mind for this
feature?

Cheers,
Greg

[1]
https://github.com/apache/mesos/blob/be127e6eca1312bf8c2b039646f6909fa42cd342/3rdparty/libprocess/src/process.cpp#L571

On Mon, Dec 19, 2016 at 3:54 AM, pangbingqiang <pa...@huawei.com>
wrote:

> Hi all:
>
>    What’s the latest information about MESOS-6240
> https://issues.apache.org/jira/browse/MESOS-6240 ,have any demo or design
> achieve?
>
> I see libprocess have support domain socket communication, does agent and
> executor have support communication by domain socket too?
>
> If have any related imformation, please let me know, thanks~.
>
>
>
> [image: cid:image001.png@01D0E8C5.8D08F440]
>
>
>
> Bingqiang Pang(庞兵强)
>
>
>
> Distributed and Parallel Software Lab
>
> Huawei Technologies Co., Ltd.
>
> Email:pangbingqiang@huawei.com <su...@huawei.com>
>
>
>
>
>