You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@trafficserver.apache.org by Di Li <di...@apple.com> on 2016/11/09 18:58:47 UTC
question for mapping client to origin servers
Hi,
Questions for how to map client to origin servers, is there any programmable interface that we can expose the connection between client to origin servers.
I know that we have stats over http, but that’s the counters for the system or each ATS instance instead of drill down to each connections, I also see the builtin web interface “{net}” or httpsm, but those are not programmable way.
Only thing that I can see the mapping is from logs , which eventually goes to a log parse or big data platform, then eventually get the mapping.
Is there other ways to achieve this instead of logs method ?
Thanks,
Di Li
Re: question for mapping client to origin servers
Posted by Miles Libbey <ml...@apache.org>.
ATS can log all the requests -- you have a fair amount of
customization available:
https://docs.trafficserver.apache.org/en/latest/admin-guide/logging/formatting.en.html
On Fri, Nov 18, 2016 at 9:43 AM, Di Li <di...@apple.com> wrote:
> I think from the forward proxy perspective, I definitely want to know what
> our clients using our proxy to connect to, I can see the outgoing socket
> where they connect to, and incoming socket who are connecting to us, but
> without the mapping, I don’t who’s going to where, such info can be
> collected to further analysis our clients activities and behavior, and in
> case we see abnormal stuff, we can take more actions based on that.
>
>
> Thanks,
> Di Li
>
>
>
>
> On Nov 17, 2016, at 10:29 PM, James Peach <jp...@apache.org> wrote:
>
>
> On Nov 17, 2016, at 5:49 PM, Di Li <di...@apple.com> wrote:
>
> Like the built-in UI Enable for {net} and {http}, those net and httpsm has
> the information for each of the connection, but they don’t have the client
> IP address associate with them, and they are not really like json format
> that we can parse easier.
>
>
> What are you hoping to do with this information? AFAIK it is not really
> intended for monitoring.
>
>
> So far only the mapping I can see is from Log, is there other place I can
> see the mapping between client to destination.
>
>
> Thanks,
> Di Li
>
>
> On Nov 12, 2016, at 5:15 PM, Bryan Call <bc...@apache.org> wrote:
>
> What do you mean by “expose the connection between the client to the origin
> servers”?
>
> -Bryan
>
>
>
>
> On Nov 10, 2016, at 3:58 AM, Di Li <di...@apple.com> wrote:
>
> Hi,
>
> Questions for how to map client to origin servers, is there any programmable
> interface that we can expose the connection between client to origin
> servers.
>
> I know that we have stats over http, but that’s the counters for the system
> or each ATS instance instead of drill down to each connections, I also see
> the builtin web interface “{net}” or httpsm, but those are not programmable
> way.
>
> Only thing that I can see the mapping is from logs , which eventually goes
> to a log parse or big data platform, then eventually get the mapping.
>
> Is there other ways to achieve this instead of logs method ?
>
>
> Thanks,
> Di Li
>
>
>
>
>
>
Re: question for mapping client to origin servers
Posted by Di Li <di...@apple.com>.
I think from the forward proxy perspective, I definitely want to know what our clients using our proxy to connect to, I can see the outgoing socket where they connect to, and incoming socket who are connecting to us, but without the mapping, I don’t who’s going to where, such info can be collected to further analysis our clients activities and behavior, and in case we see abnormal stuff, we can take more actions based on that.
Thanks,
Di Li
> On Nov 17, 2016, at 10:29 PM, James Peach <jp...@apache.org> wrote:
>
>
>> On Nov 17, 2016, at 5:49 PM, Di Li <di...@apple.com> wrote:
>>
>> Like the built-in UI Enable for {net} and {http}, those net and httpsm has the information for each of the connection, but they don’t have the client IP address associate with them, and they are not really like json format that we can parse easier.
>
> What are you hoping to do with this information? AFAIK it is not really intended for monitoring.
>
>>
>> So far only the mapping I can see is from Log, is there other place I can see the mapping between client to destination.
>>
>>
>> Thanks,
>> Di Li
>>
>>
>>> On Nov 12, 2016, at 5:15 PM, Bryan Call <bc...@apache.org> wrote:
>>>
>>> What do you mean by “expose the connection between the client to the origin servers”?
>>>
>>> -Bryan
>>>
>>>
>>>
>>>
>>>> On Nov 10, 2016, at 3:58 AM, Di Li <di...@apple.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> Questions for how to map client to origin servers, is there any programmable interface that we can expose the connection between client to origin servers.
>>>>
>>>> I know that we have stats over http, but that’s the counters for the system or each ATS instance instead of drill down to each connections, I also see the builtin web interface “{net}” or httpsm, but those are not programmable way.
>>>>
>>>> Only thing that I can see the mapping is from logs , which eventually goes to a log parse or big data platform, then eventually get the mapping.
>>>>
>>>> Is there other ways to achieve this instead of logs method ?
>>>>
>>>>
>>>> Thanks,
>>>> Di Li
>>>>
>>>>
>>>
>>
>
Re: question for mapping client to origin servers
Posted by James Peach <jp...@apache.org>.
> On Nov 17, 2016, at 5:49 PM, Di Li <di...@apple.com> wrote:
>
> Like the built-in UI Enable for {net} and {http}, those net and httpsm has the information for each of the connection, but they don’t have the client IP address associate with them, and they are not really like json format that we can parse easier.
What are you hoping to do with this information? AFAIK it is not really intended for monitoring.
>
> So far only the mapping I can see is from Log, is there other place I can see the mapping between client to destination.
>
>
> Thanks,
> Di Li
>
>
>> On Nov 12, 2016, at 5:15 PM, Bryan Call <bc...@apache.org> wrote:
>>
>> What do you mean by “expose the connection between the client to the origin servers”?
>>
>> -Bryan
>>
>>
>>
>>
>>> On Nov 10, 2016, at 3:58 AM, Di Li <di...@apple.com> wrote:
>>>
>>> Hi,
>>>
>>> Questions for how to map client to origin servers, is there any programmable interface that we can expose the connection between client to origin servers.
>>>
>>> I know that we have stats over http, but that’s the counters for the system or each ATS instance instead of drill down to each connections, I also see the builtin web interface “{net}” or httpsm, but those are not programmable way.
>>>
>>> Only thing that I can see the mapping is from logs , which eventually goes to a log parse or big data platform, then eventually get the mapping.
>>>
>>> Is there other ways to achieve this instead of logs method ?
>>>
>>>
>>> Thanks,
>>> Di Li
>>>
>>>
>>
>
Re: question for mapping client to origin servers
Posted by Di Li <di...@apple.com>.
Like the built-in UI Enable for {net} and {http}, those net and httpsm has the information for each of the connection, but they don’t have the client IP address associate with them, and they are not really like json format that we can parse easier.
So far only the mapping I can see is from Log, is there other place I can see the mapping between client to destination.
Thanks,
Di Li
> On Nov 12, 2016, at 5:15 PM, Bryan Call <bc...@apache.org> wrote:
>
> What do you mean by “expose the connection between the client to the origin servers”?
>
> -Bryan
>
>
>
>
>> On Nov 10, 2016, at 3:58 AM, Di Li <di_li@apple.com <ma...@apple.com>> wrote:
>>
>> Hi,
>>
>> Questions for how to map client to origin servers, is there any programmable interface that we can expose the connection between client to origin servers.
>>
>> I know that we have stats over http, but that’s the counters for the system or each ATS instance instead of drill down to each connections, I also see the builtin web interface “{net}” or httpsm, but those are not programmable way.
>>
>> Only thing that I can see the mapping is from logs , which eventually goes to a log parse or big data platform, then eventually get the mapping.
>>
>> Is there other ways to achieve this instead of logs method ?
>>
>>
>> Thanks,
>> Di Li
>>
>>
>
Re: question for mapping client to origin servers
Posted by Bryan Call <bc...@apache.org>.
What do you mean by “expose the connection between the client to the origin servers”?
-Bryan
> On Nov 10, 2016, at 3:58 AM, Di Li <di...@apple.com> wrote:
>
> Hi,
>
> Questions for how to map client to origin servers, is there any programmable interface that we can expose the connection between client to origin servers.
>
> I know that we have stats over http, but that’s the counters for the system or each ATS instance instead of drill down to each connections, I also see the builtin web interface “{net}” or httpsm, but those are not programmable way.
>
> Only thing that I can see the mapping is from logs , which eventually goes to a log parse or big data platform, then eventually get the mapping.
>
> Is there other ways to achieve this instead of logs method ?
>
>
> Thanks,
> Di Li
>
>