You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@logging.apache.org by Matt Sicker <bo...@gmail.com> on 2019/12/02 15:48:17 UTC
[Meta] Article about syslog security limitations
https://www.myname.website/whats-wrong-with-syslog-a-lot-actually
This article seems fairly interesting. I know that we support TLS for
syslog, though I didn't know that wasn't standard (or is that specific
to RFC 5424?). In fact, I'm not even sure how much of his criticism
apply to the updated standard and which are specific to the original
syslog format.
--
Matt Sicker <bo...@gmail.com>
Re: [Meta] Article about syslog security limitations
Posted by Ralph Goers <ra...@dslextreme.com>.
> On Dec 2, 2019, at 12:35 PM, Matt Sicker <bo...@gmail.com> wrote:
>
> On Mon, 2 Dec 2019 at 13:00, Ralph Goers <ra...@dslextreme.com> wrote:
>>> It's hard to find the right balance between not enough and too much coffee
>>> ;-)
>>>
>>
>> For me that balance is easy - I hate coffee!
>>
>> Ralph
>>
>>
>
> Yet you program in Java :trollface:
>
Yup. Although I have never visited I have been told that Java is a very beautiful and relaxing place. That sounds just like the language I code in.
Ralph
Re: [Meta] Article about syslog security limitations
Posted by Matt Sicker <bo...@gmail.com>.
On Mon, 2 Dec 2019 at 13:00, Ralph Goers <ra...@dslextreme.com> wrote:
> > It's hard to find the right balance between not enough and too much coffee
> > ;-)
> >
>
> For me that balance is easy - I hate coffee!
>
> Ralph
>
>
Yet you program in Java :trollface:
--
Matt Sicker <bo...@gmail.com>
Re: [Meta] Article about syslog security limitations
Posted by Ralph Goers <ra...@dslextreme.com>.
> On Dec 2, 2019, at 10:41 AM, Gary Gregory <ga...@gmail.com> wrote:
>
> On Mon, Dec 2, 2019 at 11:34 AM Ralph Goers <ra...@dslextreme.com>
> wrote:
>
>>
>>
>>> On Dec 2, 2019, at 9:25 AM, Gary Gregory <ga...@gmail.com> wrote:
>>>
>>> On Mon, Dec 2, 2019 at 11:24 AM Ralph Goers <ra...@dslextreme.com>
>>> wrote:
>>>
>>>> All of this would apply to the original BSD syslog format. RFC 5424
>> solves
>>>> most of his problems - presuming the SIEM supports it. Nowadays I am
>>>> seeing more JSON logging. The latest updates to “Logging in the Cloud”
>> in
>>>> 2.13.0 will recommend using GELF.
>>>>
>>>
>>> When you do you we can get a 2.13.0?
>>>
>>
>> That is almost Engilish ;-) See my other thread for what I think is the
>> answer to the question you almost asked.
>>
>
> It's hard to find the right balance between not enough and too much coffee
> ;-)
>
For me that balance is easy - I hate coffee!
Ralph
Re: [Meta] Article about syslog security limitations
Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Dec 2, 2019 at 11:34 AM Ralph Goers <ra...@dslextreme.com>
wrote:
>
>
> > On Dec 2, 2019, at 9:25 AM, Gary Gregory <ga...@gmail.com> wrote:
> >
> > On Mon, Dec 2, 2019 at 11:24 AM Ralph Goers <ra...@dslextreme.com>
> > wrote:
> >
> >> All of this would apply to the original BSD syslog format. RFC 5424
> solves
> >> most of his problems - presuming the SIEM supports it. Nowadays I am
> >> seeing more JSON logging. The latest updates to “Logging in the Cloud”
> in
> >> 2.13.0 will recommend using GELF.
> >>
> >
> > When you do you we can get a 2.13.0?
> >
>
> That is almost Engilish ;-) See my other thread for what I think is the
> answer to the question you almost asked.
>
It's hard to find the right balance between not enough and too much coffee
;-)
Gary
>
> Ralph
>
>
>
Re: [Meta] Article about syslog security limitations
Posted by Ralph Goers <ra...@dslextreme.com>.
> On Dec 2, 2019, at 9:25 AM, Gary Gregory <ga...@gmail.com> wrote:
>
> On Mon, Dec 2, 2019 at 11:24 AM Ralph Goers <ra...@dslextreme.com>
> wrote:
>
>> All of this would apply to the original BSD syslog format. RFC 5424 solves
>> most of his problems - presuming the SIEM supports it. Nowadays I am
>> seeing more JSON logging. The latest updates to “Logging in the Cloud” in
>> 2.13.0 will recommend using GELF.
>>
>
> When you do you we can get a 2.13.0?
>
That is almost Engilish ;-) See my other thread for what I think is the answer to the question you almost asked.
Ralph
Re: [Meta] Article about syslog security limitations
Posted by Gary Gregory <ga...@gmail.com>.
On Mon, Dec 2, 2019 at 11:24 AM Ralph Goers <ra...@dslextreme.com>
wrote:
> All of this would apply to the original BSD syslog format. RFC 5424 solves
> most of his problems - presuming the SIEM supports it. Nowadays I am
> seeing more JSON logging. The latest updates to “Logging in the Cloud” in
> 2.13.0 will recommend using GELF.
>
When you do you we can get a 2.13.0?
Gary
>
> Ralph
>
> > On Dec 2, 2019, at 9:02 AM, Gary Gregory <ga...@gmail.com> wrote:
> >
> > This is what it says for me:
> > "
> > What's wrong with syslog? A lot, actually.
> >
> > I recently had a post
> > <https://www.myname.website/im-not-burned-out-im-pissed-off> make it to
> Hacker
> > News <https://news.ycombinator.com/item?id=21645114>, lobste.rs
> > <https://lobste.rs/s/a3hunb/i_m_not_burned_out_i_m_pissed_off>,
> Pinboard,
> > and reddit
> > <
> https://www.reddit.com/r/hackernews/comments/e2dkoe/im_not_burned_out_im_pissed_off/
> >.
> > This was not my doing, I have no idea who submitted the article.
> Honestly I
> > was a bit surprised that anyone even knows this blog exists. I write this
> > as a therapy session for myself when I need to rant but no one needs to
> > hear it. If I knew I was writing for an audience, I would have written a
> > *lot* differently.
> >
> > The biggest thing I noticed from reading the comments especially on HN
> was
> > people who don't understand what I'm talking about and think I'm just
> > ranting because I hate everyone and everything. That may be true, but
> > that's not exactly the point of the article. I can easily forgive the
> > misunderstanding because, again, I wasn't writing for an audience I was
> > writing for myself. So let me take the opportunity to clear some things
> up.
> >
> > This misconception was repeated a few times in various comments: “*syslog
> > is a standard and if you don't like it you just don't like your job*”. So
> > let me explain, from a security standpoint, what's wrong with syslog.
> > Syslog for security logging
> >
> > Let's start by explaining what syslog is and what it does in a security
> > context. The product I work with is a SIEM, which basically collect logs
> > from around your environment and combines them together to find security
> > problems. So we'll collect logs from firewalls, domain controllers, IDPS
> > systems, Linux servers, AV systems, Tripwire, everything. Everything we
> can
> > get. The majority of these send their logs in syslog format. The logs we
> > get aren't necessarily sensitive on their own, but they do provide a lot
> of
> > contextual data that attackers would love. If you want to see alerts
> around
> > one account failing to authenticate, you necessarily need to send that
> > account's username, as well as the source and destination of the traffic.
> > Now any attacker who sees that log knows the address of your domain
> > controller, a valid username, and what IP address they're connecting
> from.
> > This is... problematic from a security standpoint.
> >
> > Now mainly my biggest beef is security vendors who want to ship syslog
> over
> > the Internet. I've only encountered it a handful of times, but once is
> bad
> > enough. But that's just an add-on to how outdated syslog is, especially
> for
> > security logging. Syslog is a *clear text protocol*. There are methods of
> > encrypting syslog traffic, but it's not standardized and it's up to the
> > vendor to support it. Most don't. The best way is using a VPN tunnel, but
> > then you have the overhead of a VPN tunnel and another point of failure.
> > Again, we're talking security logs. If your logging goes down, so does
> your
> > security.
> >
> > And if you don't want the overhead of syslog over TLS or stunnel or
> > OpenVPN, that means you're sending clear text *security logs* over the
> > wire. On your internal network that's bad enough, but *over the
> Internet*?!
> > You'd have to be insane (or work for Cisco).
> > No standard formatting
> >
> > So that's the security beef with syslog over the Internet. Let's move on
> to
> > all the other reasons syslog is terrible. The next one is the lack of a
> > logging standard. IBM's product uses LEEF format, that looks like this:
> >
> > LEEF:2.0|Trend Micro|Deep Security Manager||192|cat=System name=Alert
> Ended
> > desc=Alert: CPU Warning Threshold Exceeded\nSubject:
> > 10.201.114.164\nSeverity: Warning sev=3 src=10.201.114.164 usrName=System
> > msg=Alert: CPU Warning Threshold Exceeded\nSubject:
> > 10.201.114.164\nSeverity: Warning TrendMicroDsTenant=Primary
> >
> > HP/MicroFocus's product uses CEF, which looks like this:
> >
> > CEF:0|Trend Micro|Deep Security Manager||600|User Signed
> > In|3|src=10.52.116.160 suser=admin target=admin msg=User signed in from
> > 2001:db8::5
> >
> > Linux syslog generates unstructured garbage which looks like this:
> >
> > Failed password for root from 192.168.50.65 port 34780 ssh2
> >
> > Cisco's IOS products generate logs like this:
> >
> > 00:00:48: %LINEPROTO-5-UPDOWN: Line protocol on Interface
> > GigabitEthernet0/1, changed state to down 2
> >
> > So now your syslog receiver (the SIEM in this case) has to parse these
> > multitude of logging formats. CEF and LEEF are nice because it's at least
> > *some* structure to key your searches/regex on. Other syslog formats...
> not
> > so much. And in the world of security, parsing a message wrong means
> > potentially labeling the victim as the source of the attack and the
> > attacker as the destination. In a real SOC, this would be the difference
> > between an external hacker breaching a machine and getting a foothold in
> > your network versus malware on one of your workstations talking to a
> > command and control server. That is a *massive* difference and working
> > under the wrong assumption could cost hours or days of investigation.
> > Meanwhile the hacker is still crawling around your network.
> > Unreliable for security logs
> >
> > So now that we know syslog is insecure and difficult to parse, let's move
> > on to its reliability. Syslog has no function built in to retry a
> message.
> > If your network goes down or your syslog receiver (your SIEM) goes down,
> > those messages are lost. The sender has no idea that no one is listening.
> > Sure you can use TCP
> > <https://bitbucket.org/blog/follow-up-on-our-downtime-last-week> but...
> I
> > wouldn't recommend it.
> >
> > So in the real world, this means if you're under a DoS attack and your
> > syslog receiver/SIEM goes over license, or over capacity of the hardware,
> > or the network link drops, or the attacker changes your firewall rules or
> > any number of ways the link between the sender and receiver can be
> severed
> > during an active attack... all those logs are lost. Gone forever. You
> will
> > never get them back. Which means once the dust has settled, chances are
> you
> > still have no idea where the attack came from, what happened, or if the
> > attacker is still inside your network. Because your devices were
> dutifully
> > sending syslog blindly assuming the receiver was listening, and the
> system
> > has no idea those logs were lost. Unless you're using TCP, in which case
> > when the link gets severed for whatever reason, your entire network
> crashes.
> > What would a robust logging solution look like?
> >
> > There's plenty of robust logging solutions. Sure lots of them have their
> > own issues, but let's take it one step at a time. JDBC is a good one.
> It's
> > a pull not a push, so as long as your SIEM can remember where it left
> off,
> > it will always get the missing logs after the dust settles. Many Windows
> > agents are good at this too (many are not... YMMV). Properly written API
> > clients can do this as well, remembering where they left off and resuming
> > when the network connection drops. With the added benefit of, if you're
> > using HTTPS you're not sending clear text logs across the network or
> > Internet.
> >
> > So, for the love of god, stop using syslog for security logging."
> >
> > On Mon, Dec 2, 2019 at 11:01 AM Ralph Goers <ra...@dslextreme.com>
> > wrote:
> >
> >> I am unable to view that web site as it has certificate problems.
> >>
> >> Ralph
> >>
> >>> On Dec 2, 2019, at 8:48 AM, Matt Sicker <bo...@gmail.com> wrote:
> >>>
> >>> https://www.myname.website/whats-wrong-with-syslog-a-lot-actually
> >>>
> >>> This article seems fairly interesting. I know that we support TLS for
> >>> syslog, though I didn't know that wasn't standard (or is that specific
> >>> to RFC 5424?). In fact, I'm not even sure how much of his criticism
> >>> apply to the updated standard and which are specific to the original
> >>> syslog format.
> >>>
> >>> --
> >>> Matt Sicker <bo...@gmail.com>
> >>>
> >>
> >>
> >>
>
>
>
Re: [Meta] Article about syslog security limitations
Posted by Ralph Goers <ra...@dslextreme.com>.
All of this would apply to the original BSD syslog format. RFC 5424 solves most of his problems - presuming the SIEM supports it. Nowadays I am seeing more JSON logging. The latest updates to “Logging in the Cloud” in 2.13.0 will recommend using GELF.
Ralph
> On Dec 2, 2019, at 9:02 AM, Gary Gregory <ga...@gmail.com> wrote:
>
> This is what it says for me:
> "
> What's wrong with syslog? A lot, actually.
>
> I recently had a post
> <https://www.myname.website/im-not-burned-out-im-pissed-off> make it to Hacker
> News <https://news.ycombinator.com/item?id=21645114>, lobste.rs
> <https://lobste.rs/s/a3hunb/i_m_not_burned_out_i_m_pissed_off>, Pinboard,
> and reddit
> <https://www.reddit.com/r/hackernews/comments/e2dkoe/im_not_burned_out_im_pissed_off/>.
> This was not my doing, I have no idea who submitted the article. Honestly I
> was a bit surprised that anyone even knows this blog exists. I write this
> as a therapy session for myself when I need to rant but no one needs to
> hear it. If I knew I was writing for an audience, I would have written a
> *lot* differently.
>
> The biggest thing I noticed from reading the comments especially on HN was
> people who don't understand what I'm talking about and think I'm just
> ranting because I hate everyone and everything. That may be true, but
> that's not exactly the point of the article. I can easily forgive the
> misunderstanding because, again, I wasn't writing for an audience I was
> writing for myself. So let me take the opportunity to clear some things up.
>
> This misconception was repeated a few times in various comments: “*syslog
> is a standard and if you don't like it you just don't like your job*”. So
> let me explain, from a security standpoint, what's wrong with syslog.
> Syslog for security logging
>
> Let's start by explaining what syslog is and what it does in a security
> context. The product I work with is a SIEM, which basically collect logs
> from around your environment and combines them together to find security
> problems. So we'll collect logs from firewalls, domain controllers, IDPS
> systems, Linux servers, AV systems, Tripwire, everything. Everything we can
> get. The majority of these send their logs in syslog format. The logs we
> get aren't necessarily sensitive on their own, but they do provide a lot of
> contextual data that attackers would love. If you want to see alerts around
> one account failing to authenticate, you necessarily need to send that
> account's username, as well as the source and destination of the traffic.
> Now any attacker who sees that log knows the address of your domain
> controller, a valid username, and what IP address they're connecting from.
> This is... problematic from a security standpoint.
>
> Now mainly my biggest beef is security vendors who want to ship syslog over
> the Internet. I've only encountered it a handful of times, but once is bad
> enough. But that's just an add-on to how outdated syslog is, especially for
> security logging. Syslog is a *clear text protocol*. There are methods of
> encrypting syslog traffic, but it's not standardized and it's up to the
> vendor to support it. Most don't. The best way is using a VPN tunnel, but
> then you have the overhead of a VPN tunnel and another point of failure.
> Again, we're talking security logs. If your logging goes down, so does your
> security.
>
> And if you don't want the overhead of syslog over TLS or stunnel or
> OpenVPN, that means you're sending clear text *security logs* over the
> wire. On your internal network that's bad enough, but *over the Internet*?!
> You'd have to be insane (or work for Cisco).
> No standard formatting
>
> So that's the security beef with syslog over the Internet. Let's move on to
> all the other reasons syslog is terrible. The next one is the lack of a
> logging standard. IBM's product uses LEEF format, that looks like this:
>
> LEEF:2.0|Trend Micro|Deep Security Manager||192|cat=System name=Alert Ended
> desc=Alert: CPU Warning Threshold Exceeded\nSubject:
> 10.201.114.164\nSeverity: Warning sev=3 src=10.201.114.164 usrName=System
> msg=Alert: CPU Warning Threshold Exceeded\nSubject:
> 10.201.114.164\nSeverity: Warning TrendMicroDsTenant=Primary
>
> HP/MicroFocus's product uses CEF, which looks like this:
>
> CEF:0|Trend Micro|Deep Security Manager||600|User Signed
> In|3|src=10.52.116.160 suser=admin target=admin msg=User signed in from
> 2001:db8::5
>
> Linux syslog generates unstructured garbage which looks like this:
>
> Failed password for root from 192.168.50.65 port 34780 ssh2
>
> Cisco's IOS products generate logs like this:
>
> 00:00:48: %LINEPROTO-5-UPDOWN: Line protocol on Interface
> GigabitEthernet0/1, changed state to down 2
>
> So now your syslog receiver (the SIEM in this case) has to parse these
> multitude of logging formats. CEF and LEEF are nice because it's at least
> *some* structure to key your searches/regex on. Other syslog formats... not
> so much. And in the world of security, parsing a message wrong means
> potentially labeling the victim as the source of the attack and the
> attacker as the destination. In a real SOC, this would be the difference
> between an external hacker breaching a machine and getting a foothold in
> your network versus malware on one of your workstations talking to a
> command and control server. That is a *massive* difference and working
> under the wrong assumption could cost hours or days of investigation.
> Meanwhile the hacker is still crawling around your network.
> Unreliable for security logs
>
> So now that we know syslog is insecure and difficult to parse, let's move
> on to its reliability. Syslog has no function built in to retry a message.
> If your network goes down or your syslog receiver (your SIEM) goes down,
> those messages are lost. The sender has no idea that no one is listening.
> Sure you can use TCP
> <https://bitbucket.org/blog/follow-up-on-our-downtime-last-week> but... I
> wouldn't recommend it.
>
> So in the real world, this means if you're under a DoS attack and your
> syslog receiver/SIEM goes over license, or over capacity of the hardware,
> or the network link drops, or the attacker changes your firewall rules or
> any number of ways the link between the sender and receiver can be severed
> during an active attack... all those logs are lost. Gone forever. You will
> never get them back. Which means once the dust has settled, chances are you
> still have no idea where the attack came from, what happened, or if the
> attacker is still inside your network. Because your devices were dutifully
> sending syslog blindly assuming the receiver was listening, and the system
> has no idea those logs were lost. Unless you're using TCP, in which case
> when the link gets severed for whatever reason, your entire network crashes.
> What would a robust logging solution look like?
>
> There's plenty of robust logging solutions. Sure lots of them have their
> own issues, but let's take it one step at a time. JDBC is a good one. It's
> a pull not a push, so as long as your SIEM can remember where it left off,
> it will always get the missing logs after the dust settles. Many Windows
> agents are good at this too (many are not... YMMV). Properly written API
> clients can do this as well, remembering where they left off and resuming
> when the network connection drops. With the added benefit of, if you're
> using HTTPS you're not sending clear text logs across the network or
> Internet.
>
> So, for the love of god, stop using syslog for security logging."
>
> On Mon, Dec 2, 2019 at 11:01 AM Ralph Goers <ra...@dslextreme.com>
> wrote:
>
>> I am unable to view that web site as it has certificate problems.
>>
>> Ralph
>>
>>> On Dec 2, 2019, at 8:48 AM, Matt Sicker <bo...@gmail.com> wrote:
>>>
>>> https://www.myname.website/whats-wrong-with-syslog-a-lot-actually
>>>
>>> This article seems fairly interesting. I know that we support TLS for
>>> syslog, though I didn't know that wasn't standard (or is that specific
>>> to RFC 5424?). In fact, I'm not even sure how much of his criticism
>>> apply to the updated standard and which are specific to the original
>>> syslog format.
>>>
>>> --
>>> Matt Sicker <bo...@gmail.com>
>>>
>>
>>
>>
Re: [Meta] Article about syslog security limitations
Posted by Gary Gregory <ga...@gmail.com>.
This is what it says for me:
"
What's wrong with syslog? A lot, actually.
I recently had a post
<https://www.myname.website/im-not-burned-out-im-pissed-off> make it to Hacker
News <https://news.ycombinator.com/item?id=21645114>, lobste.rs
<https://lobste.rs/s/a3hunb/i_m_not_burned_out_i_m_pissed_off>, Pinboard,
and reddit
<https://www.reddit.com/r/hackernews/comments/e2dkoe/im_not_burned_out_im_pissed_off/>.
This was not my doing, I have no idea who submitted the article. Honestly I
was a bit surprised that anyone even knows this blog exists. I write this
as a therapy session for myself when I need to rant but no one needs to
hear it. If I knew I was writing for an audience, I would have written a
*lot* differently.
The biggest thing I noticed from reading the comments especially on HN was
people who don't understand what I'm talking about and think I'm just
ranting because I hate everyone and everything. That may be true, but
that's not exactly the point of the article. I can easily forgive the
misunderstanding because, again, I wasn't writing for an audience I was
writing for myself. So let me take the opportunity to clear some things up.
This misconception was repeated a few times in various comments: “*syslog
is a standard and if you don't like it you just don't like your job*”. So
let me explain, from a security standpoint, what's wrong with syslog.
Syslog for security logging
Let's start by explaining what syslog is and what it does in a security
context. The product I work with is a SIEM, which basically collect logs
from around your environment and combines them together to find security
problems. So we'll collect logs from firewalls, domain controllers, IDPS
systems, Linux servers, AV systems, Tripwire, everything. Everything we can
get. The majority of these send their logs in syslog format. The logs we
get aren't necessarily sensitive on their own, but they do provide a lot of
contextual data that attackers would love. If you want to see alerts around
one account failing to authenticate, you necessarily need to send that
account's username, as well as the source and destination of the traffic.
Now any attacker who sees that log knows the address of your domain
controller, a valid username, and what IP address they're connecting from.
This is... problematic from a security standpoint.
Now mainly my biggest beef is security vendors who want to ship syslog over
the Internet. I've only encountered it a handful of times, but once is bad
enough. But that's just an add-on to how outdated syslog is, especially for
security logging. Syslog is a *clear text protocol*. There are methods of
encrypting syslog traffic, but it's not standardized and it's up to the
vendor to support it. Most don't. The best way is using a VPN tunnel, but
then you have the overhead of a VPN tunnel and another point of failure.
Again, we're talking security logs. If your logging goes down, so does your
security.
And if you don't want the overhead of syslog over TLS or stunnel or
OpenVPN, that means you're sending clear text *security logs* over the
wire. On your internal network that's bad enough, but *over the Internet*?!
You'd have to be insane (or work for Cisco).
No standard formatting
So that's the security beef with syslog over the Internet. Let's move on to
all the other reasons syslog is terrible. The next one is the lack of a
logging standard. IBM's product uses LEEF format, that looks like this:
LEEF:2.0|Trend Micro|Deep Security Manager||192|cat=System name=Alert Ended
desc=Alert: CPU Warning Threshold Exceeded\nSubject:
10.201.114.164\nSeverity: Warning sev=3 src=10.201.114.164 usrName=System
msg=Alert: CPU Warning Threshold Exceeded\nSubject:
10.201.114.164\nSeverity: Warning TrendMicroDsTenant=Primary
HP/MicroFocus's product uses CEF, which looks like this:
CEF:0|Trend Micro|Deep Security Manager||600|User Signed
In|3|src=10.52.116.160 suser=admin target=admin msg=User signed in from
2001:db8::5
Linux syslog generates unstructured garbage which looks like this:
Failed password for root from 192.168.50.65 port 34780 ssh2
Cisco's IOS products generate logs like this:
00:00:48: %LINEPROTO-5-UPDOWN: Line protocol on Interface
GigabitEthernet0/1, changed state to down 2
So now your syslog receiver (the SIEM in this case) has to parse these
multitude of logging formats. CEF and LEEF are nice because it's at least
*some* structure to key your searches/regex on. Other syslog formats... not
so much. And in the world of security, parsing a message wrong means
potentially labeling the victim as the source of the attack and the
attacker as the destination. In a real SOC, this would be the difference
between an external hacker breaching a machine and getting a foothold in
your network versus malware on one of your workstations talking to a
command and control server. That is a *massive* difference and working
under the wrong assumption could cost hours or days of investigation.
Meanwhile the hacker is still crawling around your network.
Unreliable for security logs
So now that we know syslog is insecure and difficult to parse, let's move
on to its reliability. Syslog has no function built in to retry a message.
If your network goes down or your syslog receiver (your SIEM) goes down,
those messages are lost. The sender has no idea that no one is listening.
Sure you can use TCP
<https://bitbucket.org/blog/follow-up-on-our-downtime-last-week> but... I
wouldn't recommend it.
So in the real world, this means if you're under a DoS attack and your
syslog receiver/SIEM goes over license, or over capacity of the hardware,
or the network link drops, or the attacker changes your firewall rules or
any number of ways the link between the sender and receiver can be severed
during an active attack... all those logs are lost. Gone forever. You will
never get them back. Which means once the dust has settled, chances are you
still have no idea where the attack came from, what happened, or if the
attacker is still inside your network. Because your devices were dutifully
sending syslog blindly assuming the receiver was listening, and the system
has no idea those logs were lost. Unless you're using TCP, in which case
when the link gets severed for whatever reason, your entire network crashes.
What would a robust logging solution look like?
There's plenty of robust logging solutions. Sure lots of them have their
own issues, but let's take it one step at a time. JDBC is a good one. It's
a pull not a push, so as long as your SIEM can remember where it left off,
it will always get the missing logs after the dust settles. Many Windows
agents are good at this too (many are not... YMMV). Properly written API
clients can do this as well, remembering where they left off and resuming
when the network connection drops. With the added benefit of, if you're
using HTTPS you're not sending clear text logs across the network or
Internet.
So, for the love of god, stop using syslog for security logging."
On Mon, Dec 2, 2019 at 11:01 AM Ralph Goers <ra...@dslextreme.com>
wrote:
> I am unable to view that web site as it has certificate problems.
>
> Ralph
>
> > On Dec 2, 2019, at 8:48 AM, Matt Sicker <bo...@gmail.com> wrote:
> >
> > https://www.myname.website/whats-wrong-with-syslog-a-lot-actually
> >
> > This article seems fairly interesting. I know that we support TLS for
> > syslog, though I didn't know that wasn't standard (or is that specific
> > to RFC 5424?). In fact, I'm not even sure how much of his criticism
> > apply to the updated standard and which are specific to the original
> > syslog format.
> >
> > --
> > Matt Sicker <bo...@gmail.com>
> >
>
>
>
Re: [Meta] Article about syslog security limitations
Posted by Ralph Goers <ra...@dslextreme.com>.
I am unable to view that web site as it has certificate problems.
Ralph
> On Dec 2, 2019, at 8:48 AM, Matt Sicker <bo...@gmail.com> wrote:
>
> https://www.myname.website/whats-wrong-with-syslog-a-lot-actually
>
> This article seems fairly interesting. I know that we support TLS for
> syslog, though I didn't know that wasn't standard (or is that specific
> to RFC 5424?). In fact, I'm not even sure how much of his criticism
> apply to the updated standard and which are specific to the original
> syslog format.
>
> --
> Matt Sicker <bo...@gmail.com>
>