You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@roller.apache.org by Dave Johnson <da...@rollerweblogger.org> on 2005/10/25 23:40:11 UTC
State of spam prevention in Roller 2.0
I've started a proposal for improving spam prevention measures in
Roller. I've posted the first part of the proposal on the Roller wiki,
which covers the state of spam prevention in Roller (as of Roller 2.0,
which is just about ready for release). Later this week, I'll propose
specific features and bug fixes to address Roller's limitations in this
area.
http://rollerweblogger.org/wiki/Wiki.jsp?page=Proposal_SpamPrevention
Comments, suggestions and flames are welcome - Dave
Re: State of spam prevention in Roller 2.0
Posted by Allen Gilliland <Al...@Sun.COM>.
Of course it is, but like I said, you can get that information from webserver logs. Why spend valuable processing time managing that data and saving it into the database on each request? Why not let that data be extracted by a log processing application?
-- Allen
On Wed, 2005-10-26 at 12:13, Lance Lavandowska wrote:
> It's fun to see *from where* people are coming to your log.
>
> On 10/26/05, Allen Gilliland <Al...@sun.com> wrote:
>
> > I can definitely see the value in counting referers so that users have some sense of how many hits their blog is seeing, but I guess I just have never understood the reason for tracking the rest of the referer information.
> >
Re: State of spam prevention in Roller 2.0
Posted by Lance Lavandowska <la...@gmail.com>.
It's fun to see *from where* people are coming to your log.
On 10/26/05, Allen Gilliland <Al...@sun.com> wrote:
> I can definitely see the value in counting referers so that users have some sense of how many hits their blog is seeing, but I guess I just have never understood the reason for tracking the rest of the referer information.
>
Re: State of spam prevention in Roller 2.0
Posted by Matt Raible <mr...@gmail.com>.
On 10/26/05, Elias Torres <el...@torrez.us> wrote:
> I understand how useful the function is, but I think that the cost is
> too high. I have not checked this yet, but are we doing an insert on
> the database for every page hit? I think this might not be best for
> performance and we might disable it on our internal deployment.
> Nothing like a good log analysis tool that parses Apache log files.
As long as it's performant and doesn't have memory leaks, I don't see
a problem with this. I think it's useful for many developers wanting
to install and try out Roller. It's one of those features you really
enjoy out of the box. I agree that corporate installations will
probably use a different strategy - but I think we should develop for
everyone (especially developers that want to blog), not just
corporations. ;-)
Matt
> :-)
>
> Regards,
>
> Elias
>
> On 10/26/05, Dave Johnson <da...@rollerweblogger.org> wrote:
> >
> > On Oct 26, 2005, at 2:28 PM, Allen Gilliland wrote:
> > > I'm not sure I full understand why we make referers available in the
> > > first place. What is the value in tracking what the referer url is
> > > inside of the application? Wouldn't curious site admins just look
> > > that stuff up from webserver logs?
> >
> > Individual bloggers like to know who is linking to them. Even with blog
> > search sites like Technorati that tell me who is linking to my site, I
> > still look at my referrers at least once a day.
> >
> > - Dave
> >
> >
>
Re: POLL: Log Readers? Analyzers?
Posted by Mihir Sambhus <mi...@sambhus.com>.
I have been using awstats and am very happy with the statistics it provides.
-mihir
On 10/26/05, Matt Raible <mr...@gmail.com> wrote:
>
> I use Webalizer, only because it's provided by my ISP by default.
>
> http://raibledesigns.com/stats/
>
> Matt
>
> On 10/26/05, Greg Hamer <ro...@gmail.com> wrote:
> > Quick Poll ...
> >
> > A post on the roller-dev list mentioned:
> > Nothing like a good log analysis tool that parses Apache log files.
> > Are any among you using a program that they would recommend?
> >
> > Thanks,
> >
> > g
> >
>
--
Mihir
http://mihir.desihub.com/weblog
http://weblog.sambhus.com
Re: POLL: Log Readers? Analyzers?
Posted by Matt Raible <mr...@gmail.com>.
I use Webalizer, only because it's provided by my ISP by default.
http://raibledesigns.com/stats/
Matt
On 10/26/05, Greg Hamer <ro...@gmail.com> wrote:
> Quick Poll ...
>
> A post on the roller-dev list mentioned:
> Nothing like a good log analysis tool that parses Apache log files.
> Are any among you using a program that they would recommend?
>
> Thanks,
>
> g
>
Re: POLL: Log Readers? Analyzers?
Posted by Chris Keilitz <ke...@gmail.com>.
I too recommend awstats. http://www.awstats.org/
CK
On 10/26/05, Greg Hamer <ro...@gmail.com> wrote:
> Quick Poll ...
>
> A post on the roller-dev list mentioned:
> Nothing like a good log analysis tool that parses Apache log files.
> Are any among you using a program that they would recommend?
>
> Thanks,
>
> g
>
POLL: Log Readers? Analyzers?
Posted by Greg Hamer <ro...@gmail.com>.
Quick Poll ...
A post on the roller-dev list mentioned:
Nothing like a good log analysis tool that parses Apache log files.
Are any among you using a program that they would recommend?
Thanks,
g
Re: State of spam prevention in Roller 2.0
Posted by Elias Torres <el...@torrez.us>.
I understand how useful the function is, but I think that the cost is
too high. I have not checked this yet, but are we doing an insert on
the database for every page hit? I think this might not be best for
performance and we might disable it on our internal deployment.
Nothing like a good log analysis tool that parses Apache log files.
:-)
Regards,
Elias
On 10/26/05, Dave Johnson <da...@rollerweblogger.org> wrote:
>
> On Oct 26, 2005, at 2:28 PM, Allen Gilliland wrote:
> > I'm not sure I full understand why we make referers available in the
> > first place. What is the value in tracking what the referer url is
> > inside of the application? Wouldn't curious site admins just look
> > that stuff up from webserver logs?
>
> Individual bloggers like to know who is linking to them. Even with blog
> search sites like Technorati that tell me who is linking to my site, I
> still look at my referrers at least once a day.
>
> - Dave
>
>
Re: State of spam prevention in Roller 2.0
Posted by Dave Johnson <da...@rollerweblogger.org>.
On Oct 26, 2005, at 2:28 PM, Allen Gilliland wrote:
> I'm not sure I full understand why we make referers available in the
> first place. What is the value in tracking what the referer url is
> inside of the application? Wouldn't curious site admins just look
> that stuff up from webserver logs?
Individual bloggers like to know who is linking to them. Even with blog
search sites like Technorati that tell me who is linking to my site, I
still look at my referrers at least once a day.
- Dave
Re: State of spam prevention in Roller 2.0
Posted by Allen Gilliland <Al...@Sun.COM>.
I'm not sure I full understand why we make referers available in the first place. What is the value in tracking what the referer url is inside of the application? Wouldn't curious site admins just look that stuff up from webserver logs?
I can definitely see the value in counting referers so that users have some sense of how many hits their blog is seeing, but I guess I just have never understood the reason for tracking the rest of the referer information.
In any case, I agree with Matthew that the ability to simplify/limit what happens with referers seems like a very good idea.
-- Allen
On Wed, 2005-10-26 at 04:54, Matthew P. Schmidt wrote:
> Our biggest issue at JRoller has been referrer spam. It comes from all
> over the place and even with a healthy list of dirty words, we still
> have to clean out on average at least 5000 bad referrers before 9am.
> One way to limit some of this would be to not allow a page call Referers
> or Referrers as a lot of people seem to be hitting that page
> specifically. The other way to limit it may be to see if a particular
> user or page has been requested too many times with the same or similar
> referrer or IP and block the requesting IP for a certain amount of time.
>
> -Matt
>
> Brian Blakeley wrote:
>
> >Hi Everyone!
> >
> >This issue is of urgent importance to me. Che Blogs is being flooded
> >with trackback, referer and comment spam without mercy for the past few
> >months.
> >
> >The situation is totally out of hand and I have tried everything that I
> >can think of (with suggestions from Dave and others.).
> >
> >I have been cleaning the database directly, disabled the trackback
> >server, removed the referer list from my blog (and asked other users to
> >do the same).
> >
> >Currently I am getting 5,000+ visits a day and have moved over 25G of
> >traffic this month (that is going to hurt!).
> >
> >So, I have a excellent source of spamming problems for testing out
> >solutions and I can open up the server to a few developers for testing
> >out solutions if that would be helpful.
> >
> >The Good news is that the current Roller 1.0 set up of cheblogs.com can
> >really handle the traffic!!!
> >
> >Let me know if this offer is required and thanks for the awesome work!
> >
> >
> >Brian
> >www.cheblogs.com
> >
> >
> >On Tue, 2005-10-25 at 22:05, Matt Raible wrote:
> >
> >
> >>On 10/25/05, Allen Gilliland <Al...@sun.com> wrote:
> >>
> >>
> >>>good summary.
> >>>
> >>>one thing i think is worth evaluating is what effectiveness are we
> >>>currently having with preventing spam using what we have now? and where
> >>>specifically are we being ineffective?
> >>>
> >>>for example, with one of the comment authenticators enabled, how much
> >>>spam are people really getting? are these authenticators effectively
> >>>preventing comment spam? and is the MT blacklist really helping this
> >>>process? my guess is that the blacklist really isn't helping prevent
> >>>comment spam where one of the authenticators is enabled.
> >>>
> >>>
> >>The Math Authenticator seems to be *very* effective on both
> >>raibledesigns.com and jroller.com/page/raible. I get a spam comment
> >>(that I receive e-mail for) about once a month on raibledesigns.com.
> >>
> >>
> >>
> >>>i would also add to the "current problems" that we don't have any
> >>>authenticators that work on trackbacks.
> >>>
> >>>
> >>I agree, but I believe I saw some code committed that fixed this.
> >>
> >>Matt
> >>
> >>
> >>
> >>>-- Allen
> >>>
> >>>
> >>>On Tue, 2005-10-25 at 14:40, Dave Johnson wrote:
> >>>
> >>>
> >>>>I've started a proposal for improving spam prevention measures in
> >>>>Roller. I've posted the first part of the proposal on the Roller wiki,
> >>>>which covers the state of spam prevention in Roller (as of Roller 2.0,
> >>>>which is just about ready for release). Later this week, I'll propose
> >>>>specific features and bug fixes to address Roller's limitations in this
> >>>>area.
> >>>>
> >>>>http://rollerweblogger.org/wiki/Wiki.jsp?page=Proposal_SpamPrevention
> >>>>
> >>>>Comments, suggestions and flames are welcome - Dave
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >
> >
> >
Re: State of spam prevention in Roller 2.0
Posted by "Matthew P. Schmidt" <ma...@javalobby.org>.
Our biggest issue at JRoller has been referrer spam. It comes from all
over the place and even with a healthy list of dirty words, we still
have to clean out on average at least 5000 bad referrers before 9am.
One way to limit some of this would be to not allow a page call Referers
or Referrers as a lot of people seem to be hitting that page
specifically. The other way to limit it may be to see if a particular
user or page has been requested too many times with the same or similar
referrer or IP and block the requesting IP for a certain amount of time.
-Matt
Brian Blakeley wrote:
>Hi Everyone!
>
>This issue is of urgent importance to me. Che Blogs is being flooded
>with trackback, referer and comment spam without mercy for the past few
>months.
>
>The situation is totally out of hand and I have tried everything that I
>can think of (with suggestions from Dave and others.).
>
>I have been cleaning the database directly, disabled the trackback
>server, removed the referer list from my blog (and asked other users to
>do the same).
>
>Currently I am getting 5,000+ visits a day and have moved over 25G of
>traffic this month (that is going to hurt!).
>
>So, I have a excellent source of spamming problems for testing out
>solutions and I can open up the server to a few developers for testing
>out solutions if that would be helpful.
>
>The Good news is that the current Roller 1.0 set up of cheblogs.com can
>really handle the traffic!!!
>
>Let me know if this offer is required and thanks for the awesome work!
>
>
>Brian
>www.cheblogs.com
>
>
>On Tue, 2005-10-25 at 22:05, Matt Raible wrote:
>
>
>>On 10/25/05, Allen Gilliland <Al...@sun.com> wrote:
>>
>>
>>>good summary.
>>>
>>>one thing i think is worth evaluating is what effectiveness are we
>>>currently having with preventing spam using what we have now? and where
>>>specifically are we being ineffective?
>>>
>>>for example, with one of the comment authenticators enabled, how much
>>>spam are people really getting? are these authenticators effectively
>>>preventing comment spam? and is the MT blacklist really helping this
>>>process? my guess is that the blacklist really isn't helping prevent
>>>comment spam where one of the authenticators is enabled.
>>>
>>>
>>The Math Authenticator seems to be *very* effective on both
>>raibledesigns.com and jroller.com/page/raible. I get a spam comment
>>(that I receive e-mail for) about once a month on raibledesigns.com.
>>
>>
>>
>>>i would also add to the "current problems" that we don't have any
>>>authenticators that work on trackbacks.
>>>
>>>
>>I agree, but I believe I saw some code committed that fixed this.
>>
>>Matt
>>
>>
>>
>>>-- Allen
>>>
>>>
>>>On Tue, 2005-10-25 at 14:40, Dave Johnson wrote:
>>>
>>>
>>>>I've started a proposal for improving spam prevention measures in
>>>>Roller. I've posted the first part of the proposal on the Roller wiki,
>>>>which covers the state of spam prevention in Roller (as of Roller 2.0,
>>>>which is just about ready for release). Later this week, I'll propose
>>>>specific features and bug fixes to address Roller's limitations in this
>>>>area.
>>>>
>>>>http://rollerweblogger.org/wiki/Wiki.jsp?page=Proposal_SpamPrevention
>>>>
>>>>Comments, suggestions and flames are welcome - Dave
>>>>
>>>>
>>>>
>>>
>>>
>
>
>
Re: State of spam prevention in Roller 2.0
Posted by Brian Blakeley <we...@labourunions.com>.
Hi Everyone!
This issue is of urgent importance to me. Che Blogs is being flooded
with trackback, referer and comment spam without mercy for the past few
months.
The situation is totally out of hand and I have tried everything that I
can think of (with suggestions from Dave and others.).
I have been cleaning the database directly, disabled the trackback
server, removed the referer list from my blog (and asked other users to
do the same).
Currently I am getting 5,000+ visits a day and have moved over 25G of
traffic this month (that is going to hurt!).
So, I have a excellent source of spamming problems for testing out
solutions and I can open up the server to a few developers for testing
out solutions if that would be helpful.
The Good news is that the current Roller 1.0 set up of cheblogs.com can
really handle the traffic!!!
Let me know if this offer is required and thanks for the awesome work!
Brian
www.cheblogs.com
On Tue, 2005-10-25 at 22:05, Matt Raible wrote:
> On 10/25/05, Allen Gilliland <Al...@sun.com> wrote:
> > good summary.
> >
> > one thing i think is worth evaluating is what effectiveness are we
> > currently having with preventing spam using what we have now? and where
> > specifically are we being ineffective?
> >
> > for example, with one of the comment authenticators enabled, how much
> > spam are people really getting? are these authenticators effectively
> > preventing comment spam? and is the MT blacklist really helping this
> > process? my guess is that the blacklist really isn't helping prevent
> > comment spam where one of the authenticators is enabled.
>
> The Math Authenticator seems to be *very* effective on both
> raibledesigns.com and jroller.com/page/raible. I get a spam comment
> (that I receive e-mail for) about once a month on raibledesigns.com.
>
> >
> > i would also add to the "current problems" that we don't have any
> > authenticators that work on trackbacks.
>
> I agree, but I believe I saw some code committed that fixed this.
>
> Matt
>
> >
> > -- Allen
> >
> >
> > On Tue, 2005-10-25 at 14:40, Dave Johnson wrote:
> > > I've started a proposal for improving spam prevention measures in
> > > Roller. I've posted the first part of the proposal on the Roller wiki,
> > > which covers the state of spam prevention in Roller (as of Roller 2.0,
> > > which is just about ready for release). Later this week, I'll propose
> > > specific features and bug fixes to address Roller's limitations in this
> > > area.
> > >
> > > http://rollerweblogger.org/wiki/Wiki.jsp?page=Proposal_SpamPrevention
> > >
> > > Comments, suggestions and flames are welcome - Dave
> > >
> >
> >
Re: State of spam prevention in Roller 2.0
Posted by Matt Raible <mr...@gmail.com>.
On 10/25/05, Allen Gilliland <Al...@sun.com> wrote:
> good summary.
>
> one thing i think is worth evaluating is what effectiveness are we
> currently having with preventing spam using what we have now? and where
> specifically are we being ineffective?
>
> for example, with one of the comment authenticators enabled, how much
> spam are people really getting? are these authenticators effectively
> preventing comment spam? and is the MT blacklist really helping this
> process? my guess is that the blacklist really isn't helping prevent
> comment spam where one of the authenticators is enabled.
The Math Authenticator seems to be *very* effective on both
raibledesigns.com and jroller.com/page/raible. I get a spam comment
(that I receive e-mail for) about once a month on raibledesigns.com.
>
> i would also add to the "current problems" that we don't have any
> authenticators that work on trackbacks.
I agree, but I believe I saw some code committed that fixed this.
Matt
>
> -- Allen
>
>
> On Tue, 2005-10-25 at 14:40, Dave Johnson wrote:
> > I've started a proposal for improving spam prevention measures in
> > Roller. I've posted the first part of the proposal on the Roller wiki,
> > which covers the state of spam prevention in Roller (as of Roller 2.0,
> > which is just about ready for release). Later this week, I'll propose
> > specific features and bug fixes to address Roller's limitations in this
> > area.
> >
> > http://rollerweblogger.org/wiki/Wiki.jsp?page=Proposal_SpamPrevention
> >
> > Comments, suggestions and flames are welcome - Dave
> >
>
>
Re: State of spam prevention in Roller 2.0
Posted by Allen Gilliland <Al...@Sun.COM>.
good summary.
one thing i think is worth evaluating is what effectiveness are we
currently having with preventing spam using what we have now? and where
specifically are we being ineffective?
for example, with one of the comment authenticators enabled, how much
spam are people really getting? are these authenticators effectively
preventing comment spam? and is the MT blacklist really helping this
process? my guess is that the blacklist really isn't helping prevent
comment spam where one of the authenticators is enabled.
i would also add to the "current problems" that we don't have any
authenticators that work on trackbacks.
-- Allen
On Tue, 2005-10-25 at 14:40, Dave Johnson wrote:
> I've started a proposal for improving spam prevention measures in
> Roller. I've posted the first part of the proposal on the Roller wiki,
> which covers the state of spam prevention in Roller (as of Roller 2.0,
> which is just about ready for release). Later this week, I'll propose
> specific features and bug fixes to address Roller's limitations in this
> area.
>
> http://rollerweblogger.org/wiki/Wiki.jsp?page=Proposal_SpamPrevention
>
> Comments, suggestions and flames are welcome - Dave
>
Re: State of spam prevention in Roller 2.0
Posted by Allen Gilliland <Al...@Sun.COM>.
good summary.
one thing i think is worth evaluating is what effectiveness are we
currently having with preventing spam using what we have now? and where
specifically are we being ineffective?
for example, with one of the comment authenticators enabled, how much
spam are people really getting? are these authenticators effectively
preventing comment spam? and is the MT blacklist really helping this
process? my guess is that the blacklist really isn't helping prevent
comment spam where one of the authenticators is enabled.
i would also add to the "current problems" that we don't have any
authenticators that work on trackbacks.
-- Allen
On Tue, 2005-10-25 at 14:40, Dave Johnson wrote:
> I've started a proposal for improving spam prevention measures in
> Roller. I've posted the first part of the proposal on the Roller wiki,
> which covers the state of spam prevention in Roller (as of Roller 2.0,
> which is just about ready for release). Later this week, I'll propose
> specific features and bug fixes to address Roller's limitations in this
> area.
>
> http://rollerweblogger.org/wiki/Wiki.jsp?page=Proposal_SpamPrevention
>
> Comments, suggestions and flames are welcome - Dave
>