Queries Regarding GSoC Project

[Saahil Sirowa <cs...@iith.ac.in>]

Hi Kevin and SpamAssassin Dev-Community,

I have some queries regarding GSoC Project "SpamAssassin Statistical
Classifier Plugin". They are as follows:
1) What version control software will I be using for the project.
2) An advice about the build and testing mechanisms that can be used.
3) What am I supposed to write under the "Deliverables" header.


Temporary Draft of my GSoC Proposal
GSoC 2018 Proposal
<https://docs.google.com/document/d/1-OCNv79sHvVViKwnrRYtlMiKWLCzz4xUW4tNOlmaTmw/edit?usp=sharing>

I need some help with the following headers:
1) Testing Methodology
2) TImeline
3) Deliverables
4) Testing and Build Methodology

Please rigorously review it and suggest appropriate changes.

Thanks...
Saahil Sirowa
B. Tech Computer Science and Engineering
Indian Institute of Technology Hyderabad

Re: Queries Regarding GSoC Project

["Kevin A. McGrail" <km...@apache.org>]

 On Sat, Mar 17, 2018 at 6:16 PM, Saahil Sirowa <cs...@iith.ac.in>
wrote:

> Hi Kevin and SpamAssassin Dev-Community,
>
> I have some queries regarding GSoC Project "SpamAssassin Statistical
> Classifier Plugin". They are as follows:
> 1) What version control software will I be using for the project.
>

I believe I addressed your other questions in an email I just sent.  Re:
#1, the Apache SpamAssassin project uses SubVersioN and while I think
moving to GIT is a good thought, it's unlikely to be done in a timeframe
that affects your GSoC proposal.

Regards,
KAM

--
Kevin A. McGrail
Asst. Treasurer & VP Fundraising, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
https://www.linkedin.com/in/kmcgrail - 703.798.0171

>
>

Re: Queries Regarding GSoC Project

[RW <rw...@googlemail.com>]

On Sun, 18 Mar 2018 03:46:58 +0530
Saahil Sirowa wrote:

> Temporary Draft of my GSoC Proposal
> GSoC 2018 Proposal
> <https://docs.google.com/document/d/1-OCNv79sHvVViKwnrRYtlMiKWLCzz4xUW4tNOlmaTmw/edit?usp=sharing>

A few points.


You're placing too much emphasis on the lack of statistical
independence in tokens. This is at most a minor problem and Paul Graham
alluded to some theoretical work that suggests it might actually be
beneficial - I've not seen it though.  The assumption of independence
can be removed by switching to the chi-squared method which SA did
many years ago. It's still called Bayes, but it's not Bayesian anymore.

 
You say that [Naive] Bayes doesn't scale well, but it actually does. The
important thing is how classification and training scale with the number
of tokens in the database and that can be O(1) with the right database
back-end.


Also note that Bayes only needs access to an email at the time of
[auto]training. It doesn't require the admin to maintain corpora of spam
and ham. Any statistical filter that needs that will likely be less
widely used than one that doesn't.