You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by "Stefano Bagnara (JIRA)" <se...@james.apache.org> on 2005/08/03 10:02:48 UTC
[jira] Created: (JAMES-387) Exception in BayesianAnalysis
Exception in BayesianAnalysis
-----------------------------
Key: JAMES-387
URL: http://issues.apache.org/jira/browse/JAMES-387
Project: James
Type: Bug
Components: Matchers/Mailets (bundled)
Versions: 3.0
Environment: James from svn-trunk 2005-08-01.
MySQL 4.0
Reporter: Stefano Bagnara
Priority: Minor
Got this exception for every incoming mail:
02/08/05 00:39:25 INFO James.Mailet: BayesianAnalysis: Exception: java.lang.Integer
java.lang.ClassCastException: java.lang.Integer
at org.apache.james.util.BayesianAnalyzer.getTokenProbabilityStrengths(BayesianAnalyzer.java:591)
at org.apache.james.util.BayesianAnalyzer.computeSpamProbability(BayesianAnalyzer.java:340)
at org.apache.james.transport.mailets.BayesianAnalysis.service(BayesianAnalysis.java:289)
at org.apache.james.transport.LinearProcessor.service(LinearProcessor.java:407)
at org.apache.james.transport.JamesSpoolManager.process(JamesSpoolManager.java:460)
at org.apache.james.transport.JamesSpoolManager.run(JamesSpoolManager.java:369)
at java.lang.Thread.run(Unknown Source)
If I clean my spam/ham db the exceptions disappears but they start again when the spam/ham db become large.
My bayesiananalysis_spam contains 200000 rows.
The following are the spam tokens with higher "occurrences".
+---------------------------+-------------+
| token | occurrences |
+---------------------------+-------------+
| 3D | 82151 |
| a | 59953 |
| the | 45295 |
| FONT | 42771 |
| Content-Type | 39058 |
| to | 36626 |
| com | 32902 |
| http | 32886 |
| of | 32504 |
| font | 31803 |
| and | 31577 |
| Content-Transfer-Encoding | 31576 |
| p | 29746 |
| text | 29482 |
| in | 29418 |
| it | 28498 |
| br | 28037 |
| DIV | 27431 |
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [jira] Created: (JAMES-387) Exception in BayesianAnalysis
Posted by Vincenzo Gianferrari Pini <vi...@praxis.it>.
Danny Angus wrote:
>On 03/08/05, Stefano Bagnara (JIRA) <se...@james.apache.org> wrote:
>
>
>>My bayesiananalysis_spam contains 200000 rows.
>>
>>The following are the spam tokens with higher "occurrences".
>>
>>
>
>At one point I had a different modified version of this code, and
>optimised it to not count things which were either steadily neutral or
>already at one or other extreme.
>
>
It's already like that...
>Of course this makes it more brittle, and won't (I think) cope well
>with things which need to move far, but it did cut down on the size of
>the corpus and seemed to work accpetably for me at least.
>
>d.
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>For additional commands, e-mail: server-dev-help@james.apache.org
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [jira] Created: (JAMES-387) Exception in BayesianAnalysis
Posted by Danny Angus <da...@gmail.com>.
On 03/08/05, Stefano Bagnara (JIRA) <se...@james.apache.org> wrote:
> My bayesiananalysis_spam contains 200000 rows.
>
> The following are the spam tokens with higher "occurrences".
At one point I had a different modified version of this code, and
optimised it to not count things which were either steadily neutral or
already at one or other extreme.
Of course this makes it more brittle, and won't (I think) cope well
with things which need to move far, but it did cut down on the size of
the corpus and seemed to work accpetably for me at least.
d.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: Hello dudes.
Posted by Serge Sozonoff <se...@sozonoff.com>.
Pong, I would say its really quit....
Serge
Danny Angus wrote:
> Ping, is this list broken or just *really* quiet..
>
> d.
>
>
>
> ***************************************************************************
> The information in this e-mail is confidential and for use by the addressee(s) only. If you are not the intended recipient (or responsible for delivery of the message to the intended recipient) please notify us immediately on 0141 306 2050 and delete the message from your computer. You may not copy or forward it or use or disclose its contents to any other person. As Internet communications are capable of data corruption Student Loans Company Limited does not accept any responsibility for changes made to this message after it was sent. For this reason it may be inappropriate to rely on advice or opinions contained in an e-mail without obtaining written confirmation of it. Neither Student Loans Company Limited or the sender accepts any liability or responsibility for viruses as it is your responsibility to scan attachments (if any). Opinions and views expressed in this e-mail are those of the sender and may not reflect the opinions and views of The Student Loans Company Li
mit
> ed.
>
> This footnote also confirms that this email message has been swept for the presence of computer viruses.
>
> **************************************************************************
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: Hello dudes.
Posted by "Noel J. Bergman" <no...@devtech.com>.
Just quiet the past week or so.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Hello dudes.
Posted by Danny Angus <Da...@slc.co.uk>.
Ping, is this list broken or just *really* quiet..
d.
***************************************************************************
The information in this e-mail is confidential and for use by the addressee(s) only. If you are not the intended recipient (or responsible for delivery of the message to the intended recipient) please notify us immediately on 0141 306 2050 and delete the message from your computer. You may not copy or forward it or use or disclose its contents to any other person. As Internet communications are capable of data corruption Student Loans Company Limited does not accept any responsibility for changes made to this message after it was sent. For this reason it may be inappropriate to rely on advice or opinions contained in an e-mail without obtaining written confirmation of it. Neither Student Loans Company Limited or the sender accepts any liability or responsibility for viruses as it is your responsibility to scan attachments (if any). Opinions and views expressed in this e-mail are those of the sender and may not reflect the opinions and views of The Student Loans Company Limit
ed.
This footnote also confirms that this email message has been swept for the presence of computer viruses.
**************************************************************************
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Commented: (JAMES-387) Exception in BayesianAnalysis
Posted by "Vincenzo Gianferrari Pini (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-387?page=comments#action_12358507 ]
Vincenzo Gianferrari Pini commented on JAMES-387:
-------------------------------------------------
Bernd is right: buildCorpus() is in a synchronized block to avoid messing when new mails are fed (in a separate thread), but I forgot to handle synchronization problems between buildCorpus() and getTokenProbabilityStrengths().
I will refactor builCorpus() to avoid this dirty double use of corpus.
Moreover corpus, hamTokenCounts and spamTokenCounts seem to be not cleared when loading/building an updated new corpus from the database.
> Exception in BayesianAnalysis
> -----------------------------
>
> Key: JAMES-387
> URL: http://issues.apache.org/jira/browse/JAMES-387
> Project: James
> Type: Bug
> Components: Matchers/Mailets (bundled)
> Versions: 3.0
> Environment: James from svn-trunk 2005-08-01.
> MySQL 4.0
> Reporter: Stefano Bagnara
> Assignee: Vincenzo Gianferrari Pini
> Priority: Minor
>
> Got this exception for every incoming mail:
> 02/08/05 00:39:25 INFO James.Mailet: BayesianAnalysis: Exception: java.lang.Integer
> java.lang.ClassCastException: java.lang.Integer
> at org.apache.james.util.BayesianAnalyzer.getTokenProbabilityStrengths(BayesianAnalyzer.java:591)
> at org.apache.james.util.BayesianAnalyzer.computeSpamProbability(BayesianAnalyzer.java:340)
> at org.apache.james.transport.mailets.BayesianAnalysis.service(BayesianAnalysis.java:289)
> at org.apache.james.transport.LinearProcessor.service(LinearProcessor.java:407)
> at org.apache.james.transport.JamesSpoolManager.process(JamesSpoolManager.java:460)
> at org.apache.james.transport.JamesSpoolManager.run(JamesSpoolManager.java:369)
> at java.lang.Thread.run(Unknown Source)
> If I clean my spam/ham db the exceptions disappears but they start again when the spam/ham db become large.
> My bayesiananalysis_spam contains 200000 rows.
> The following are the spam tokens with higher "occurrences".
> +---------------------------+-------------+
> | token | occurrences |
> +---------------------------+-------------+
> | 3D | 82151 |
> | a | 59953 |
> | the | 45295 |
> | FONT | 42771 |
> | Content-Type | 39058 |
> | to | 36626 |
> | com | 32902 |
> | http | 32886 |
> | of | 32504 |
> | font | 31803 |
> | and | 31577 |
> | Content-Transfer-Encoding | 31576 |
> | p | 29746 |
> | text | 29482 |
> | in | 29418 |
> | it | 28498 |
> | br | 28037 |
> | DIV | 27431 |
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Commented: (JAMES-387) Exception in BayesianAnalysis
Posted by "Bernd Fondermann (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-387?page=comments#action_12358307 ]
Bernd Fondermann commented on JAMES-387:
----------------------------------------
I looked at the Mailet code and found that in buildCorpus(), instance variable "corpus" is filled with all ham and spam tokens which appear to be Maps of (String, Integer) pairs. Afterwards, the map is iterated and all values are replaced by Doubles, but while this is running (and taking longer every time) there could still be a fair amount of Integer-typed values.
If another thread is stepping into line 591 at the same time this is still in process the error could very well occur because "corpus" is read there.
Are new mails fed in a separate thread?
The class cast in line 591 could be changed to "Number", as a very simple solution. Maybe it would also be appropriate to refactor buildCorpus() to work on a local map until it is ready with re-filling it with Doubles.
Hope this analysis makes some sense and I did not completely misread this whole case... :-)
> Exception in BayesianAnalysis
> -----------------------------
>
> Key: JAMES-387
> URL: http://issues.apache.org/jira/browse/JAMES-387
> Project: James
> Type: Bug
> Components: Matchers/Mailets (bundled)
> Versions: 3.0
> Environment: James from svn-trunk 2005-08-01.
> MySQL 4.0
> Reporter: Stefano Bagnara
> Assignee: Vincenzo Gianferrari Pini
> Priority: Minor
>
> Got this exception for every incoming mail:
> 02/08/05 00:39:25 INFO James.Mailet: BayesianAnalysis: Exception: java.lang.Integer
> java.lang.ClassCastException: java.lang.Integer
> at org.apache.james.util.BayesianAnalyzer.getTokenProbabilityStrengths(BayesianAnalyzer.java:591)
> at org.apache.james.util.BayesianAnalyzer.computeSpamProbability(BayesianAnalyzer.java:340)
> at org.apache.james.transport.mailets.BayesianAnalysis.service(BayesianAnalysis.java:289)
> at org.apache.james.transport.LinearProcessor.service(LinearProcessor.java:407)
> at org.apache.james.transport.JamesSpoolManager.process(JamesSpoolManager.java:460)
> at org.apache.james.transport.JamesSpoolManager.run(JamesSpoolManager.java:369)
> at java.lang.Thread.run(Unknown Source)
> If I clean my spam/ham db the exceptions disappears but they start again when the spam/ham db become large.
> My bayesiananalysis_spam contains 200000 rows.
> The following are the spam tokens with higher "occurrences".
> +---------------------------+-------------+
> | token | occurrences |
> +---------------------------+-------------+
> | 3D | 82151 |
> | a | 59953 |
> | the | 45295 |
> | FONT | 42771 |
> | Content-Type | 39058 |
> | to | 36626 |
> | com | 32902 |
> | http | 32886 |
> | of | 32504 |
> | font | 31803 |
> | and | 31577 |
> | Content-Transfer-Encoding | 31576 |
> | p | 29746 |
> | text | 29482 |
> | in | 29418 |
> | it | 28498 |
> | br | 28037 |
> | DIV | 27431 |
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Assigned: (JAMES-387) Exception in BayesianAnalysis
Posted by "Vincenzo Gianferrari Pini (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-387?page=all ]
Vincenzo Gianferrari Pini reassigned JAMES-387:
-----------------------------------------------
Assign To: Vincenzo Gianferrari Pini
> Exception in BayesianAnalysis
> -----------------------------
>
> Key: JAMES-387
> URL: http://issues.apache.org/jira/browse/JAMES-387
> Project: James
> Type: Bug
> Components: Matchers/Mailets (bundled)
> Versions: 3.0
> Environment: James from svn-trunk 2005-08-01.
> MySQL 4.0
> Reporter: Stefano Bagnara
> Assignee: Vincenzo Gianferrari Pini
> Priority: Minor
>
> Got this exception for every incoming mail:
> 02/08/05 00:39:25 INFO James.Mailet: BayesianAnalysis: Exception: java.lang.Integer
> java.lang.ClassCastException: java.lang.Integer
> at org.apache.james.util.BayesianAnalyzer.getTokenProbabilityStrengths(BayesianAnalyzer.java:591)
> at org.apache.james.util.BayesianAnalyzer.computeSpamProbability(BayesianAnalyzer.java:340)
> at org.apache.james.transport.mailets.BayesianAnalysis.service(BayesianAnalysis.java:289)
> at org.apache.james.transport.LinearProcessor.service(LinearProcessor.java:407)
> at org.apache.james.transport.JamesSpoolManager.process(JamesSpoolManager.java:460)
> at org.apache.james.transport.JamesSpoolManager.run(JamesSpoolManager.java:369)
> at java.lang.Thread.run(Unknown Source)
> If I clean my spam/ham db the exceptions disappears but they start again when the spam/ham db become large.
> My bayesiananalysis_spam contains 200000 rows.
> The following are the spam tokens with higher "occurrences".
> +---------------------------+-------------+
> | token | occurrences |
> +---------------------------+-------------+
> | 3D | 82151 |
> | a | 59953 |
> | the | 45295 |
> | FONT | 42771 |
> | Content-Type | 39058 |
> | to | 36626 |
> | com | 32902 |
> | http | 32886 |
> | of | 32504 |
> | font | 31803 |
> | and | 31577 |
> | Content-Transfer-Encoding | 31576 |
> | p | 29746 |
> | text | 29482 |
> | in | 29418 |
> | it | 28498 |
> | br | 28037 |
> | DIV | 27431 |
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Commented: (JAMES-387) Exception in BayesianAnalysis
Posted by "Vincenzo Gianferrari Pini (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-387?page=comments#action_12322588 ]
Vincenzo Gianferrari Pini commented on JAMES-387:
-------------------------------------------------
I gave a careful look to the code and couldn't find anything wrong. I have a spam table with more than 258000 rows and everything works fine for me.
IMHO a possible explanation of Stefano's exceptions is the following:
The ham/spam corpus hashmaps may take a lot of memory. Accordingly, I gave a lot of -Xmx memory to the JVM.
I remember some time ago, in a java (non James) application, an unpredictable JVM behaviour (strange exceptions thrown) when the available heap was just about the needed heap. Decreasing a little bit the -Xmx size I was getting OutOfMemoryError, and increasing it everything was fine.
Stefano, can you try with more memory?
> Exception in BayesianAnalysis
> -----------------------------
>
> Key: JAMES-387
> URL: http://issues.apache.org/jira/browse/JAMES-387
> Project: James
> Type: Bug
> Components: Matchers/Mailets (bundled)
> Versions: 3.0
> Environment: James from svn-trunk 2005-08-01.
> MySQL 4.0
> Reporter: Stefano Bagnara
> Assignee: Vincenzo Gianferrari Pini
> Priority: Minor
>
> Got this exception for every incoming mail:
> 02/08/05 00:39:25 INFO James.Mailet: BayesianAnalysis: Exception: java.lang.Integer
> java.lang.ClassCastException: java.lang.Integer
> at org.apache.james.util.BayesianAnalyzer.getTokenProbabilityStrengths(BayesianAnalyzer.java:591)
> at org.apache.james.util.BayesianAnalyzer.computeSpamProbability(BayesianAnalyzer.java:340)
> at org.apache.james.transport.mailets.BayesianAnalysis.service(BayesianAnalysis.java:289)
> at org.apache.james.transport.LinearProcessor.service(LinearProcessor.java:407)
> at org.apache.james.transport.JamesSpoolManager.process(JamesSpoolManager.java:460)
> at org.apache.james.transport.JamesSpoolManager.run(JamesSpoolManager.java:369)
> at java.lang.Thread.run(Unknown Source)
> If I clean my spam/ham db the exceptions disappears but they start again when the spam/ham db become large.
> My bayesiananalysis_spam contains 200000 rows.
> The following are the spam tokens with higher "occurrences".
> +---------------------------+-------------+
> | token | occurrences |
> +---------------------------+-------------+
> | 3D | 82151 |
> | a | 59953 |
> | the | 45295 |
> | FONT | 42771 |
> | Content-Type | 39058 |
> | to | 36626 |
> | com | 32902 |
> | http | 32886 |
> | of | 32504 |
> | font | 31803 |
> | and | 31577 |
> | Content-Transfer-Encoding | 31576 |
> | p | 29746 |
> | text | 29482 |
> | in | 29418 |
> | it | 28498 |
> | br | 28037 |
> | DIV | 27431 |
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Commented: (JAMES-387) Exception in BayesianAnalysis
Posted by "Stefano Bagnara (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-387?page=comments#action_12358106 ]
Stefano Bagnara commented on JAMES-387:
---------------------------------------
PS: James never logged an OutOfMemory and the exception is always identical so I don't think it is a memory problem.
> Exception in BayesianAnalysis
> -----------------------------
>
> Key: JAMES-387
> URL: http://issues.apache.org/jira/browse/JAMES-387
> Project: James
> Type: Bug
> Components: Matchers/Mailets (bundled)
> Versions: 3.0
> Environment: James from svn-trunk 2005-08-01.
> MySQL 4.0
> Reporter: Stefano Bagnara
> Assignee: Vincenzo Gianferrari Pini
> Priority: Minor
>
> Got this exception for every incoming mail:
> 02/08/05 00:39:25 INFO James.Mailet: BayesianAnalysis: Exception: java.lang.Integer
> java.lang.ClassCastException: java.lang.Integer
> at org.apache.james.util.BayesianAnalyzer.getTokenProbabilityStrengths(BayesianAnalyzer.java:591)
> at org.apache.james.util.BayesianAnalyzer.computeSpamProbability(BayesianAnalyzer.java:340)
> at org.apache.james.transport.mailets.BayesianAnalysis.service(BayesianAnalysis.java:289)
> at org.apache.james.transport.LinearProcessor.service(LinearProcessor.java:407)
> at org.apache.james.transport.JamesSpoolManager.process(JamesSpoolManager.java:460)
> at org.apache.james.transport.JamesSpoolManager.run(JamesSpoolManager.java:369)
> at java.lang.Thread.run(Unknown Source)
> If I clean my spam/ham db the exceptions disappears but they start again when the spam/ham db become large.
> My bayesiananalysis_spam contains 200000 rows.
> The following are the spam tokens with higher "occurrences".
> +---------------------------+-------------+
> | token | occurrences |
> +---------------------------+-------------+
> | 3D | 82151 |
> | a | 59953 |
> | the | 45295 |
> | FONT | 42771 |
> | Content-Type | 39058 |
> | to | 36626 |
> | com | 32902 |
> | http | 32886 |
> | of | 32504 |
> | font | 31803 |
> | and | 31577 |
> | Content-Transfer-Encoding | 31576 |
> | p | 29746 |
> | text | 29482 |
> | in | 29418 |
> | it | 28498 |
> | br | 28037 |
> | DIV | 27431 |
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Commented: (JAMES-387) Exception in BayesianAnalysis
Posted by "Stefano Bagnara (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-387?page=comments#action_12358105 ]
Stefano Bagnara commented on JAMES-387:
---------------------------------------
I increased the total memory for my "personal" james to 800MB and it only handle my own mail (around 1000 messages per day) but it still stop checking my messages.
Here is the exception
20/11/05 00:14:57 INFO James.Mailet: BayesianAnalysis: Exception: java.lang.Integer
java.lang.ClassCastException: java.lang.Integer
at org.apache.james.util.BayesianAnalyzer.getTokenProbabilityStrengths(BayesianAnalyzer.java:591)
at org.apache.james.util.BayesianAnalyzer.computeSpamProbability(BayesianAnalyzer.java:340)
at org.apache.james.transport.mailets.BayesianAnalysis.service(BayesianAnalysis.java:289)
at org.apache.james.transport.LinearProcessor.service(LinearProcessor.java:407)
at org.apache.james.transport.JamesSpoolManager.process(JamesSpoolManager.java:460)
at org.apache.james.transport.JamesSpoolManager.run(JamesSpoolManager.java:369)
at java.lang.Thread.run(Unknown Source)
If I restart James it works for almost a day and then it break again.
My bayesiananalysis_spam count 853685 rows, while the ham counts 21253.
I configured james to automatically feed spam and ham for messages I recognize so the bayesian can be improved. Maybe my continuous feeding is not good for the bayesian mailet.
Any Idea?
Who is using this Matchers/Mailets? What are your spam/ham sizes? How often do you feed ham/spam? how much memory you reserved to James? how much messages through the bayesian mailets?
> Exception in BayesianAnalysis
> -----------------------------
>
> Key: JAMES-387
> URL: http://issues.apache.org/jira/browse/JAMES-387
> Project: James
> Type: Bug
> Components: Matchers/Mailets (bundled)
> Versions: 3.0
> Environment: James from svn-trunk 2005-08-01.
> MySQL 4.0
> Reporter: Stefano Bagnara
> Assignee: Vincenzo Gianferrari Pini
> Priority: Minor
>
> Got this exception for every incoming mail:
> 02/08/05 00:39:25 INFO James.Mailet: BayesianAnalysis: Exception: java.lang.Integer
> java.lang.ClassCastException: java.lang.Integer
> at org.apache.james.util.BayesianAnalyzer.getTokenProbabilityStrengths(BayesianAnalyzer.java:591)
> at org.apache.james.util.BayesianAnalyzer.computeSpamProbability(BayesianAnalyzer.java:340)
> at org.apache.james.transport.mailets.BayesianAnalysis.service(BayesianAnalysis.java:289)
> at org.apache.james.transport.LinearProcessor.service(LinearProcessor.java:407)
> at org.apache.james.transport.JamesSpoolManager.process(JamesSpoolManager.java:460)
> at org.apache.james.transport.JamesSpoolManager.run(JamesSpoolManager.java:369)
> at java.lang.Thread.run(Unknown Source)
> If I clean my spam/ham db the exceptions disappears but they start again when the spam/ham db become large.
> My bayesiananalysis_spam contains 200000 rows.
> The following are the spam tokens with higher "occurrences".
> +---------------------------+-------------+
> | token | occurrences |
> +---------------------------+-------------+
> | 3D | 82151 |
> | a | 59953 |
> | the | 45295 |
> | FONT | 42771 |
> | Content-Type | 39058 |
> | to | 36626 |
> | com | 32902 |
> | http | 32886 |
> | of | 32504 |
> | font | 31803 |
> | and | 31577 |
> | Content-Transfer-Encoding | 31576 |
> | p | 29746 |
> | text | 29482 |
> | in | 29418 |
> | it | 28498 |
> | br | 28037 |
> | DIV | 27431 |
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
[jira] Resolved: (JAMES-387) Exception in BayesianAnalysis
Posted by "Vincenzo Gianferrari Pini (JIRA)" <se...@james.apache.org>.
[ http://issues.apache.org/jira/browse/JAMES-387?page=all ]
Vincenzo Gianferrari Pini resolved JAMES-387:
---------------------------------------------
Fix Version: 2.3.0
Resolution: Fixed
The corpus reload activity was possibly conflicting with any ongoing analysis of messages, and the corpus could screw up.
Now such reload activity is done on a new hashmap, that at the end of the reload becomes the actual corpus. In the meantime any analysis is done on the old corpus and no conflict occurs. The old corpus will eventually be garbage collected.
> Exception in BayesianAnalysis
> -----------------------------
>
> Key: JAMES-387
> URL: http://issues.apache.org/jira/browse/JAMES-387
> Project: James
> Type: Bug
> Components: Matchers/Mailets (bundled)
> Versions: 3.0
> Environment: James from svn-trunk 2005-08-01.
> MySQL 4.0
> Reporter: Stefano Bagnara
> Assignee: Vincenzo Gianferrari Pini
> Priority: Minor
> Fix For: 2.3.0
>
> Got this exception for every incoming mail:
> 02/08/05 00:39:25 INFO James.Mailet: BayesianAnalysis: Exception: java.lang.Integer
> java.lang.ClassCastException: java.lang.Integer
> at org.apache.james.util.BayesianAnalyzer.getTokenProbabilityStrengths(BayesianAnalyzer.java:591)
> at org.apache.james.util.BayesianAnalyzer.computeSpamProbability(BayesianAnalyzer.java:340)
> at org.apache.james.transport.mailets.BayesianAnalysis.service(BayesianAnalysis.java:289)
> at org.apache.james.transport.LinearProcessor.service(LinearProcessor.java:407)
> at org.apache.james.transport.JamesSpoolManager.process(JamesSpoolManager.java:460)
> at org.apache.james.transport.JamesSpoolManager.run(JamesSpoolManager.java:369)
> at java.lang.Thread.run(Unknown Source)
> If I clean my spam/ham db the exceptions disappears but they start again when the spam/ham db become large.
> My bayesiananalysis_spam contains 200000 rows.
> The following are the spam tokens with higher "occurrences".
> +---------------------------+-------------+
> | token | occurrences |
> +---------------------------+-------------+
> | 3D | 82151 |
> | a | 59953 |
> | the | 45295 |
> | FONT | 42771 |
> | Content-Type | 39058 |
> | to | 36626 |
> | com | 32902 |
> | http | 32886 |
> | of | 32504 |
> | font | 31803 |
> | and | 31577 |
> | Content-Transfer-Encoding | 31576 |
> | p | 29746 |
> | text | 29482 |
> | in | 29418 |
> | it | 28498 |
> | br | 28037 |
> | DIV | 27431 |
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org