You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@opennlp.apache.org by "Jason Baldridge (JIRA)" <ji...@apache.org> on 2011/04/14 05:22:05 UTC
[jira] [Created] (OPENNLP-154) normalization in perceptron
normalization in perceptron
---------------------------
Key: OPENNLP-154
URL: https://issues.apache.org/jira/browse/OPENNLP-154
Project: OpenNLP
Issue Type: Bug
Components: Maxent
Affects Versions: maxent-3.0.1-incubating
Reporter: Jason Baldridge
Assignee: Jason Baldridge
Priority: Minor
I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OPENNLP-154) normalization in perceptron
Posted by "Jason Baldridge (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044889#comment-13044889 ]
Jason Baldridge commented on OPENNLP-154:
-----------------------------------------
Sure, go ahead.
2011/6/6 Jörn Kottmann (JIRA) <ji...@apache.org>
--
Jason Baldridge
Assistant Professor, Department of Linguistics
The University of Texas at Austin
http://www.jasonbaldridge.com
http://twitter.com/jasonbaldridge
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Fix For: tools-1.5.2-incubating, maxent-3.0.2-incubating
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OPENNLP-154) normalization in perceptron
Posted by "Jason Baldridge (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13045114#comment-13045114 ]
Jason Baldridge commented on OPENNLP-154:
-----------------------------------------
I guess that was my Scala coming through, e.g. where I would have done the loops:
for (int oid = 0; oid < numOutcomes; oid++)
prior[oid] = Math.exp(prior[oid]/maxPrior);
double normal = 0.0;
for (int oid = 0; oid < numOutcomes; oid++)
normal += prior[oid];
for (int oid = 0; oid < numOutcomes; oid++)
prior[oid] /= normal;
As something along the lines of :
val priorExp = prior map (outcomePrior => Math.exp(outcomePrior/maxPrior))
val normal = priorExp sum
val priorNorm = priorExp map (_/normal)
Anyway, I've changed it to be one loop:
double normal = 0.0;
for (int oid = 0; oid < numOutcomes; oid++) {
prior[oid] = Math.exp(prior[oid]/maxPrior);
normal += prior[oid];
}
Jason
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Fix For: tools-1.5.2-incubating, maxent-3.0.2-incubating
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (OPENNLP-154) normalization in perceptron
Posted by "Jörn Kottmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jörn Kottmann closed OPENNLP-154.
---------------------------------
Resolution: Fixed
Fix Version/s: maxent-3.0.1-incubating
tools-1.5.1-incubating
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Fix For: tools-1.5.1-incubating, maxent-3.0.1-incubating
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OPENNLP-154) normalization in perceptron
Posted by "Jörn Kottmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044825#comment-13044825 ]
Jörn Kottmann commented on OPENNLP-154:
---------------------------------------
Ok, I actually already did. I can check it in if you want.
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Fix For: tools-1.5.2-incubating, maxent-3.0.2-incubating
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OPENNLP-154) normalization in perceptron
Posted by "Jason Baldridge (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13043886#comment-13043886 ]
Jason Baldridge commented on OPENNLP-154:
-----------------------------------------
Yes.
2011/6/2 Jörn Kottmann (JIRA) <ji...@apache.org>
--
Jason Baldridge
Assistant Professor, Department of Linguistics
The University of Texas at Austin
http://www.jasonbaldridge.com
http://twitter.com/jasonbaldridge
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Fix For: tools-1.5.2-incubating, maxent-3.0.2-incubating
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (OPENNLP-154) normalization in perceptron
Posted by "Jörn Kottmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jörn Kottmann closed OPENNLP-154.
---------------------------------
Resolution: Fixed
Fix is applied as discussed. Issue can be closed.
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Fix For: tools-1.5.2-incubating, maxent-3.0.2-incubating
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OPENNLP-154) normalization in perceptron
Posted by "Jörn Kottmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044915#comment-13044915 ]
Jörn Kottmann commented on OPENNLP-154:
---------------------------------------
I believe there are two loops which should be merged, in the first the priors are transformed with the exponential function, and in the second the transformed values are summed up. That could be done in one loop.
Is there a reason why it is done in two loops? Otherwise I suggest that we re-factor to do it in one loop.
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Fix For: tools-1.5.2-incubating, maxent-3.0.2-incubating
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OPENNLP-154) normalization in perceptron
Posted by "Jörn Kottmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13042702#comment-13042702 ]
Jörn Kottmann commented on OPENNLP-154:
---------------------------------------
Jason, is the replacement normalization you put in softmax normalization?
Like described here?
http://en.wikipedia.org/wiki/Softmax_activation_function
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Fix For: tools-1.5.2-incubating, maxent-3.0.2-incubating
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (OPENNLP-154) normalization in perceptron
Posted by "Jörn Kottmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jörn Kottmann reopened OPENNLP-154:
-----------------------------------
Reopened, new normalization outputs "Infinity" instead of normalized score.
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Fix For: tools-1.5.1-incubating, maxent-3.0.1-incubating
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OPENNLP-154) normalization in perceptron
Posted by "Jörn Kottmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044736#comment-13044736 ]
Jörn Kottmann commented on OPENNLP-154:
---------------------------------------
Can we do the following to fix the issue:
- Determine the maximum absolute prior value
- Divide all priors by the maximum value
This will result in priors ranging from -1 to 1 on which
we can safely perform the current normalization.
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Fix For: tools-1.5.2-incubating, maxent-3.0.2-incubating
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (OPENNLP-154) normalization in perceptron
Posted by "Jörn Kottmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jörn Kottmann reopened OPENNLP-154:
-----------------------------------
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Fix For: tools-1.5.1-incubating, maxent-3.0.1-incubating
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OPENNLP-154) normalization in perceptron
Posted by "Jason Baldridge (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044814#comment-13044814 ]
Jason Baldridge commented on OPENNLP-154:
-----------------------------------------
Sure. I'm going to work on the perceptron today, and will do this as part of that.
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Fix For: tools-1.5.2-incubating, maxent-3.0.2-incubating
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (OPENNLP-154) normalization in perceptron
Posted by "Jörn Kottmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jörn Kottmann updated OPENNLP-154:
----------------------------------
Fix Version/s: (was: maxent-3.0.1-incubating)
(was: tools-1.5.1-incubating)
maxent-3.0.2-incubating
tools-1.5.2-incubating
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Fix For: tools-1.5.2-incubating, maxent-3.0.2-incubating
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OPENNLP-154) normalization in perceptron
Posted by "Jörn Kottmann (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044910#comment-13044910 ]
Jörn Kottmann commented on OPENNLP-154:
---------------------------------------
I checked in the fix, please review.
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Fix For: tools-1.5.2-incubating, maxent-3.0.2-incubating
>
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (OPENNLP-154) normalization in perceptron
Posted by "Jason Baldridge (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OPENNLP-154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jason Baldridge resolved OPENNLP-154.
-------------------------------------
Resolution: Fixed
I changed it to exponentiation and then normalization.
> normalization in perceptron
> ---------------------------
>
> Key: OPENNLP-154
> URL: https://issues.apache.org/jira/browse/OPENNLP-154
> Project: OpenNLP
> Issue Type: Bug
> Components: Maxent
> Affects Versions: maxent-3.0.1-incubating
> Reporter: Jason Baldridge
> Assignee: Jason Baldridge
> Priority: Minor
> Original Estimate: 0h
> Remaining Estimate: 0h
>
> I found some issues with the way perceptron output was normalized. It was sort of a strange way to handle negative numbers that didn't really work.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira