You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Karl Wettin (JIRA)" <ji...@apache.org> on 2007/10/02 23:34:50 UTC
[jira] Created: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
TermVectorAccessor, transparent vector space access
----------------------------------------------------
Key: LUCENE-1016
URL: https://issues.apache.org/jira/browse/LUCENE-1016
Project: Lucene - Java
Issue Type: New Feature
Components: Term Vectors
Affects Versions: 2.2
Reporter: Karl Wettin
Priority: Minor
This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Michael McCandless (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michael McCandless reassigned LUCENE-1016:
------------------------------------------
Assignee: Karl Wettin
Looks like you have this one Karl...thanks!
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Assignee: Karl Wettin
> Priority: Minor
> Fix For: 2.4
>
> Attachments: LUCENE-1016.txt, LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1016:
--------------------------------
Attachment: LUCENE-1016.txt
Oups, prior patch contained some other stuff too by misstake.
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1016:
--------------------------------
Attachment: (was: LUCENE-1016.txt)
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558935#action_12558935 ]
Karl Wettin commented on LUCENE-1016:
-------------------------------------
{quote}
I'm curious if the build part of this would be faster than reanalyzing a document.
{quote}
It is a slow process on an index with many terms. Each one has to be iterated and mached against the document number.
{quote}
Just thinking outloud, but I have wondering about a Highlighter that uses the new TermVectorMapper, but using that doesn't account for non-TermVector based Documents that need to be analyzed. Was thinking this might account for both cases, all through the TermVectorMapper mechanism. Just doesn't seem like it would be very fast.
{quote}
This patch is mostly about when you don't have access to the source data. It was used together with a TermVectorMappingCachedTokenStreamFactory to extract re-indexable documents from any directory.
If you think of this peice of code and highlighter together, I would consider something else, perhaps a tool that could add the term vector to all documents missing one in a single iteration sweep of the index. I know very little about the file format and the highlighter though.
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1016:
--------------------------------
Attachment: (was: out.png)
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016.txt, LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1016:
--------------------------------
Attachment: (was: LUCENE-1016.txt)
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12531964 ]
Karl Wettin commented on LUCENE-1016:
-------------------------------------
I think a kD-tree will be the next step here. Does that fit in this project, or is that something I should post in UIMA or so?
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016-Tanimoto.txt, LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Issue Comment Edited: (LUCENE-1016) TermVectorAccessor,
transparent vector space access
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558825#action_12558825 ]
gsingers edited comment on LUCENE-1016 at 1/14/08 2:57 PM:
------------------------------------------------------------------
I'm curious if the build part of this would be faster than reanalyzing a document. Just thinking outloud, but I have wondering about a Highlighter that uses the new TermVectorMapper, but using that doesn't account for non-TermVector based Documents that need to be analyzed. Was thinking this might account for both cases, all through the TermVectorMapper mechanism. Just doesn't seem like it would be very fast.
was (Author: gsingers):
I'm curious if the build part of this would be faster than reanalyzing a document. Just thinking outloud, but I have wondering about a Highlighter that uses the new TermVectorMapper, but that doesn't account for non-TermVector based. Was thinking this might account for both cases.
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1016:
--------------------------------
Attachment: (was: LUCENE-1016-clusterer.txt)
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016.txt, LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12531972 ]
Grant Ingersoll commented on LUCENE-1016:
-----------------------------------------
Java 1.5 -> Java 1.4 :-(
Soon, very soon (in Lucene terms), we will have 1.5
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016-Tanimoto.txt, LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12558825#action_12558825 ]
Grant Ingersoll commented on LUCENE-1016:
-----------------------------------------
I'm curious if the build part of this would be faster than reanalyzing a document. Just thinking outloud, but I have wondering about a Highlighter that uses the new TermVectorMapper, but that doesn't account for non-TermVector based. Was thinking this might account for both cases.
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12531973 ]
Karl Wettin commented on LUCENE-1016:
-------------------------------------
Grant Ingersoll - 02/Oct/07 05:40 PM
> Java 1.5 -> Java 1.4
>
> Soon, very soon (in Lucene terms), we will have 1.5
This is why I placed it in contrib/misc, I was under the impression contrib allowed 1.5?
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016-Tanimoto.txt, LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1016:
--------------------------------
Attachment: (was: LUCENE-1016-Tanimoto.txt)
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016.txt, LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1016:
--------------------------------
Attachment: (was: LUCENE-1016.txt)
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1016:
--------------------------------
Attachment: LUCENE-1016-clusterer.txt
out.png
Sorry for flooding. This JIRA issue is sort of turning more off topic for each post.. I hope you don't mind.
LUCENE-1016-clusterer.txt now contains a refactor of the Tanimoto similarity, it does the same thing, but with less messy code.
And as the filename hints, I thought it would be fun to demonstrate the similarity by adding a very simple two dimensional decision tree clusterer.
For the test I feed it with 17 news articles representing 3 news stories I got from Google news. Attached is also a graphviz diagram that shows the tree with the news stories clustered together. I did not look at how to draw the line between the clusters yet, but I could probably come up with something simple enough. Legend: floating numbers represents the distance between two children. The leafs are the actual articles, prefixed with new story identity and suffixed with news article identity.
(The clusterer sure needs optimization, use carrot instead. This is just me fooling aroung.)
Have fun!
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016-clusterer.txt, LUCENE-1016-Tanimoto.txt, LUCENE-1016.txt, out.png
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532006 ]
Karl Wettin commented on LUCENE-1016:
-------------------------------------
Also, don't pay too much attention at the quite ugly code in TanimotoDocumentSimilarity. I'll post something nice and refactored soon. I was just really thrilled that I managed to figure out all them greek characters in the whitepaper.
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016-Tanimoto.txt, LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Closed: (LUCENE-1016) TermVectorAccessor, transparent vector
space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin closed LUCENE-1016.
-------------------------------
Resolution: Fixed
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Assignee: Karl Wettin
> Priority: Minor
> Fix For: 2.4
>
> Attachments: LUCENE-1016.txt, LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532227 ]
Karl Wettin commented on LUCENE-1016:
-------------------------------------
TermVectorMapper should probably also be able to extract the term vector from a document prior to it beeing indexed. That was the original reason for me to introduce tokenStreamValue(). However, I suppose there could probably be problems with token streams and readers beeing exhausted.
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016-clusterer.txt, LUCENE-1016-Tanimoto.txt, LUCENE-1016.txt, out.png
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1016:
--------------------------------
Attachment: LUCENE-1016.txt
This patch:
* All Java 1.4
* Bugfix, could throw a nullexception in some cases before
This patch is TermVectorAccessor code only, nothing else.
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016-clusterer.txt, LUCENE-1016-Tanimoto.txt, LUCENE-1016.txt, LUCENE-1016.txt, out.png
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1016:
--------------------------------
Fix Version/s: 2.4
I'll commit this soon.
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Fix For: 2.4
>
> Attachments: LUCENE-1016.txt, LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540518 ]
Karl Wettin commented on LUCENE-1016:
-------------------------------------
I think this is interesting:
http://www.nabble.com/How-to-generate-TermFreqVector-from-an-existing-index-tf4756257.html#a13601345
I'll have to look in to the file format and see if it is possible to persist a term vector retreived from the inverted index. That could be a nice addition to this issue.
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1016:
--------------------------------
Attachment: LUCENE-1016.txt
Documentation
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Fix For: 2.4
>
> Attachments: LUCENE-1016.txt, LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532229 ]
Karl Wettin commented on LUCENE-1016:
-------------------------------------
Karl Wettin - 03/Oct/07 12:52 PM
> TermVectorMapper should probably also be able..
TermVectorAccessor, that is.
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016-clusterer.txt, LUCENE-1016-Tanimoto.txt, LUCENE-1016.txt, out.png
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1016:
--------------------------------
Attachment: LUCENE-1016.txt
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1016:
--------------------------------
Attachment: LUCENE-1016.txt
In this patch:
* Java 1.4 for real
And then I removed everything that had nothing to do with this patch.
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Updated: (LUCENE-1016) TermVectorAccessor, transparent
vector space access
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin updated LUCENE-1016:
--------------------------------
Attachment: LUCENE-1016-Tanimoto.txt
TanimotoDocumentSimilarity, depends on TermVectorAccessor, used to calculate the distance between the vector space of two documents.
My math skills are pretty lame, but I think I got it right.
> TermVectorAccessor, transparent vector space access
> ----------------------------------------------------
>
> Key: LUCENE-1016
> URL: https://issues.apache.org/jira/browse/LUCENE-1016
> Project: Lucene - Java
> Issue Type: New Feature
> Components: Term Vectors
> Affects Versions: 2.2
> Reporter: Karl Wettin
> Priority: Minor
> Attachments: LUCENE-1016-Tanimoto.txt, LUCENE-1016.txt
>
>
> This class visits TermVectorMapper and populates it with information transparent by either passing it down to the default terms cache (documents indexed with Field.TermVector) or by resolving the inverted index.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org