You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by KR <Ke...@siemens.com> on 2007/11/09 15:24:31 UTC
Re: Create and populate a field when indexing
Grant Ingersoll-6 wrote:
>
> When you are indexing the file and adding the Document, you will need
> to parse out your filename per your regular expression, and then
> create the appropriate field:
>
> Document doc = new Document()
> String cat = getCategoryFromFileName(inputFileName)
> doc.add(new Field("category", cat, ...)
> //do the rest of your adds
>
> Just locate where in the demo the Document add is taking place (I
> forget the exact spot) and then add in the appropriate stuff from
> above. Obviously, you need to implement the method I stubbed called
> getCategoryFromFileName.
>
> HTH,
> Grant
>
Thanks, Grant. That was just the hint I needed.
I found that the fields are populated in HTMLDocument.
I added:
doc.add(new Field("category", "test", Field.Store.YES,
Field.Index.TOKENIZED));
and then used Luke to verify that this field had been added. It had.
Now I am trying to get a quick-and-dirty way of setting the field based on
the filename, but I'm running into problems that I don't really understand
well enough to fix quickly.
I have only very limited experience of Java programming, so I might be using
the wrong terms, but I think the problem is variable scope. I get a
compilation error:
HTMLDocument.java:86: cannot find symbol
symbol : variable url
location: class org.apache.lucene.demo.HTMLDocument
if (url.indexOf("-ov-") != -1) {
I thought I'd be able to use a simple mechanism based on indexOf() to check
the existence of a short sequence of characters within the filename. For
example, "-sys-". I know that this sequence, if it exists anywhere in the
full path must be in the filename.
So I put in a series of if statements like this:
if (url.indexOf("-sys-") != -1) {
string category = "system";
}
then right at the end:
doc.add(new Field("category", category, Field.Store.YES,
Field.Index.TOKENIZED));
Am I right in thinking that the variable url is undefined at this point in
the code? It certainly seems to be defined earlier on in the file:
public static String uid2url(String uid) {
String url = uid.replace('\u0000', '/'); // replace nulls with slashes
return url.substring(0, url.lastIndexOf('/')); // remove date from end
}
Is there some way for me to perhaps chop down to the filename here, and make
that available later in the code?
K.
--
View this message in context: http://www.nabble.com/Create-and-populate-a-field-when-indexing-tf4713018.html#a13667927
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Create and populate a field when indexing
Posted by Erick Erickson <er...@gmail.com>.
Sure, that'd work. It would also work to just continue to use
f.getPath() whenever you need it.
FWIW
Erick
On Nov 12, 2007 1:43 PM, KR <Ke...@siemens.com> wrote:
>
>
> Erick Erickson wrote:
> >
> > Sure, just define it in the same scope as you want to refer to it.
> > Of course, that tells you nothing <G>...
> >
> > Java variables go out of scope when the last '}' *at the same level*
> > is passed. For intance:
> >
> > {
> > string s1;
> > {
> > string s2;
> > } // s2 is out of scope after this line.
> > // s1 is still available
> > }
> > //s1 no longer available.
> >
> > So just define your category string "at the appropriate place" outside
> the
> > if statement and it will be available *after* the if. You may need to
> move
> > it outside the enclosing braces. Or outside the enclosing braces
> > outside the enclosing braces (as many levels as your braces are
> > nested that you want to refer to that variable).
> >
> >
>
>
> Thanks, I understand that better now... (I hope).
>
> It looks like the best way to do this would be to extract the filename
> from
> the path just after the code snippet below:
>
>
> public static Document Document(File f)
> throws IOException, InterruptedException {
> // make a new, empty document
> Document doc = new Document();
>
> // Add the url as a field named "path". Use a field that is
> // indexed (i.e. searchable), but don't tokenize the field into words.
> doc.add(new Field("path", f.getPath().replace(dirSep, '/'),
> Field.Store.YES,
> Field.Index.UN_TOKENIZED));
>
>
> This way the filename is available for all the subsequent if {} blocks.
>
> Keith.
> --
> View this message in context:
> http://www.nabble.com/Create-and-populate-a-field-when-indexing-tf4713018.html#a13711541
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
Re: Create and populate a field when indexing
Posted by KR <Ke...@siemens.com>.
Erick Erickson wrote:
>
> Sure, just define it in the same scope as you want to refer to it.
> Of course, that tells you nothing <G>...
>
> Java variables go out of scope when the last '}' *at the same level*
> is passed. For intance:
>
> {
> string s1;
> {
> string s2;
> } // s2 is out of scope after this line.
> // s1 is still available
> }
> //s1 no longer available.
>
> So just define your category string "at the appropriate place" outside the
> if statement and it will be available *after* the if. You may need to move
> it outside the enclosing braces. Or outside the enclosing braces
> outside the enclosing braces (as many levels as your braces are
> nested that you want to refer to that variable).
>
>
Thanks, I understand that better now... (I hope).
It looks like the best way to do this would be to extract the filename from
the path just after the code snippet below:
public static Document Document(File f)
throws IOException, InterruptedException {
// make a new, empty document
Document doc = new Document();
// Add the url as a field named "path". Use a field that is
// indexed (i.e. searchable), but don't tokenize the field into words.
doc.add(new Field("path", f.getPath().replace(dirSep, '/'),
Field.Store.YES,
Field.Index.UN_TOKENIZED));
This way the filename is available for all the subsequent if {} blocks.
Keith.
--
View this message in context: http://www.nabble.com/Create-and-populate-a-field-when-indexing-tf4713018.html#a13711541
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Create and populate a field when indexing
Posted by Erick Erickson <er...@gmail.com>.
Sure, just define it in the same scope as you want to refer to it.
Of course, that tells you nothing <G>...
Java variables go out of scope when the last '}' *at the same level*
is passed. For intance:
{
string s1;
{
string s2;
} // s2 is out of scope after this line.
// s1 is still available
}
//s1 no longer available.
So just define your category string "at the appropriate place" outside the
if statement and it will be available *after* the if. You may need to move
it outside the enclosing braces. Or outside the enclosing braces
outside the enclosing braces (as many levels as your braces are
nested that you want to refer to that variable).
Erick
On Nov 9, 2007 9:24 AM, KR <Ke...@siemens.com> wrote:
>
>
> Grant Ingersoll-6 wrote:
> >
> > When you are indexing the file and adding the Document, you will need
> > to parse out your filename per your regular expression, and then
> > create the appropriate field:
> >
> > Document doc = new Document()
> > String cat = getCategoryFromFileName(inputFileName)
> > doc.add(new Field("category", cat, ...)
> > //do the rest of your adds
> >
> > Just locate where in the demo the Document add is taking place (I
> > forget the exact spot) and then add in the appropriate stuff from
> > above. Obviously, you need to implement the method I stubbed called
> > getCategoryFromFileName.
> >
> > HTH,
> > Grant
> >
>
> Thanks, Grant. That was just the hint I needed.
>
> I found that the fields are populated in HTMLDocument.
>
> I added:
>
> doc.add(new Field("category", "test", Field.Store.YES,
> Field.Index.TOKENIZED));
>
> and then used Luke to verify that this field had been added. It had.
>
> Now I am trying to get a quick-and-dirty way of setting the field based on
> the filename, but I'm running into problems that I don't really understand
> well enough to fix quickly.
>
> I have only very limited experience of Java programming, so I might be
> using
> the wrong terms, but I think the problem is variable scope. I get a
> compilation error:
>
> HTMLDocument.java:86: cannot find symbol
> symbol : variable url
> location: class org.apache.lucene.demo.HTMLDocument
> if (url.indexOf("-ov-") != -1) {
>
>
> I thought I'd be able to use a simple mechanism based on indexOf() to
> check
> the existence of a short sequence of characters within the filename. For
> example, "-sys-". I know that this sequence, if it exists anywhere in the
> full path must be in the filename.
>
> So I put in a series of if statements like this:
>
> if (url.indexOf("-sys-") != -1) {
> string category = "system";
> }
>
> then right at the end:
> doc.add(new Field("category", category, Field.Store.YES,
> Field.Index.TOKENIZED));
>
> Am I right in thinking that the variable url is undefined at this point in
> the code? It certainly seems to be defined earlier on in the file:
>
> public static String uid2url(String uid) {
> String url = uid.replace('\u0000', '/'); // replace nulls with
> slashes
> return url.substring(0, url.lastIndexOf('/')); // remove date from end
> }
>
> Is there some way for me to perhaps chop down to the filename here, and
> make
> that available later in the code?
>
> K.
> --
> View this message in context:
> http://www.nabble.com/Create-and-populate-a-field-when-indexing-tf4713018.html#a13667927
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>