You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Craig Stires <cr...@gmail.com> on 2011/04/20 05:17:26 UTC
Creating a TrieDateField (and other Trie fields) from Lucene Java
Wanted to share this, as I've seen a couple discussions on different boards.
The solution has been either:
1. use the solrj client
2. import as csv
3. use the streamingupdatesolrserver
The barrier I have is that I need to build this offline (without using a
solr server, solrconfig.xml, or schema.xml), instead using the solr java
classes to create the fields in a Lucene index. And, the Solr API for
FieldType doesn't seem to expose enough to allow this. To get done what I
needed, I created a class under the package org.apache.solr.schema (in my
own jar). There is probably a better way to get this done, but I just
needed to explicitly create TrieDateFields, and this got it done.
Sharing, if this helps someone in the same situation. Or, if someone has
got a better approach, I'd most appreciate it.
Thanks,
-Craig
<code>
package org.apache.solr.schema;
import java.util.Map;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import org.xml.sax.InputSource;
import org.xml.sax.SAXException;
import javax.xml.parsers.ParserConfigurationException;
import org.apache.lucene.document.Field;
import org.apache.solr.core.SolrConfig;
public class MY_SolrField {
// bit values for boolean field properties.
final public static int INDEXED = FieldProperties.INDEXED;
final public static int TOKENIZED = FieldProperties.TOKENIZED;
final public static int STORED = FieldProperties.STORED;
final public static int BINARY = FieldProperties.BINARY;
final public static int OMIT_NORMS = FieldProperties.OMIT_NORMS;
final public static int OMIT_TF_POSITIONS =
FieldProperties.OMIT_TF_POSITIONS;
final public static int STORE_TERMVECTORS =
FieldProperties.STORE_TERMVECTORS;
final public static int STORE_TERMPOSITIONS =
FieldProperties.STORE_TERMPOSITIONS;
final public static int STORE_TERMOFFSETS =
FieldProperties.STORE_TERMOFFSETS;
private FieldType ft = null;
private SchemaField sf = null;
public MY_SolrField(String fieldname, FieldType fieldtype, int
properties, Map<String,String> args, String defaultValue) {
ft = fieldtype;
init(args);
sf = new SchemaField(fieldname, ft, properties, defaultValue);
}
private void init(Map<String,String> args) {
IndexSchema ischema = null;
try {
String configMin = "<?xml version=\"1.0\" ?><config
name=\"min\"/>";
ByteArrayInputStream bisc = new
ByteArrayInputStream(configMin.getBytes());
InputSource isc = new InputSource(bisc);
SolrConfig sconfig = new SolrConfig("x", isc);
String schemaMin = "<?xml version=\"1.0\" ?><schema name=\"min\"
version=\"1.2\"/>";
ByteArrayInputStream biss = new
ByteArrayInputStream(schemaMin.getBytes());
InputSource iss = new InputSource(biss);
ischema = new IndexSchema(sconfig, "x", iss);
}
catch (ParserConfigurationException epc) {
epc.printStackTrace(System.out);
}
catch (SAXException epc) {
epc.printStackTrace(System.out);
}
catch (IOException eio) {
eio.printStackTrace(System.out);
}
ft.setArgs(ischema, args);
}
public Field createField(String fieldval, float boost) {
Field field = ft.createField(sf, fieldval, boost);
return (field);
}
}
</code>
And to create a TrieDateField
<code>
String name = "changed";
TrieDateField tdate = new TrieDateField();
int properties = (MY_SolrField.INDEXED | MY_SolrField.TOKENIZED |
MY_SolrField.STORED | MY_SolrField.OMIT_NORMS |
MY_SolrField.STORE_TERMVECTORS);
Map<String,String> myargs = new HashMap<String,String>();
myargs.put("precisionStep", "6");
myargs.put("positionIncrementGap", "0");
String defaultValue = "1970-01-01T00:00:00Z";
MY_SolrField myfield = new MY_SolrField(name, tdate, properties, myargs,
defaultValue);
Field field = myfield.createField("2011-04-18T18:30:00Z", 0);
doc.add(field);
</code>
RE: Creating a TrieDateField (and other Trie fields) from Lucene Java
Posted by Craig Stires <cr...@gmail.com>.
Hi Yonik,
The limitations I need to work within, have to do with the index already
being built as part of an existing process.
Currently, the Solr server is in read-only mode and receives new indexes
daily from a Java application. The Java app runs Lucene/Tika and is
indexing resources within the local network. It builds off of a different
schema framework, then moves the finished indexes over to the Solr
deployment path. The Solr server swaps over at that point. The Solr server
isn't the only consumer of the indexes. There are other Java apps which
read/write to the Lucene index, during the staging process.
This was working without issues, when using types were part of Lucene core
(String, Boolean, Integer, etc), because they just resolved to Strings.
But, the TrieDateField works off of byte data, so needed to find a way to
create those fields, using the existing classes.
Thanks,
-Craig
-----Original Message-----
From: yseeley@gmail.com [mailto:yseeley@gmail.com] On Behalf Of Yonik Seeley
Sent: Wednesday, 20 April 2011 11:19 PM
To: solr-user@lucene.apache.org
Subject: Re: Creating a TrieDateField (and other Trie fields) from Lucene
Java
On Tue, Apr 19, 2011 at 11:17 PM, Craig Stires <cr...@gmail.com>
wrote:
> The barrier I have is that I need to build this offline (without using a
> solr server, solrconfig.xml, or schema.xml)
This is pretty unusual... can you share your use case?
Solr can also be run in embedded mode if you can't run a stand-alone
server for some reason.
-Yonik
http://www.lucenerevolution.org -- Lucene/Solr User Conference, May
25-26, San Francisco
Re: Creating a TrieDateField (and other Trie fields) from Lucene Java
Posted by Yonik Seeley <yo...@lucidimagination.com>.
On Tue, Apr 19, 2011 at 11:17 PM, Craig Stires <cr...@gmail.com> wrote:
> The barrier I have is that I need to build this offline (without using a
> solr server, solrconfig.xml, or schema.xml)
This is pretty unusual... can you share your use case?
Solr can also be run in embedded mode if you can't run a stand-alone
server for some reason.
-Yonik
http://www.lucenerevolution.org -- Lucene/Solr User Conference, May
25-26, San Francisco