You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by jawedshamshedi <ja...@gmail.com> on 2011/11/24 14:38:27 UTC
Index a null text field
Hi all,
I am indexing a table that has a field by the name of solr_keywords of type
text in mysql. And it contains null values also. While creating index in
solr, this field is not getting indexed.
Any help will be appreciated.
Thanks
--
View this message in context: http://lucene.472066.n3.nabble.com/Index-a-null-text-field-tp3533636p3533636.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Index a null text field
Posted by Erick Erickson <er...@gmail.com>.
Are you committing after the run?
Best
Erick
On Fri, Nov 25, 2011 at 1:32 PM, Young, Cody <Co...@move.com> wrote:
> I don't see anything wrong so far other than a typo here (missing a p in
> the second price):
> <field column="lastbid_price" name="lastbid_rice" />
>
> Can you see if there are any warnings in the log about documents not
> being able to be created?
>
> Also, you should have a field type definition for text in your schema.
> It will look something like
>
> <fieldtype name="text" class="solr.TextField"
> positionIncrementGap="100">
> <analyzer type="index">
>
> Can you send the full field type definition along as well?
>
> You can also try running a query like:
> ?q=keyword_stock:[* TO *]
> That will return any documents where keyword_stock is populated.
>
> Thanks,
> Cody
>
> -----Original Message-----
> From: jawedshamshedi [mailto:jawedshamshedi@gmail.com]
> Sent: Thursday, November 24, 2011 9:42 PM
> To: solr-user@lucene.apache.org
> Subject: RE: Index a null text field
>
> Hi Cody,
>
> Thanks for the reply.
>
> Please find the detail of that I am doing.
>
> Yes, I am using dataimport handler and the code snippet of it from
> solrconfig.xml is given below.
>
> <requestHandler name="/dataimport"
> class="org.apache.solr.handler.dataimport.DataImportHandler">
> <lst name="defaults">
> <str name="config">data-config.xml</str>
> </lst>
> </requestHandler>
>
> The data-config.xml is give below.
>
> <dataConfig>
> <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
> url="jdbc:mysql://localhost/database?zeroDateTimeBehavior=convertToNull"
> user="username" password="password"/>
> <document name="content">
> <entity name="auction" query="query(it's working file)">
> <field column="keyword" name="keyword_stock" />
> <field column="start_bidprice"
> name="start_bidprice" />
> <field column="end_date" name="end_date" />
> <field column="start_date" name="start_date" />
> <field column="lastbid_price"
> name="lastbid_rice" />
>
> <field column="un_id" name="un_id" />
>
> </entity>
> </document>
> </dataConfig>
>
> schema.xml
>
>
> <fields>
>
>
> <field name="un_id" type="string" indexed="true" stored="true"
> required="true" />
> <field name="keyword_stock" type="text" indexed="true"
> stored="true" />
> <field name="start_bidprice" type="tlong" indexed="true"
> stored="true" />
> <field name="end_date" type="tdate" indexed="true"
> stored="true" />
> <field name="start_date" type="tdate" indexed="true"
> stored="true" />
> <field name="lastbid_price" type="tfloat" indexed="true"
> stored="true" />
>
>
> </fields>
>
>
> <uniqueKey>un_id</uniqueKey>
>
>
> <defaultSearchField>ST_Name</defaultSearchField>
>
> he date type in mysql is given below.
>
> keyword text
> start_bidprice float(12,2)
> end_date datetime
> start_bidprice float(12,2)
> start_date datetime
>
>
> for some fields that are simple float, there index are being created. I
> also added this in data-config.xml's url
> zeroDateTimeBehavior=convertToNull but no avail.
>
> Please help Thanks in advance.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Index-a-null-text-field-tp3533636p353
> 5376.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
RE: Index a null text field
Posted by "Young, Cody" <Co...@move.com>.
I don't see anything wrong so far other than a typo here (missing a p in
the second price):
<field column="lastbid_price" name="lastbid_rice" />
Can you see if there are any warnings in the log about documents not
being able to be created?
Also, you should have a field type definition for text in your schema.
It will look something like
<fieldtype name="text" class="solr.TextField"
positionIncrementGap="100">
<analyzer type="index">
Can you send the full field type definition along as well?
You can also try running a query like:
?q=keyword_stock:[* TO *]
That will return any documents where keyword_stock is populated.
Thanks,
Cody
-----Original Message-----
From: jawedshamshedi [mailto:jawedshamshedi@gmail.com]
Sent: Thursday, November 24, 2011 9:42 PM
To: solr-user@lucene.apache.org
Subject: RE: Index a null text field
Hi Cody,
Thanks for the reply.
Please find the detail of that I am doing.
Yes, I am using dataimport handler and the code snippet of it from
solrconfig.xml is given below.
<requestHandler name="/dataimport"
class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">data-config.xml</str>
</lst>
</requestHandler>
The data-config.xml is give below.
<dataConfig>
<dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/database?zeroDateTimeBehavior=convertToNull"
user="username" password="password"/>
<document name="content">
<entity name="auction" query="query(it's working file)">
<field column="keyword" name="keyword_stock" />
<field column="start_bidprice"
name="start_bidprice" />
<field column="end_date" name="end_date" />
<field column="start_date" name="start_date" />
<field column="lastbid_price"
name="lastbid_rice" />
<field column="un_id" name="un_id" />
</entity>
</document>
</dataConfig>
schema.xml
<fields>
<field name="un_id" type="string" indexed="true" stored="true"
required="true" />
<field name="keyword_stock" type="text" indexed="true"
stored="true" />
<field name="start_bidprice" type="tlong" indexed="true"
stored="true" />
<field name="end_date" type="tdate" indexed="true"
stored="true" />
<field name="start_date" type="tdate" indexed="true"
stored="true" />
<field name="lastbid_price" type="tfloat" indexed="true"
stored="true" />
</fields>
<uniqueKey>un_id</uniqueKey>
<defaultSearchField>ST_Name</defaultSearchField>
he date type in mysql is given below.
keyword text
start_bidprice float(12,2)
end_date datetime
start_bidprice float(12,2)
start_date datetime
for some fields that are simple float, there index are being created. I
also added this in data-config.xml's url
zeroDateTimeBehavior=convertToNull but no avail.
Please help Thanks in advance.
--
View this message in context:
http://lucene.472066.n3.nabble.com/Index-a-null-text-field-tp3533636p353
5376.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: Index a null text field
Posted by jawedshamshedi <ja...@gmail.com>.
Hi Cody,
Thanks for the reply.
Please find the detail of that I am doing.
Yes, I am using dataimport handler and the code snippet of it from
solrconfig.xml is given below.
<requestHandler name="/dataimport"
class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">data-config.xml</str>
</lst>
</requestHandler>
The data-config.xml is give below.
<dataConfig>
<dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost/database?zeroDateTimeBehavior=convertToNull"
user="username" password="password"/>
<document name="content">
<entity name="auction" query="query(it's working file)">
<field column="keyword" name="keyword_stock" />
<field column="start_bidprice" name="start_bidprice" />
<field column="end_date" name="end_date" />
<field column="start_date" name="start_date" />
<field column="lastbid_price" name="lastbid_rice" />
<field column="un_id" name="un_id" />
</entity>
</document>
</dataConfig>
schema.xml
<fields>
<field name="un_id" type="string" indexed="true" stored="true"
required="true" />
<field name="keyword_stock" type="text" indexed="true" stored="true" />
<field name="start_bidprice" type="tlong" indexed="true" stored="true" />
<field name="end_date" type="tdate" indexed="true" stored="true" />
<field name="start_date" type="tdate" indexed="true" stored="true" />
<field name="lastbid_price" type="tfloat" indexed="true" stored="true" />
</fields>
<uniqueKey>un_id</uniqueKey>
<defaultSearchField>ST_Name</defaultSearchField>
he date type in mysql is given below.
keyword text
start_bidprice float(12,2)
end_date datetime
start_bidprice float(12,2)
start_date datetime
for some fields that are simple float, there index are being created. I also
added this in data-config.xml's url zeroDateTimeBehavior=convertToNull but
no avail.
Please help Thanks in advance.
--
View this message in context: http://lucene.472066.n3.nabble.com/Index-a-null-text-field-tp3533636p3535376.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: Index a null text field
Posted by "Young, Cody" <Co...@move.com>.
Hello,
We'll need more information please. How are you indexing the documents?
DataImportHandler? Xml Updates?
Can you show us the relevant parts of your schema? (Field definition and
data type for the field)
Are you getting any error messages in the log files?
Tell us more about your environment. Windows? Linux?
Thanks,
Cody
-----Original Message-----
From: jawedshamshedi [mailto:jawedshamshedi@gmail.com]
Sent: Thursday, November 24, 2011 5:38 AM
To: solr-user@lucene.apache.org
Subject: Index a null text field
Hi all,
I am indexing a table that has a field by the name of solr_keywords of
type text in mysql. And it contains null values also. While creating
index in solr, this field is not getting indexed.
Any help will be appreciated.
Thanks
--
View this message in context:
http://lucene.472066.n3.nabble.com/Index-a-null-text-field-tp3533636p353
3636.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Index a null text field
Posted by Chris Hostetter <ho...@fucit.org>.
: I am indexing a table that has a field by the name of solr_keywords of type
: text in mysql. And it contains null values also. While creating index in
: solr, this field is not getting indexed.
what exactly is the problem you are seeing?
If your documents are being indexed w/o error, but some documents with
null in the "solr_keywords" database field are not getting any (stored or
indexed) values in the resulting solr index, then it sounds like
everything is working properly.
There is no concept of a null value in a Solr index. Documents either
have a field value or they do not -- if you want to index the string
"null" (or any other special string for that matter) when a document has
no valued for a field, then there a few differnet wyas to do that.
this simplest in your case would probably be adding a "default" property
on the field in your schema, or using something like the "COALESCE"
function in your SQL.
-Hoss