You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jon Drukman <jd...@gmail.com> on 2012/08/09 20:37:56 UTC

DataImportHandler WARNING: Unable to resolve variable

I'm trying to use DataImportHandler's delta-import functionality but I'm
getting loads of these every time it runs:

WARNING: Unable to resolve variable: article.url_type while parsing
expression: article:${article.url_type}:${article.id}

The definition looks like:

<entity name="article"
    query="... irrelevant ..."

    deltaQuery="select id,'dummy' as type_id FROM articles WHERE (post_date
&gt; '${dataimporter.last_index_time}' OR updated_date &gt;
'${dataimporter.last_index_time}') AND post_date &lt;= NOW() AND status = 9"

    deltaImportQuery="select id, article_seo_title,
DATE_FORMAT(post_date,'%Y-%m-%dT%H:%i:%sZ') post_date, subject,
           body, IF(url_type='', 'article', url_type) url_type,
featured_image_url from articles WHERE id = ${dataimporter.delta.id}"
           transformer="TemplateTransformer,HTMLStripTransformer">
            <field column="id" name="id" />
            <field column="post_date" name="post_date" />
            <field column="subject" name="title" />
            <field column="body" name="subhead" stripHTML="true" />
            <field column="type_id" template="article:${article.url_type}:${
article.id}" />
            <field column="type" template="2" />
            <field column="featured_image_url" name="main_image" />
            <field column="article_seo_title" name="seo_title" />
        </entity>

As you can see, I am always making sure that article.url_type has some
value.  Why am I getting the warning?

-jsd-

Re: DataImportHandler WARNING: Unable to resolve variable

Posted by Jon Drukman <jd...@gmail.com>.
That column does not allow NULL.  It's definitely an empty string, but I'm
using MySQL IF() to catch it and make sure it always has something.

On Thu, Aug 9, 2012 at 8:45 PM, Swati Swoboda <ss...@igloosoftware.com>wrote:

> Ah, my bad. I was incorrect - it was not actually indexing.
>
> @Jon - is there a possibility that your url_type is NULL, but not empty?
> Your if check only checks to see if it is empty, which is not the same as
> checking to see if it is null. If it is null, that's why you'd be having
> those errors - null values are just not accepted, it seems.
>
> Swati
>
> -----Original Message-----
> From: Swati Swoboda [mailto:sswoboda@igloosoftware.com]
> Sent: Thursday, August 09, 2012 11:09 PM
> To: solr-user@lucene.apache.org
> Subject: RE: DataImportHandler WARNING: Unable to resolve variable
>
> I am getting a similar issue when while using a Template Transformer. My
> fields *always* have a value as well - it is getting indexed correctly.
>
> Furthermore, the number of warnings I get seems arbitrary. I imported one
> document (debug mode) and I got roughly ~400 of those warning messages for
> the single field.
>
> -----Original Message-----
> From: Jon Drukman [mailto:jdrukman@gmail.com]
> Sent: Thursday, August 09, 2012 2:38 PM
> To: solr-user@lucene.apache.org
> Subject: DataImportHandler WARNING: Unable to resolve variable
>
> I'm trying to use DataImportHandler's delta-import functionality but I'm
> getting loads of these every time it runs:
>
> WARNING: Unable to resolve variable: article.url_type while parsing
> expression: article:${article.url_type}:${article.id}
>
> The definition looks like:
>
> <entity name="article"
>     query="... irrelevant ..."
>
>     deltaQuery="select id,'dummy' as type_id FROM articles WHERE
> (post_date &gt; '${dataimporter.last_index_time}' OR updated_date &gt;
> '${dataimporter.last_index_time}') AND post_date &lt;= NOW() AND status =
> 9"
>
>     deltaImportQuery="select id, article_seo_title,
> DATE_FORMAT(post_date,'%Y-%m-%dT%H:%i:%sZ') post_date, subject,
>            body, IF(url_type='', 'article', url_type) url_type,
> featured_image_url from articles WHERE id = ${dataimporter.delta.id}"
>            transformer="TemplateTransformer,HTMLStripTransformer">
>             <field column="id" name="id" />
>             <field column="post_date" name="post_date" />
>             <field column="subject" name="title" />
>             <field column="body" name="subhead" stripHTML="true" />
>             <field column="type_id"
> template="article:${article.url_type}:${
> article.id}" />
>             <field column="type" template="2" />
>             <field column="featured_image_url" name="main_image" />
>             <field column="article_seo_title" name="seo_title" />
>         </entity>
>
> As you can see, I am always making sure that article.url_type has some
> value.  Why am I getting the warning?
>
> -jsd-
>

RE: DataImportHandler WARNING: Unable to resolve variable

Posted by Swati Swoboda <ss...@igloosoftware.com>.
Ah, my bad. I was incorrect - it was not actually indexing. 

@Jon - is there a possibility that your url_type is NULL, but not empty? Your if check only checks to see if it is empty, which is not the same as checking to see if it is null. If it is null, that's why you'd be having those errors - null values are just not accepted, it seems.

Swati

-----Original Message-----
From: Swati Swoboda [mailto:sswoboda@igloosoftware.com] 
Sent: Thursday, August 09, 2012 11:09 PM
To: solr-user@lucene.apache.org
Subject: RE: DataImportHandler WARNING: Unable to resolve variable

I am getting a similar issue when while using a Template Transformer. My fields *always* have a value as well - it is getting indexed correctly.

Furthermore, the number of warnings I get seems arbitrary. I imported one document (debug mode) and I got roughly ~400 of those warning messages for the single field.

-----Original Message-----
From: Jon Drukman [mailto:jdrukman@gmail.com] 
Sent: Thursday, August 09, 2012 2:38 PM
To: solr-user@lucene.apache.org
Subject: DataImportHandler WARNING: Unable to resolve variable

I'm trying to use DataImportHandler's delta-import functionality but I'm getting loads of these every time it runs:

WARNING: Unable to resolve variable: article.url_type while parsing
expression: article:${article.url_type}:${article.id}

The definition looks like:

<entity name="article"
    query="... irrelevant ..."

    deltaQuery="select id,'dummy' as type_id FROM articles WHERE (post_date &gt; '${dataimporter.last_index_time}' OR updated_date &gt;
'${dataimporter.last_index_time}') AND post_date &lt;= NOW() AND status = 9"

    deltaImportQuery="select id, article_seo_title,
DATE_FORMAT(post_date,'%Y-%m-%dT%H:%i:%sZ') post_date, subject,
           body, IF(url_type='', 'article', url_type) url_type, featured_image_url from articles WHERE id = ${dataimporter.delta.id}"
           transformer="TemplateTransformer,HTMLStripTransformer">
            <field column="id" name="id" />
            <field column="post_date" name="post_date" />
            <field column="subject" name="title" />
            <field column="body" name="subhead" stripHTML="true" />
            <field column="type_id" template="article:${article.url_type}:${
article.id}" />
            <field column="type" template="2" />
            <field column="featured_image_url" name="main_image" />
            <field column="article_seo_title" name="seo_title" />
        </entity>

As you can see, I am always making sure that article.url_type has some value.  Why am I getting the warning?

-jsd-

RE: DataImportHandler WARNING: Unable to resolve variable

Posted by Swati Swoboda <ss...@igloosoftware.com>.
I am getting a similar issue when while using a Template Transformer. My fields *always* have a value as well - it is getting indexed correctly.

Furthermore, the number of warnings I get seems arbitrary. I imported one document (debug mode) and I got roughly ~400 of those warning messages for the single field.

-----Original Message-----
From: Jon Drukman [mailto:jdrukman@gmail.com] 
Sent: Thursday, August 09, 2012 2:38 PM
To: solr-user@lucene.apache.org
Subject: DataImportHandler WARNING: Unable to resolve variable

I'm trying to use DataImportHandler's delta-import functionality but I'm getting loads of these every time it runs:

WARNING: Unable to resolve variable: article.url_type while parsing
expression: article:${article.url_type}:${article.id}

The definition looks like:

<entity name="article"
    query="... irrelevant ..."

    deltaQuery="select id,'dummy' as type_id FROM articles WHERE (post_date &gt; '${dataimporter.last_index_time}' OR updated_date &gt;
'${dataimporter.last_index_time}') AND post_date &lt;= NOW() AND status = 9"

    deltaImportQuery="select id, article_seo_title,
DATE_FORMAT(post_date,'%Y-%m-%dT%H:%i:%sZ') post_date, subject,
           body, IF(url_type='', 'article', url_type) url_type, featured_image_url from articles WHERE id = ${dataimporter.delta.id}"
           transformer="TemplateTransformer,HTMLStripTransformer">
            <field column="id" name="id" />
            <field column="post_date" name="post_date" />
            <field column="subject" name="title" />
            <field column="body" name="subhead" stripHTML="true" />
            <field column="type_id" template="article:${article.url_type}:${
article.id}" />
            <field column="type" template="2" />
            <field column="featured_image_url" name="main_image" />
            <field column="article_seo_title" name="seo_title" />
        </entity>

As you can see, I am always making sure that article.url_type has some value.  Why am I getting the warning?

-jsd-