You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Maria Vazquez <ma...@dexone.com> on 2011/09/14 07:53:44 UTC
DIH delta last_index_time
Hi,
How do you handle the situation where the time on the server running Solr
doesn¹t match the time in the database?
I¹m using the last_index_time saved by Solr in the delta query checking it
against lastModifiedDate field in the database but the times are not in sync
so I might lose some changes.
Can we use something else other than last_index_time? Maybe something like
last_pk or something.
Thanks in advance.
Maria
Re: DIH delta last_index_time
Posted by "tamanjit.bindra@yahoo.co.in" <ta...@yahoo.co.in>.
Rahul is right.
You may add a script to change the date in data-import.properties to half
hour before the last modi time before each delta-import.
--
View this message in context: http://lucene.472066.n3.nabble.com/DIH-delta-last-index-time-tp3334992p3336203.html
Sent from the Solr - User mailing list archive at Nabble.com.
RE: DIH delta last_index_time
Posted by Claudia Robles <cl...@envisioninc.com>.
The solution that I am currently using is converting the last_index_time to UTC before comparing to the LastModified field in the DB.
LastModified > DATEADD(Hour, DATEDIFF(Hour, GETDATE(), GETUTCDATE()), '${dateimporter.last_index_time}')
This may be another option if the LastModified date in the DB is stored in UTC.
Regards,
Claudia
-----Original Message-----
From: Vazquez, Maria (STM) [mailto:Maria.Vazquez@DexOne.com]
Sent: Wednesday, September 14, 2011 9:27 AM
To: solr-user@lucene.apache.org
Subject: Re: DIH delta last_index_time
Thanks Rahul
That sounds like a good solution, I will change the code to support different timezones. Maybe this could be included in next release of Solr since a few people mentioned this problem too.
Thanks again
Maria
Sent from my Motorola ATRIX™ 4G on AT&T
-----Original message-----
From: Rahul Warawdekar <ra...@gmail.com>
To: solr-user@lucene.apache.org
Sent: Wed, Sep 14, 2011 14:01:08 GMT+00:00
Subject: Re: DIH delta last_index_time
Hi Maria/Gora,
I see this as more of a problem with the timezones in which the Solr server
and the database server are located.
Is this true ?
If yes, one more possibility of handling this scenario would be to customize
DataImportHandler code as follows
1. Add one more configuration property named "dbTimeZone" at the entity
level in "data-config.xml" file
2. While saving the lastIndexTime in the properties file, save it according
to the timezone specified in the config so that it is in sync with the
database
server time.
Basically customize the code so that all the time related updates to the
dataimport.properties file should be timezone specific.
On Wed, Sep 14, 2011 at 4:31 AM, Gora Mohanty <go...@mimirtech.com> wrote:
> On Wed, Sep 14, 2011 at 11:23 AM, Maria Vazquez
> <ma...@dexone.com> wrote:
> > Hi,
> > How do you handle the situation where the time on the server running Solr
> > doesn¹t match the time in the database?
>
> Firstly, why is that the case? NTP is pretty universal
> these days.
>
> > I¹m using the last_index_time saved by Solr in the delta query checking
> it
> > against lastModifiedDate field in the database but the times are not in
> sync
> > so I might lose some changes.
> > Can we use something else other than last_index_time? Maybe something
> like
> > last_pk or something.
>
> One possible way is to edit dataimport.properties, manually or through
> a script, to put the last_index_time back to a "safe" value.
>
> Regards,
> Gora
>
--
Thanks and Regards
Rahul A. Warawdekar
Re: DIH delta last_index_time
Posted by Gora Mohanty <go...@mimirtech.com>.
On Wed, Sep 14, 2011 at 9:56 PM, Vazquez, Maria (STM)
<Ma...@dexone.com> wrote:
> Thanks Rahul
> That sounds like a good solution, I will change the code to support different timezones. Maybe this could be included in next release of Solr since a few people mentioned this problem too.
[...]
If it was indeed a timezone issue, Solr can hardly be automatically
aware of a difference in timezones. At least, IMHO, this is an
implementation issue, and a timezone difference should normally
be easy to diagnose.
Regards,
Gora
Re: DIH delta last_index_time
Posted by "Vazquez, Maria (STM)" <Ma...@DexOne.com>.
Thanks Rahul
That sounds like a good solution, I will change the code to support different timezones. Maybe this could be included in next release of Solr since a few people mentioned this problem too.
Thanks again
Maria
Sent from my Motorola ATRIX™ 4G on AT&T
-----Original message-----
From: Rahul Warawdekar <ra...@gmail.com>
To: solr-user@lucene.apache.org
Sent: Wed, Sep 14, 2011 14:01:08 GMT+00:00
Subject: Re: DIH delta last_index_time
Hi Maria/Gora,
I see this as more of a problem with the timezones in which the Solr server
and the database server are located.
Is this true ?
If yes, one more possibility of handling this scenario would be to customize
DataImportHandler code as follows
1. Add one more configuration property named "dbTimeZone" at the entity
level in "data-config.xml" file
2. While saving the lastIndexTime in the properties file, save it according
to the timezone specified in the config so that it is in sync with the
database
server time.
Basically customize the code so that all the time related updates to the
dataimport.properties file should be timezone specific.
On Wed, Sep 14, 2011 at 4:31 AM, Gora Mohanty <go...@mimirtech.com> wrote:
> On Wed, Sep 14, 2011 at 11:23 AM, Maria Vazquez
> <ma...@dexone.com> wrote:
> > Hi,
> > How do you handle the situation where the time on the server running Solr
> > doesn¹t match the time in the database?
>
> Firstly, why is that the case? NTP is pretty universal
> these days.
>
> > I¹m using the last_index_time saved by Solr in the delta query checking
> it
> > against lastModifiedDate field in the database but the times are not in
> sync
> > so I might lose some changes.
> > Can we use something else other than last_index_time? Maybe something
> like
> > last_pk or something.
>
> One possible way is to edit dataimport.properties, manually or through
> a script, to put the last_index_time back to a "safe" value.
>
> Regards,
> Gora
>
--
Thanks and Regards
Rahul A. Warawdekar
Re: DIH delta last_index_time
Posted by Rahul Warawdekar <ra...@gmail.com>.
Hi Maria/Gora,
I see this as more of a problem with the timezones in which the Solr server
and the database server are located.
Is this true ?
If yes, one more possibility of handling this scenario would be to customize
DataImportHandler code as follows
1. Add one more configuration property named "dbTimeZone" at the entity
level in "data-config.xml" file
2. While saving the lastIndexTime in the properties file, save it according
to the timezone specified in the config so that it is in sync with the
database
server time.
Basically customize the code so that all the time related updates to the
dataimport.properties file should be timezone specific.
On Wed, Sep 14, 2011 at 4:31 AM, Gora Mohanty <go...@mimirtech.com> wrote:
> On Wed, Sep 14, 2011 at 11:23 AM, Maria Vazquez
> <ma...@dexone.com> wrote:
> > Hi,
> > How do you handle the situation where the time on the server running Solr
> > doesn¹t match the time in the database?
>
> Firstly, why is that the case? NTP is pretty universal
> these days.
>
> > I¹m using the last_index_time saved by Solr in the delta query checking
> it
> > against lastModifiedDate field in the database but the times are not in
> sync
> > so I might lose some changes.
> > Can we use something else other than last_index_time? Maybe something
> like
> > last_pk or something.
>
> One possible way is to edit dataimport.properties, manually or through
> a script, to put the last_index_time back to a "safe" value.
>
> Regards,
> Gora
>
--
Thanks and Regards
Rahul A. Warawdekar
Re: DIH delta last_index_time
Posted by Gora Mohanty <go...@mimirtech.com>.
On Wed, Sep 14, 2011 at 11:23 AM, Maria Vazquez
<ma...@dexone.com> wrote:
> Hi,
> How do you handle the situation where the time on the server running Solr
> doesn¹t match the time in the database?
Firstly, why is that the case? NTP is pretty universal
these days.
> I¹m using the last_index_time saved by Solr in the delta query checking it
> against lastModifiedDate field in the database but the times are not in sync
> so I might lose some changes.
> Can we use something else other than last_index_time? Maybe something like
> last_pk or something.
One possible way is to edit dataimport.properties, manually or through
a script, to put the last_index_time back to a "safe" value.
Regards,
Gora