You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "Zajkowski, Radoslaw" <Ra...@proximity.ca> on 2012/02/09 23:03:04 UTC

regular expression in solrcore.config to be passed to dataConfig via DataImportHandler

Hi,

I have a good number of files which will be broken into a dozen + cores.

To make config management easier I have been using global xml files and passing settings to them as needed. My settings reside in solrcore.config and are passed to solr config and dataConfig as default values of the dataimporthandler

Settings file looks like this:
For Spanish core:
core.languagegroup=es
core.filenamefilter=.*(spa|spl|sppr|spus|esci|ese|esep|eses)\.(xml)

Fore English core:
core.languagegroup=en
core.filenamefilter=.*(eeau|eaw|eez|eep|eeap|eeat|eebe|eeci|eedk)\.(xml)

I am adding the core.filenamefilter value to the dataimporthandler as a default value like this:

<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
    <lst name="defaults">
                <str name="config">../../global_core_configs/DataConfig.xml</str>
                <str name="MetaDataLanguageGroup">${core.languagegroup}</str>
                <str name="MetaDataFileNameFilter ">${core.filenamefilter}</str>
        </lst>
  </requestHandler>

and then accessing in the dataConfig section like this

<entity name="f" processor="FileListEntityProcessor" baseDir="C: /repository/metadata/" fileName="${MetaDataFileNameFilter}" newerThan="'NOW-3DAYS'" recursive="false" rootEntity="false" dataSource="null">

It seems that the value gets ignored or not passed correctly or parsed as proper regex at the dataConfig level.

Any help greatly appreciated, thank you,

Radek.




Radoslaw Zajkowski
Senior Developer
O°
proximity
CANADA
t: 416-972-1505 ext.7306
c: 647-281-2567
f: 416-944-7886

2011 ADCC Interactive Agency of the Year
2011 Strategy Magazine Digital Agency of the Year

http://www.proximityworld.com/

Join us on:
Facebook - http://www.facebook.com/ProximityCanada
Twitter - http://twitter.com/ProximityWW
YouTube - http://www.youtube.com/proximitycanada





Please consider the environment before printing this e-mail.

This message and any attachments contain information, which may be confidential or privileged. If you are not the intended recipient, please refrain from any disclosure, copying, distribution or use of this information. Please be aware that such actions are prohibited. If you have received this transmission in error, kindly notify us by e-mail to mailto:helpdesk@bbdo.com. We appreciate your cooperation.

RE: regular expression in solrcore.config to be passed to dataConfig via DataImportHandler

Posted by "Dyer, James" <Ja...@ingrambook.com>.
I wouldn't feel too bad about this.  This is a pretty common gotcha and going forward it would be nice if we can make it easier to parameterize data-config.xml...

James Dyer
E-Commerce Systems
Ingram Content Group
(615) 213-4311


-----Original Message-----
From: Zajkowski, Radoslaw [mailto:Radoslaw.Zajkowski@proximity.ca] 
Sent: Thursday, February 09, 2012 4:16 PM
To: solr-user@lucene.apache.org
Subject: RE: regular expression in solrcore.config to be passed to dataConfig via DataImportHandler

Nevermind everybody, you're with stupid, the correct way to access these vars is to prefix with dataimporter.request

Got the answer here earlier today:

http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201008.mbox/%3C9F8B39CB3B7C6D4594293EA29CCF438B01702F22@ICQ-MAIL.icq.il.office.aol.com%3E



Radoslaw Zajkowski
Senior Developer
O°
proximity
CANADA
t: 416-972-1505 ext.7306
c: 647-281-2567
f: 416-944-7886

2011 ADCC Interactive Agency of the Year
2011 Strategy Magazine Digital Agency of the Year

http://www.proximityworld.com/

Join us on:
Facebook - http://www.facebook.com/ProximityCanada
Twitter - http://twitter.com/ProximityWW
YouTube - http://www.youtube.com/proximitycanada

-----Original Message-----

From: Zajkowski, Radoslaw [mailto:Radoslaw.Zajkowski@proximity.ca]
Sent: Thursday, February 09, 2012 5:03 PM
To: solr-user@lucene.apache.org
Subject: regular expression in solrcore.config to be passed to dataConfig via DataImportHandler

Hi,

I have a good number of files which will be broken into a dozen + cores.

To make config management easier I have been using global xml files and passing settings to them as needed. My settings reside in solrcore.config and are passed to solr config and dataConfig as default values of the dataimporthandler

Settings file looks like this:
For Spanish core:
core.languagegroup=es
core.filenamefilter=.*(spa|spl|sppr|spus|esci|ese|esep|eses)\.(xml)

Fore English core:
core.languagegroup=en
core.filenamefilter=.*(eeau|eaw|eez|eep|eeap|eeat|eebe|eeci|eedk)\.(xml)

I am adding the core.filenamefilter value to the dataimporthandler as a default value like this:

<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
    <lst name="defaults">
                <str name="config">../../global_core_configs/DataConfig.xml</str>
                <str name="MetaDataLanguageGroup">${core.languagegroup}</str>
                <str name="MetaDataFileNameFilter ">${core.filenamefilter}</str>
        </lst>
  </requestHandler>

and then accessing in the dataConfig section like this

<entity name="f" processor="FileListEntityProcessor" baseDir="C: /repository/metadata/" fileName="${MetaDataFileNameFilter}" newerThan="'NOW-3DAYS'" recursive="false" rootEntity="false" dataSource="null">

It seems that the value gets ignored or not passed correctly or parsed as proper regex at the dataConfig level.

Any help greatly appreciated, thank you,

Radek.




Radoslaw Zajkowski
Senior Developer
O°
proximity
CANADA
t: 416-972-1505 ext.7306
c: 647-281-2567
f: 416-944-7886

2011 ADCC Interactive Agency of the Year
2011 Strategy Magazine Digital Agency of the Year

http://www.proximityworld.com/

Join us on:
Facebook - http://www.facebook.com/ProximityCanada
Twitter - http://twitter.com/ProximityWW YouTube - http://www.youtube.com/proximitycanada





Please consider the environment before printing this e-mail.

This message and any attachments contain information, which may be confidential or privileged. If you are not the intended recipient, please refrain from any disclosure, copying, distribution or use of this information. Please be aware that such actions are prohibited. If you have received this transmission in error, kindly notify us by e-mail to mailto:helpdesk@bbdo.com. We appreciate your cooperation.



-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2012.0.1913 / Virus Database: 2112/4798 - Release Date: 02/09/12

 -----Original Message-----

Please consider the environment before printing this e-mail.

This message and any attachments contain information, which may be confidential or privileged. If you are not the intended recipient, please refrain from any disclosure, copying, distribution or use of this information. Please be aware that such actions are prohibited. If you have received this transmission in error, kindly notify us by e-mail to mailto:helpdesk@bbdo.com. We appreciate your cooperation.

RE: regular expression in solrcore.config to be passed to dataConfig via DataImportHandler

Posted by "Zajkowski, Radoslaw" <Ra...@proximity.ca>.
Nevermind everybody, you're with stupid, the correct way to access these vars is to prefix with dataimporter.request

Got the answer here earlier today:

http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201008.mbox/%3C9F8B39CB3B7C6D4594293EA29CCF438B01702F22@ICQ-MAIL.icq.il.office.aol.com%3E



Radoslaw Zajkowski
Senior Developer
O°
proximity
CANADA
t: 416-972-1505 ext.7306
c: 647-281-2567
f: 416-944-7886

2011 ADCC Interactive Agency of the Year
2011 Strategy Magazine Digital Agency of the Year

http://www.proximityworld.com/

Join us on:
Facebook - http://www.facebook.com/ProximityCanada
Twitter - http://twitter.com/ProximityWW
YouTube - http://www.youtube.com/proximitycanada

-----Original Message-----

From: Zajkowski, Radoslaw [mailto:Radoslaw.Zajkowski@proximity.ca]
Sent: Thursday, February 09, 2012 5:03 PM
To: solr-user@lucene.apache.org
Subject: regular expression in solrcore.config to be passed to dataConfig via DataImportHandler

Hi,

I have a good number of files which will be broken into a dozen + cores.

To make config management easier I have been using global xml files and passing settings to them as needed. My settings reside in solrcore.config and are passed to solr config and dataConfig as default values of the dataimporthandler

Settings file looks like this:
For Spanish core:
core.languagegroup=es
core.filenamefilter=.*(spa|spl|sppr|spus|esci|ese|esep|eses)\.(xml)

Fore English core:
core.languagegroup=en
core.filenamefilter=.*(eeau|eaw|eez|eep|eeap|eeat|eebe|eeci|eedk)\.(xml)

I am adding the core.filenamefilter value to the dataimporthandler as a default value like this:

<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler">
    <lst name="defaults">
                <str name="config">../../global_core_configs/DataConfig.xml</str>
                <str name="MetaDataLanguageGroup">${core.languagegroup}</str>
                <str name="MetaDataFileNameFilter ">${core.filenamefilter}</str>
        </lst>
  </requestHandler>

and then accessing in the dataConfig section like this

<entity name="f" processor="FileListEntityProcessor" baseDir="C: /repository/metadata/" fileName="${MetaDataFileNameFilter}" newerThan="'NOW-3DAYS'" recursive="false" rootEntity="false" dataSource="null">

It seems that the value gets ignored or not passed correctly or parsed as proper regex at the dataConfig level.

Any help greatly appreciated, thank you,

Radek.




Radoslaw Zajkowski
Senior Developer
O°
proximity
CANADA
t: 416-972-1505 ext.7306
c: 647-281-2567
f: 416-944-7886

2011 ADCC Interactive Agency of the Year
2011 Strategy Magazine Digital Agency of the Year

http://www.proximityworld.com/

Join us on:
Facebook - http://www.facebook.com/ProximityCanada
Twitter - http://twitter.com/ProximityWW YouTube - http://www.youtube.com/proximitycanada





Please consider the environment before printing this e-mail.

This message and any attachments contain information, which may be confidential or privileged. If you are not the intended recipient, please refrain from any disclosure, copying, distribution or use of this information. Please be aware that such actions are prohibited. If you have received this transmission in error, kindly notify us by e-mail to mailto:helpdesk@bbdo.com. We appreciate your cooperation.



-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2012.0.1913 / Virus Database: 2112/4798 - Release Date: 02/09/12

 -----Original Message-----

Please consider the environment before printing this e-mail.

This message and any attachments contain information, which may be confidential or privileged. If you are not the intended recipient, please refrain from any disclosure, copying, distribution or use of this information. Please be aware that such actions are prohibited. If you have received this transmission in error, kindly notify us by e-mail to mailto:helpdesk@bbdo.com. We appreciate your cooperation.