You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Bryan Bende <bb...@gmail.com> on 2016/10/03 17:41:09 UTC
EmbeddedSolrServer and Core dataDir in Solr 6.x
Curious if anyone knows how to create an EmbeddedSolrServer in Solr 6.x,
with a core where the dataDir is located somewhere outside of where the
config is located.
I'd like to do this without system properties, and all through Java code.
In Solr 5.x I was able to do this with the following code:
CoreContainer coreContainer = new CoreContainer(solrHome);
coreContainer.load();
Properties props = new Properties();
props.setProperty("dataDir", dataDir + "/" + coreName);
CoreDescriptor descriptor = new CoreDescriptor(coreContainer, coreName,
new File(coreHome, coreName).getAbsolutePath(), props);
SolrCore solrCore = coreContainer.create(descriptor);
new EmbeddedSolrServer(coreContainer, coreName);
The CoreContainer API changed a bit in 6.x and you can no longer pass in a
descriptor. I've tried a couple of things with the current API, but haven't
been able to get it working.
Any ideas are appreciated.
Thanks,
Bryan
Re: EmbeddedSolrServer and Core dataDir in Solr 6.x
Posted by Bryan Bende <bb...@gmail.com>.
After some more debugging, I think putting the dataDir in the
Map<String,String> of properties is actually working, but still running
into a couple of issues with the setup...
I created an example project that demonstrates the scenario:
https://github.com/bbende/solrcore-datdir-test/blob/master/src/test/java/org/apache/solr/EmbeddedSolrServerFactory.java
When calling coreContainer.create(String coreName, Path instancePath,
Map<String, String> parameters)...
If instancePath is relative, then the core is loaded with no errors, but it
ends up writing a core.properties relative to Solr home, so it writes:
src/test/resources/solr/src/test/resources/exampleCollection/core.properties
If instancePath is absolute, then it fails to start up because there is
already a core.properties at
/full/path/to/src/test/resources/exampleCollection, the exception is thrown
from:
at
org.apache.solr.core.CorePropertiesLocator.create(CorePropertiesLocator.java:66)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:778)
Since everything from src/test/resources is already being put under
target/test-classes as part of the build, I'm thinking a better approach
would be to reference those paths for the Solr home and instancePath.
If I remove the core.properties from src/test/resources/exampleCollection,
then it can write a new one to target/test-classes/exampleCollection, and
will even put the dataDir there by default.
On Mon, Oct 3, 2016 at 7:00 PM, Bryan Bende <bb...@gmail.com> wrote:
> Yea I'll try to put something together and report back.
>
> On Mon, Oct 3, 2016 at 6:54 PM, Alan Woodward <al...@flax.co.uk> wrote:
>
>> Ah, I see what you mean. Putting the dataDir property into the Map
>> certainly ought to work - can you write a test case that shows what’s
>> happening?
>>
>> Alan Woodward
>> www.flax.co.uk
>>
>>
>> > On 3 Oct 2016, at 23:50, Bryan Bende <bb...@gmail.com> wrote:
>> >
>> > Alan,
>> >
>> > Thanks for the response. I will double-check, but I believe that is
>> going
>> > to put the data directory for the core under coreHome/coreName.
>> >
>> > What I am trying to setup (and did a poor job of explaining) is
>> something
>> > like the following...
>> >
>> > - Solr home in src/test/resources/solr
>> > - Core home in src/test/resources/myCore
>> > - dataDir for the myCore in target/myCore (or something not in the
>> source
>> > tree).
>> >
>> > This way the unit tests can use the Solr home and core config that is
>> under
>> > version control, but the data from testing would be written somewhere
>> not
>> > under version control.
>> >
>> > in 5.x I was specifying the dataDir through the properties object... I
>> > would calculate the path to the target dir in Java code relative to the
>> > class file, and then pass that as dataDir to the following:
>> >
>> > Properties props = new Properties();
>> > props.setProperty("dataDir", dataDir + "/" + coreName);
>> >
>> > In 6.x it seems like Properties has been replaced with the
>> > Map<String,String> ? and I tried putting dataDir in there, but didn't
>> seem
>> > to do anything.
>> >
>> > For now I have just been using RAMDirectoryFactory so that no data ever
>> > gets written to disk.
>> >
>> > I'll keep trying different things, but if you have any thoughts let me
>> know.
>> >
>> > Thanks,
>> >
>> > Bryan
>> >
>> >
>> > On Mon, Oct 3, 2016 at 2:07 PM, Alan Woodward <al...@flax.co.uk> wrote:
>> >
>> >> This should work:
>> >>
>> >> SolrCore solrCore
>> >> = coreContainer.create(coreName, Paths.get(coreHome).resolve(co
>> reName),
>> >> Collections.emptyMap());
>> >>
>> >>
>> >> Alan Woodward
>> >> www.flax.co.uk
>> >>
>> >>
>> >>> On 3 Oct 2016, at 18:41, Bryan Bende <bb...@gmail.com> wrote:
>> >>>
>> >>> Curious if anyone knows how to create an EmbeddedSolrServer in Solr
>> 6.x,
>> >>> with a core where the dataDir is located somewhere outside of where
>> the
>> >>> config is located.
>> >>>
>> >>> I'd like to do this without system properties, and all through Java
>> code.
>> >>>
>> >>> In Solr 5.x I was able to do this with the following code:
>> >>>
>> >>> CoreContainer coreContainer = new CoreContainer(solrHome);
>> >>> coreContainer.load();
>> >>>
>> >>> Properties props = new Properties();
>> >>> props.setProperty("dataDir", dataDir + "/" + coreName);
>> >>>
>> >>> CoreDescriptor descriptor = new CoreDescriptor(coreContainer,
>> coreName,
>> >>> new File(coreHome, coreName).getAbsolutePath(), props);
>> >>>
>> >>> SolrCore solrCore = coreContainer.create(descriptor);
>> >>> new EmbeddedSolrServer(coreContainer, coreName);
>> >>>
>> >>>
>> >>> The CoreContainer API changed a bit in 6.x and you can no longer pass
>> in
>> >> a
>> >>> descriptor. I've tried a couple of things with the current API, but
>> >> haven't
>> >>> been able to get it working.
>> >>>
>> >>> Any ideas are appreciated.
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Bryan
>> >>
>> >>
>>
>>
>
Re: EmbeddedSolrServer and Core dataDir in Solr 6.x
Posted by Bryan Bende <bb...@gmail.com>.
Yea I'll try to put something together and report back.
On Mon, Oct 3, 2016 at 6:54 PM, Alan Woodward <al...@flax.co.uk> wrote:
> Ah, I see what you mean. Putting the dataDir property into the Map
> certainly ought to work - can you write a test case that shows what’s
> happening?
>
> Alan Woodward
> www.flax.co.uk
>
>
> > On 3 Oct 2016, at 23:50, Bryan Bende <bb...@gmail.com> wrote:
> >
> > Alan,
> >
> > Thanks for the response. I will double-check, but I believe that is going
> > to put the data directory for the core under coreHome/coreName.
> >
> > What I am trying to setup (and did a poor job of explaining) is something
> > like the following...
> >
> > - Solr home in src/test/resources/solr
> > - Core home in src/test/resources/myCore
> > - dataDir for the myCore in target/myCore (or something not in the source
> > tree).
> >
> > This way the unit tests can use the Solr home and core config that is
> under
> > version control, but the data from testing would be written somewhere not
> > under version control.
> >
> > in 5.x I was specifying the dataDir through the properties object... I
> > would calculate the path to the target dir in Java code relative to the
> > class file, and then pass that as dataDir to the following:
> >
> > Properties props = new Properties();
> > props.setProperty("dataDir", dataDir + "/" + coreName);
> >
> > In 6.x it seems like Properties has been replaced with the
> > Map<String,String> ? and I tried putting dataDir in there, but didn't
> seem
> > to do anything.
> >
> > For now I have just been using RAMDirectoryFactory so that no data ever
> > gets written to disk.
> >
> > I'll keep trying different things, but if you have any thoughts let me
> know.
> >
> > Thanks,
> >
> > Bryan
> >
> >
> > On Mon, Oct 3, 2016 at 2:07 PM, Alan Woodward <al...@flax.co.uk> wrote:
> >
> >> This should work:
> >>
> >> SolrCore solrCore
> >> = coreContainer.create(coreName, Paths.get(coreHome).resolve(
> coreName),
> >> Collections.emptyMap());
> >>
> >>
> >> Alan Woodward
> >> www.flax.co.uk
> >>
> >>
> >>> On 3 Oct 2016, at 18:41, Bryan Bende <bb...@gmail.com> wrote:
> >>>
> >>> Curious if anyone knows how to create an EmbeddedSolrServer in Solr
> 6.x,
> >>> with a core where the dataDir is located somewhere outside of where the
> >>> config is located.
> >>>
> >>> I'd like to do this without system properties, and all through Java
> code.
> >>>
> >>> In Solr 5.x I was able to do this with the following code:
> >>>
> >>> CoreContainer coreContainer = new CoreContainer(solrHome);
> >>> coreContainer.load();
> >>>
> >>> Properties props = new Properties();
> >>> props.setProperty("dataDir", dataDir + "/" + coreName);
> >>>
> >>> CoreDescriptor descriptor = new CoreDescriptor(coreContainer, coreName,
> >>> new File(coreHome, coreName).getAbsolutePath(), props);
> >>>
> >>> SolrCore solrCore = coreContainer.create(descriptor);
> >>> new EmbeddedSolrServer(coreContainer, coreName);
> >>>
> >>>
> >>> The CoreContainer API changed a bit in 6.x and you can no longer pass
> in
> >> a
> >>> descriptor. I've tried a couple of things with the current API, but
> >> haven't
> >>> been able to get it working.
> >>>
> >>> Any ideas are appreciated.
> >>>
> >>> Thanks,
> >>>
> >>> Bryan
> >>
> >>
>
>
Re: EmbeddedSolrServer and Core dataDir in Solr 6.x
Posted by Alan Woodward <al...@flax.co.uk>.
Ah, I see what you mean. Putting the dataDir property into the Map certainly ought to work - can you write a test case that shows what’s happening?
Alan Woodward
www.flax.co.uk
> On 3 Oct 2016, at 23:50, Bryan Bende <bb...@gmail.com> wrote:
>
> Alan,
>
> Thanks for the response. I will double-check, but I believe that is going
> to put the data directory for the core under coreHome/coreName.
>
> What I am trying to setup (and did a poor job of explaining) is something
> like the following...
>
> - Solr home in src/test/resources/solr
> - Core home in src/test/resources/myCore
> - dataDir for the myCore in target/myCore (or something not in the source
> tree).
>
> This way the unit tests can use the Solr home and core config that is under
> version control, but the data from testing would be written somewhere not
> under version control.
>
> in 5.x I was specifying the dataDir through the properties object... I
> would calculate the path to the target dir in Java code relative to the
> class file, and then pass that as dataDir to the following:
>
> Properties props = new Properties();
> props.setProperty("dataDir", dataDir + "/" + coreName);
>
> In 6.x it seems like Properties has been replaced with the
> Map<String,String> ? and I tried putting dataDir in there, but didn't seem
> to do anything.
>
> For now I have just been using RAMDirectoryFactory so that no data ever
> gets written to disk.
>
> I'll keep trying different things, but if you have any thoughts let me know.
>
> Thanks,
>
> Bryan
>
>
> On Mon, Oct 3, 2016 at 2:07 PM, Alan Woodward <al...@flax.co.uk> wrote:
>
>> This should work:
>>
>> SolrCore solrCore
>> = coreContainer.create(coreName, Paths.get(coreHome).resolve(coreName),
>> Collections.emptyMap());
>>
>>
>> Alan Woodward
>> www.flax.co.uk
>>
>>
>>> On 3 Oct 2016, at 18:41, Bryan Bende <bb...@gmail.com> wrote:
>>>
>>> Curious if anyone knows how to create an EmbeddedSolrServer in Solr 6.x,
>>> with a core where the dataDir is located somewhere outside of where the
>>> config is located.
>>>
>>> I'd like to do this without system properties, and all through Java code.
>>>
>>> In Solr 5.x I was able to do this with the following code:
>>>
>>> CoreContainer coreContainer = new CoreContainer(solrHome);
>>> coreContainer.load();
>>>
>>> Properties props = new Properties();
>>> props.setProperty("dataDir", dataDir + "/" + coreName);
>>>
>>> CoreDescriptor descriptor = new CoreDescriptor(coreContainer, coreName,
>>> new File(coreHome, coreName).getAbsolutePath(), props);
>>>
>>> SolrCore solrCore = coreContainer.create(descriptor);
>>> new EmbeddedSolrServer(coreContainer, coreName);
>>>
>>>
>>> The CoreContainer API changed a bit in 6.x and you can no longer pass in
>> a
>>> descriptor. I've tried a couple of things with the current API, but
>> haven't
>>> been able to get it working.
>>>
>>> Any ideas are appreciated.
>>>
>>> Thanks,
>>>
>>> Bryan
>>
>>
Re: EmbeddedSolrServer and Core dataDir in Solr 6.x
Posted by Bryan Bende <bb...@gmail.com>.
Alan,
Thanks for the response. I will double-check, but I believe that is going
to put the data directory for the core under coreHome/coreName.
What I am trying to setup (and did a poor job of explaining) is something
like the following...
- Solr home in src/test/resources/solr
- Core home in src/test/resources/myCore
- dataDir for the myCore in target/myCore (or something not in the source
tree).
This way the unit tests can use the Solr home and core config that is under
version control, but the data from testing would be written somewhere not
under version control.
in 5.x I was specifying the dataDir through the properties object... I
would calculate the path to the target dir in Java code relative to the
class file, and then pass that as dataDir to the following:
Properties props = new Properties();
props.setProperty("dataDir", dataDir + "/" + coreName);
In 6.x it seems like Properties has been replaced with the
Map<String,String> ? and I tried putting dataDir in there, but didn't seem
to do anything.
For now I have just been using RAMDirectoryFactory so that no data ever
gets written to disk.
I'll keep trying different things, but if you have any thoughts let me know.
Thanks,
Bryan
On Mon, Oct 3, 2016 at 2:07 PM, Alan Woodward <al...@flax.co.uk> wrote:
> This should work:
>
> SolrCore solrCore
> = coreContainer.create(coreName, Paths.get(coreHome).resolve(coreName),
> Collections.emptyMap());
>
>
> Alan Woodward
> www.flax.co.uk
>
>
> > On 3 Oct 2016, at 18:41, Bryan Bende <bb...@gmail.com> wrote:
> >
> > Curious if anyone knows how to create an EmbeddedSolrServer in Solr 6.x,
> > with a core where the dataDir is located somewhere outside of where the
> > config is located.
> >
> > I'd like to do this without system properties, and all through Java code.
> >
> > In Solr 5.x I was able to do this with the following code:
> >
> > CoreContainer coreContainer = new CoreContainer(solrHome);
> > coreContainer.load();
> >
> > Properties props = new Properties();
> > props.setProperty("dataDir", dataDir + "/" + coreName);
> >
> > CoreDescriptor descriptor = new CoreDescriptor(coreContainer, coreName,
> > new File(coreHome, coreName).getAbsolutePath(), props);
> >
> > SolrCore solrCore = coreContainer.create(descriptor);
> > new EmbeddedSolrServer(coreContainer, coreName);
> >
> >
> > The CoreContainer API changed a bit in 6.x and you can no longer pass in
> a
> > descriptor. I've tried a couple of things with the current API, but
> haven't
> > been able to get it working.
> >
> > Any ideas are appreciated.
> >
> > Thanks,
> >
> > Bryan
>
>
Re: EmbeddedSolrServer and Core dataDir in Solr 6.x
Posted by Alan Woodward <al...@flax.co.uk>.
This should work:
SolrCore solrCore
= coreContainer.create(coreName, Paths.get(coreHome).resolve(coreName), Collections.emptyMap());
Alan Woodward
www.flax.co.uk
> On 3 Oct 2016, at 18:41, Bryan Bende <bb...@gmail.com> wrote:
>
> Curious if anyone knows how to create an EmbeddedSolrServer in Solr 6.x,
> with a core where the dataDir is located somewhere outside of where the
> config is located.
>
> I'd like to do this without system properties, and all through Java code.
>
> In Solr 5.x I was able to do this with the following code:
>
> CoreContainer coreContainer = new CoreContainer(solrHome);
> coreContainer.load();
>
> Properties props = new Properties();
> props.setProperty("dataDir", dataDir + "/" + coreName);
>
> CoreDescriptor descriptor = new CoreDescriptor(coreContainer, coreName,
> new File(coreHome, coreName).getAbsolutePath(), props);
>
> SolrCore solrCore = coreContainer.create(descriptor);
> new EmbeddedSolrServer(coreContainer, coreName);
>
>
> The CoreContainer API changed a bit in 6.x and you can no longer pass in a
> descriptor. I've tried a couple of things with the current API, but haven't
> been able to get it working.
>
> Any ideas are appreciated.
>
> Thanks,
>
> Bryan