Re: Full import alternatives

[sami <sa...@gmail.com>]

Hi Shawan, can you please suggest a small program or atleast a backbone of a
program which can give me hints how exactly to achieve, I quote: "I send a
full-import DIH command to all of the
shards, and each one makes an SQL query to MySQL, all of them running in
parallel. "



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Full import alternatives

[sami <sa...@gmail.com>]

Dear Furkan,

I did. What i am not able to understand correctly at the moment, how to run
SOLR in parallel. 

So, i figured out that we can run indexing with SolrJ with XML file. 

http://lucene.472066.n3.nabble.com/Index-database-with-SolrJ-using-xml-file-directly-throws-an-error-td4426491.html

Now, I would like to run the job in parallel for dataimport not deltaimport
to index my documents to start with. What i m not sure, how to implement it. 

https://stackoverflow.com/questions/35690638/how-to-bulk-index-html-files-with-solr-cell

Here it is done with the multi-threading way. but how it will work with XML
file. As far as i understand till now, I need to specify XML file in conf
directory. 

and this conf directory has data-config.xml and solr-config.xml files. one
has to write several different files to override the existing one or how it
works that i m not really sure about. I thought of writing a properties
file. but then, i m confused how to implement it futher. 

Properties prop = new Properties();
	InputStream input = null;
	try {
        
		String filename = "indexer.properties";
		input = App.class.getClassLoader().getResourceAsStream(filename);
		if(input==null){
	            System.out.println("Indexer properties file not found error: "
+ filename);
		    return;
		}

		prop.load(input);

            System.out.println(prop.getProperty("xmlpath"));
	        System.out.println(prop.getProperty("solr-url"));
	        

	} catch (IOException ex) {
		ex.printStackTrace();
    } finally{
    	if(input!=null){
    		try {
			input.close();
		} catch (IOException e) {
			e.printStackTrace();
		}
    	}
    }

public void indexFiles() throws IOException, SolrServerException {
		
		ModifiableSolrParams params = new ModifiableSolrParams();
        params.set("qt", "/dataimport");
        params.set("command", "full-import");
        params.set("commit", "true");
        try {
        	solr.query(params);
        } catch (Exception e) {
            e.printStackTrace();
}
    }

I am bit lost here. 



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html

Re: Full import alternatives

[Furkan KAMACI <fu...@gmail.com>]

Hi Sami,

Did you check delta import documentation:
https://wiki.apache.org/solr/DataImportHandler#Using_delta-import_command

Kind Regards,
Furkan KAMACI

On Thu, Feb 28, 2019 at 7:24 PM sami <sa...@gmail.com> wrote:

> Hi Shawan, can you please suggest a small program or atleast a backbone of
> a
> program which can give me hints how exactly to achieve, I quote: "I send a
> full-import DIH command to all of the
> shards, and each one makes an SQL query to MySQL, all of them running in
> parallel. "
>
>
>
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>