You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Matias Alonso <ma...@gmail.com> on 2011/03/04 16:50:28 UTC
Help -DIH (mail)
Hi everyone!
I’m trying to index mails into solr through DHI (based on the
“example-DIH”). For this I´m using my personal email from gmail, but I can´t
index.
Configuration in Data-config .xml:
<dataConfig>
<document>
<entity name="email"
user="mail@gmail.com"
password="XXXXX"
host="imap.gmail.com"
fetchMailsSince="2011-01-01 00:00:00"
deltaFetch="true"
include=""
exclude=""
recurse="false"
folders="Recibidos"
includeContent="true"
processAttachments="false"
includeOtherUserFolders="false"
includeSharedFolders="false"
batchSize="100"
processor="MailEntityProcessor"
protocol="imap"/>
</document>
</dataConfig>
When I debug and verbose I get the following messange (only first lines):
<str name="EXCEPTION">
org.apache.solr.handler.dataimport.DataImportHandlerException: Connection
failed Processing Document # 1 at
I appreciate your help!
Regards,
Matias.
Re: Help -DIH (mail)
Posted by Matias Alonso <ma...@gmail.com>.
Thks Gora,
I forgot "s". Now, doesn´t have error, but not index.
This´s the answer in my command line (windows :( ).
......................
......................
......................
protocol : imaps
host : imap.gmail.com
folders :
Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,
correo,Correo
recurse : false
exclude : []
include : []
batchSize : 100
fetchSize : 32768
read timeout : 60000
conection timeout : 30000
custom filter :
fetch mail since : Sat Jan 01 00:00:00 GFT 2011
04/03/2011 14:05:40 org.apache.solr.core.SolrCore execute
INFO: [mail] webapp=/solr path=/select
params={command=status&qt=/dataimport} st
atus=0 QTime=0
04/03/2011 14:05:40 org.apache.solr.core.SolrCore execute
INFO: [mail] webapp=/solr path=/select
params={command=show-config&qt=/dataimpor
t} status=0 QTime=0
04/03/2011 14:05:40 org.apache.solr.handler.dataimport.MailEntityProcessor
conne
ctToMailBox
INFO: Connected to mailbox
04/03/2011 14:05:40 org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {} 0 0
04/03/2011 14:05:40 org.apache.solr.handler.dataimport.DocBuilder execute
INFO: Time taken = 0:0:3.31
04/03/2011 14:05:48 org.apache.solr.core.SolrCore execute
INFO: [mail] webapp=/solr path=/select
params={command=status&qt=/dataimport} st
atus=0 QTime=0
04/03/2011 14:05:48 org.apache.solr.core.SolrCore execute
INFO: [mail] webapp=/solr path=/select
params={command=show-config&qt=/dataimpor
t} status=0 QTime=0
Matias.
2011/3/4 Gora Mohanty <go...@mimirtech.com>
> On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <ma...@gmail.com>
> wrote:
> > Hi everyone!
> >
> >
> > I’m trying to index mails into solr through DHI (based on the
> > “example-DIH”). For this I´m using my personal email from gmail, but I
> can´t
> > index.
>
> Have not used the MailEntityProcessor with Gmail, but some
> points below:
>
> > Configuration in Data-config .xml:
> >
> > <dataConfig>
> >
> > <document>
> >
> > <entity name="email"
> >
> > user="mail@gmail.com"
> ^^^^^ I presume that you have put in your actual
> email address here.
> [...]
> > protocol="imap"/>
> ^^^^^ Shouldn't this be imaps, at least as
> per http://wiki.apache.org/solr/MailEntityProcessor
>
> Regards,
> Gora
>
Re: Help -DIH (mail)
Posted by Matias Alonso <ma...@gmail.com>.
Log:
09/03/2011 11:54:58 org.apache.solr.core.SolrCore execute
INFO: [mail] webapp=/solr path=/dataimport params={command=status} status=0
QTime=0
XML
<response>
-
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">0</int>
</lst>
-
<lst name="initArgs">
-
<lst name="defaults">
<str name="config">data-config.xml</str>
</lst>
</lst>
<str name="command">status</str>
<str name="status">idle</str>
<str name="importResponse"/>
-
<lst name="statusMessages">
<str name="Total Requests made to DataSource">0</str>
<str name="Total Rows Fetched">0</str>
<str name="Total Documents Skipped">0</str>
<str name="Full Dump Started">2011-03-09 11:52:01</str>
-
<str name="">
Indexing completed. Added/Updated: 0 documents. Deleted 0 documents.
</str>
<str name="Committed">2011-03-09 11:52:03</str>
<str name="Optimized">2011-03-09 11:52:03</str>
<str name="Total Documents Processed">0</str>
<str name="Time taken ">0:0:2.359</str>
</lst>
-
<str name="WARNING">
This response format is experimental. It is likely to change in the future.
</str>
</response>
Thks,
Matias.
2011/3/9 Peter Sturge <pe...@gmail.com>
> Hi,
>
> When you ran the status command, what was the output?
>
>
> On Wed, Mar 9, 2011 at 2:55 PM, Matias Alonso <ma...@gmail.com>
> wrote:
> > Peter,
> >
> > You´re right; may be I expose wrong because of my english.
> > I done everything you told me. I think that no find the "folder" when
> index.
> > What you thinking about?
> > Below I show to you part of the log.
> >
> >
> >
> > 09/03/2011 11:52:01 org.apache.solr.core.SolrCore execute
> > INFO: [mail] webapp=/solr path=/dataimport params={command=full-import}
> > status=0 QTime=0
> > 09/03/2011 11:52:01 org.apache.solr.handler.dataimport.DataImporter
> > doFullImport
> > INFO: Starting Full Import
> > 09/03/2011 11:52:01 org.apache.solr.handler.dataimport.SolrWriter
> > readIndexerProperties
> > INFO: Read dataimport.properties
> > 09/03/2011 11:52:01 org.apache.solr.update.DirectUpdateHandler2 deleteAll
> > INFO: [mail] REMOVING ALL DOCUMENTS FROM INDEX
> > 09/03/2011 11:52:01 org.apache.solr.core.SolrDeletionPolicy onInit
> > INFO: SolrDeletionPolicy.onInit: commits:num=1
> > commit{dir=D:\Search
> >
> Plugtree\trunk\solr\example\example-DIH\solr\mail\data\index,segFN=segments_c,version=1298912662799,generation=12,filenames=[segments_c]
> > 09/03/2011 11:52:01 org.apache.solr.core.SolrDeletionPolicy updateCommits
> > INFO: newest commit = 1298912662799
> > 09/03/2011 11:52:01
> org.apache.solr.handler.dataimport.MailEntityProcessor
> > logConfig
> > INFO: user : myemail@gmail.com
> > pwd : mypass
> > protocol : imaps
> > host : imap.gmail.com
> > folders :
> >
> Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo,Mail,mail,MAIL
> > recurse : false
> > exclude : []
> > include : []
> > batchSize : 100
> > fetchSize : 32768
> > read timeout : 60000
> > conection timeout : 30000
> > custom filter :
> > fetch mail since : Thu Mar 03 00:00:00 GFT 2011
> >
> > 09/03/2011 11:52:03
> org.apache.solr.handler.dataimport.MailEntityProcessor
> > connectToMailBox
> > INFO: Connected to mailbox
> > 09/03/2011 11:52:03 org.apache.solr.handler.dataimport.DocBuilder finish
> > INFO: Import completed successfully
> > 09/03/2011 11:52:03 org.apache.solr.update.DirectUpdateHandler2 commit
> > INFO: start
> >
> commit(optimize=true,waitFlush=false,waitSearcher=true,expungeDeletes=false)
> > 09/03/2011 11:52:03 org.apache.solr.core.SolrDeletionPolicy onCommit
> > INFO: SolrDeletionPolicy.onCommit: commits:num=2
> > commit{dir=D:\Search
> >
> Plugtree\trunk\solr\example\example-DIH\solr\mail\data\index,segFN=segments_c,version=1298912662799,generation=12,filenames=[segments_c]
> > commit{dir=D:\Search
> >
> Plugtree\trunk\solr\example\example-DIH\solr\mail\data\index,segFN=segments_d,version=1298912662800,generation=13,filenames=[segments_d]
> > 09/03/2011 11:52:03 org.apache.solr.core.SolrDeletionPolicy updateCommits
> > INFO: newest commit = 1298912662800
> > 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher <init>
> > INFO: Opening Searcher@1cee792 main
> > 09/03/2011 11:52:03 org.apache.solr.update.DirectUpdateHandler2 commit
> > INFO: end_commit_flush
> > 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> > INFO: autowarming Searcher@1cee792 main from Searcher@9a18a0 main
> >
> >
> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> > 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> > INFO: autowarming result for Searcher@1cee792 main
> >
> >
> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> > 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> > INFO: autowarming Searcher@1cee792 main from Searcher@9a18a0 main
> >
> >
> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> > 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> > INFO: autowarming result for Searcher@1cee792 main
> >
> >
> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> > 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> > INFO: autowarming Searcher@1cee792 main from Searcher@9a18a0 main
> >
> >
> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> > 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> > INFO: autowarming result for Searcher@1cee792 main
> >
> >
> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> > 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> > INFO: autowarming Searcher@1cee792 main from Searcher@9a18a0 main
> >
> >
> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> > 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> > INFO: autowarming result for Searcher@1cee792 main
> >
> >
> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> > 09/03/2011 11:52:03 org.apache.solr.core.QuerySenderListener newSearcher
> > INFO: QuerySenderListener sending requests to Searcher@1cee792 main
> > 09/03/2011 11:52:03 org.apache.solr.core.SolrCore execute
> > INFO: [mail] webapp=null path=null
> > params={start=0&event=newSearcher&q=solr&rows=10} hits=0 status=0 QTime=0
> > 09/03/2011 11:52:03 org.apache.solr.core.SolrCore execute
> > INFO: [mail] webapp=null path=null
> > params={start=0&event=newSearcher&q=rocks&rows=10} hits=0 status=0
> QTime=0
> > 09/03/2011 11:52:03 org.apache.solr.core.SolrCore execute
> > INFO: [mail] webapp=null path=null
> >
> params={event=newSearcher&q=static+newSearcher+warming+query+from+solrconfig.xml}
> > hits=0 status=0 QTime=0
> > 09/03/2011 11:52:03 org.apache.solr.core.QuerySenderListener newSearcher
> > INFO: QuerySenderListener done.
> > 09/03/2011 11:52:03 org.apache.solr.core.SolrCore registerSearcher
> > INFO: [mail] Registered new searcher Searcher@1cee792 main
> > 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher close
> > INFO: Closing Searcher@9a18a0 main
> >
> >
> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> >
> >
> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> >
> >
> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> >
> >
> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> > 09/03/2011 11:52:03 org.apache.solr.handler.dataimport.SolrWriter
> > readIndexerProperties
> > INFO: Read dataimport.properties
> > 09/03/2011 11:52:03 org.apache.solr.handler.dataimport.SolrWriter persist
> > INFO: Wrote last indexed time to dataimport.properties
> > 09/03/2011 11:52:03 org.apache.solr.update.processor.LogUpdateProcessor
> > finish
> > INFO: {deleteByQuery=*:*,optimize=} 0 0
> > 09/03/2011 11:52:03 org.apache.solr.handler.dataimport.DocBuilder execute
> > INFO: Time taken = 0:0:2.359
> >
> >
> >
> > 09/03/2011 11:54:58 org.apache.solr.core.SolrCore execute
> > INFO: [mail] webapp=/solr path=/dataimport params={command=status}
> status=0
> > QTime=0
> >
> >
> >
> > Thks,
> >
> > Matias.
> >
> >
> >
> >
> >
> > 2011/3/9 Peter Sturge <pe...@gmail.com>
> >
> >> Hi,
> >>
> >> You've included some output in your message, so I presume something
> >> *did* happen when you ran the 'status' command (but it might not be
> >> what you wanted to happen :-)
> >>
> >> If you run:
> >> http://localhost:8983/solr/mail/dataimport?command=status
> >>
> >> and you get something like this back:
> >> <str name="status">idle</str>
> >> <str name="importResponse"/>
> >> <lst name="statusMessages"/>
> >>
> >> It means that no full-import or delta-import has been run during the
> >> life of the JVM Solr session.
> >>
> >> You should try running:
> >> http://localhost:8983/solr/mail/dataimport?command=full-import
> >>
> >> Then run:
> >> http://localhost:8983/solr/mail/dataimport?command=status
> >>
> >> to see the status of the full-import (busy, idle, error, rolled back
> etc.)
> >>
> >> You can enable java logging by editing your JRE's lib/logging.properties
> >> file.
> >>
> >> Something like this should give you some log files:
> >> handlers= java.util.logging.FileHandler
> >> .level= INFO
> >> java.util.logging.FileHandler.pattern = ./logs/mylogs%d.log
> >> java.util.logging.FileeHandler.level = INFO
> >> java.util.logging.FileHandler.limit = 500000
> >> java.util.logging.FileHandler.count = 1
> >> java.util.logging.FileHandler.formatter =
> java.util.logging.SimpleFormatter
> >>
> >> NOTE: Make sure the 'logs' folder exists (in your $cwd) before you
> >> start, or you'll get an error.
> >>
> >> HTH
> >> Peter
> >>
> >>
> >>
> >>
> >> On Wed, Mar 9, 2011 at 12:47 PM, Matias Alonso <matiasgalonso@gmail.com
> >
> >> wrote:
> >> > Hi Peter,
> >> >
> >> > When I execute the commands you mentioned, nothing happend.
> >> > Below I show you the comands executed and the answered of they.
> >> > Sorry, but I don´t know how to enable the log; my jre is by default.
> >> > Rememeber I´m running the example-DIH
> >> (trunk\solr\example\example-DIH\solr);
> >> > java -Dsolr.solr.home="./example-DIH/solr/" -jar start.jar.
> >> >
> >> >
> >> >
> >> > Import:
> >> > http://localhost:8983/solr/mail/dataimport?command=status<
> >> > http://localhost:8983/solr/mail/dataimport?command=full-import>
> >> >
> >> > <response>
> >> > -
> >> > <lst name="responseHeader">
> >> > <int name="status">0</int>
> >> > <int name="QTime">15</int>
> >> > </lst>
> >> > -
> >> > <lst name="initArgs">
> >> > -
> >> > <lst name="defaults">
> >> > <str name="config">data-config.xml</str>
> >> > </lst>
> >> > </lst>
> >> > -
> >> > <str name="command">
> >> > full-import<
> >> http://localhost:8983/solr/mail/dataimport?command=full-import>
> >> > </str>
> >> > <str name="status">idle</str>
> >> > <str name="importResponse"/>
> >> > <lst name="statusMessages"/>
> >> > -
> >> > <str name="WARNING">
> >> > This response format is experimental. It is likely to change in the
> >> future.
> >> > </str>
> >> > </response>
> >> >
> >> >
> >> >
> >> > Status:
> >> > http://localhost:8983/solr/mail/dataimport?command=status<
> >> > http://localhost:8983/solr/mail/dataimport?command=full-import>
> >> >
> >> >
> >> > <response>
> >> > -
> >> > <lst name="responseHeader">
> >> > <int name="status">0</int>
> >> > <int name="QTime">0</int>
> >> > </lst>
> >> > -
> >> > <lst name="initArgs">
> >> > -
> >> > <lst name="defaults">
> >> > <str name="config">data-config.xml</str>
> >> > </lst>
> >> > </lst>
> >> > -
> >> > <str name="command">
> >> > status<http://localhost:8983/solr/mail/dataimport?command=full-import
> >
> >> > </str>
> >> > <str name="status">idle</str>
> >> > <str name="importResponse"/>
> >> > <lst name="statusMessages"/>
> >> > -
> >> > <str name="WARNING">
> >> > This response format is experimental. It is likely to change in the
> >> future.
> >> > </str>
> >> > </response>
> >> >
> >> >
> >> >
> >> >
> >> > Thank you for your help.
> >> >
> >> > Matias.
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >> >
> >> >> Can you try this:
> >> >>
> >> >> Issue a full import command like this:
> >> >>
> >> >> http://localhost:8983/solr/dataimport?command=full-import<
> >> >> http://localhost:8983/solr/db/dataimport?command=full-import>
> >> >>
> >> >> (There is no core name here - if you're using a core name (db?), then
> >> add
> >> >> that in between solr/ and /dataimport)
> >> >>
> >> >> then, run:
> >> >> http://localhost:8983/solr/dataimport?command=status<
> >> >> http://localhost:8983/solr/db/dataimport?command=full-import>
> >> >>
> >> >> This will show the results of the previous import. Has it been
> >> rolled-back?
> >> >> If so, there might be something in the log if it's enabled (see your
> >> jre's
> >> >> lib/logging.properties file).
> >> >> (you won't see any errors unless you run the status command - that's
> >> where
> >> >> they're stored)
> >> >>
> >> >> HTH
> >> >> Peter
> >> >>
> >> >>
> >> >>
> >> >>
> >> >> On Sat, Mar 5, 2011 at 12:46 AM, Matias Alonso <
> matiasgalonso@gmail.com
> >> >> >wrote:
> >> >>
> >> >> > I´m using the trunk.
> >> >> >
> >> >> > Thanks Peter for your preoccupation!
> >> >> >
> >> >> > Matias.
> >> >> >
> >> >> >
> >> >> >
> >> >> > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >> >> >
> >> >> > > Hi Matias,
> >> >> > >
> >> >> > > What version of Solr are you using? Are you running any patches
> >> (maybe
> >> >> > > SOLR-2245)?
> >> >> > >
> >> >> > > Thanks,
> >> >> > > Peter
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > > On Fri, Mar 4, 2011 at 8:25 PM, Matias Alonso <
> >> matiasgalonso@gmail.com
> >> >> > > >wrote:
> >> >> > >
> >> >> > > > Hi Peter,
> >> >> > > >
> >> >> > > > From "DataImportHandler Development Console" I made a
> full-import,
> >> >> but
> >> >> > > > didn´t work.
> >> >> > > >
> >> >> > > > Now, I execute "
> >> >> > > > http://localhost:8983/solr/mail/dataimport?command=full-import
> "
> >> but
> >> >> > > > nothing
> >> >> > > > happends; no index; no errors.
> >> >> > > >
> >> >> > > > thks...
> >> >> > > >
> >> >> > > > Matias.
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >> >> > > >
> >> >> > > > > Hi Mataias,
> >> >> > > > >
> >> >> > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimportaccesses
> >> >> > > > > the dataimport handler, but you need to tell it to do
> something
> >> by
> >> >> > > > > sending a command:
> >> >> > > > >
> >> >> > >
> >> >>
> >>
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
> >> >> > > > > ?command=full-import<
> >> >> > > > > http://localhost:8983/solr/db/dataimport?command=full-import
> >
> >> >> > > > >
> >> >> > > > > If you haven't already, have a look at:
> >> >> > > > >
> >> >> > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> http://www.lucidimagination.com/search/out?u=http%3A%2F%2Fwiki.apache.org%2Fsolr%2FDataImportHandler
> >> >> > > > >
> >> >> > > > > It gives very thorough and useful advice on getting the DIH
> >> >> working.
> >> >> > > > >
> >> >> > > > > Peter
> >> >> > > > >
> >> >> > > > >
> >> >> > > > >
> >> >> > > > > On Fri, Mar 4, 2011 at 6:59 PM, Matias Alonso <
> >> >> > matiasgalonso@gmail.com
> >> >> > > > > >wrote:
> >> >> > > > >
> >> >> > > > > > Hi Peter,
> >> >> > > > > >
> >> >> > > > > > I test with deltaFetch="false", but doesn´t work :(
> >> >> > > > > > I'm using "DataImportHandler Development Console" to index
> (
> >> >> > > > > >
> >> >> > > >
> >> >> >
> >>
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
> >> >> > > > > );
> >> >> > > > > > I'm working with "example-DIH".
> >> >> > > > > >
> >> >> > > > > > thks...
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > >
> >> >> > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >> >> > > > > >
> >> >> > > > > > > Hi Matias,
> >> >> > > > > > >
> >> >> > > > > > > I haven't seen it in the posts, but I may have missed it
> --
> >> >> what
> >> >> > is
> >> >> > > > the
> >> >> > > > > > > import command you're sending?
> >> >> > > > > > > Something like:
> >> >> > > > > > >
> >> http://localhost:8983/solr/db/dataimport?command=full-import
> >> >> > > > > > >
> >> >> > > > > > > Can you also test it with deltaFetch="false". I seem to
> >> >> remember
> >> >> > > > having
> >> >> > > > > > > some
> >> >> > > > > > > problems with delta in the MailEntityProcessor.
> >> >> > > > > > >
> >> >> > > > > > >
> >> >> > > > > > >
> >> >> > > > > > > On Fri, Mar 4, 2011 at 6:29 PM, Matias Alonso <
> >> >> > > > matiasgalonso@gmail.com
> >> >> > > > > > > >wrote:
> >> >> > > > > > >
> >> >> > > > > > > > <dataConfig>
> >> >> > > > > > > > <document>
> >> >> > > > > > > > <entity name="email"
> >> >> > > > > > > > user="myemail@gmail.com"
> >> >> > > > > > > > password="mypassword"
> >> >> > > > > > > > host="imap.gmail.com"
> >> >> > > > > > > > fetchMailsSince="2011-01-01 00:00:00"
> >> >> > > > > > > > deltaFetch="true"
> >> >> > > > > > > > include=""
> >> >> > > > > > > > exclude=""
> >> >> > > > > > > > recurse="false"
> >> >> > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> folders="Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo"
> >> >> > > > > > > > includeContent="true"
> >> >> > > > > > > > processAttachments="false"
> >> >> > > > > > > > includeOtherUserFolders="false"
> >> >> > > > > > > > includeSharedFolders="false"
> >> >> > > > > > > > batchSize="100"
> >> >> > > > > > > > processor="MailEntityProcessor"
> >> >> > > > > > > > protocol="imaps" />
> >> >> > > > > > > > </document>
> >> >> > > > > > > > </dataConfig>
> >> >> > > > > > > >
> >> >> > > > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >> >> > > > > > > >
> >> >> > > > > > > > > Hi Matias,
> >> >> > > > > > > > >
> >> >> > > > > > > > > Can you post your data-config.xml? (with disquised
> >> >> > > > > names/credentials)
> >> >> > > > > > > > >
> >> >> > > > > > > > > Thanks,
> >> >> > > > > > > > > Peter
> >> >> > > > > > > > >
> >> >> > > > > > > > >
> >> >> > > > > > > > > On Fri, Mar 4, 2011 at 5:13 PM, Matias Alonso <
> >> >> > > > > > matiasgalonso@gmail.com
> >> >> > > > > > > > > >wrote:
> >> >> > > > > > > > >
> >> >> > > > > > > > > > Thks Peter,
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > Yes, gmail gives me imaps (i understood that). So,
> I
> >> >> tried
> >> >> > > what
> >> >> > > > > you
> >> >> > > > > > > > > mention
> >> >> > > > > > > > > > but I had get the original mesange I posted.
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > Matias.
> >> >> > > > > > > > > >
> >> >> > > > > > > > > >
> >> >> > > > > > > > > >
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >> >> > > > > > > > > >
> >> >> > > > > > > > > > > Hi,
> >> >> > > > > > > > > > >
> >> >> > > > > > > > > > > You need to put your password in as well. You
> should
> >> >> use
> >> >> > > > > > > > > protocol="imap"
> >> >> > > > > > > > > > > unless your gmail is set for imaps (I don't
> believe
> >> the
> >> >> > > free
> >> >> > > > > > gmail
> >> >> > > > > > > > > gives
> >> >> > > > > > > > > > > you
> >> >> > > > > > > > > > > this).
> >> >> > > > > > > > > > >
> >> >> > > > > > > > > > > <entity name="email"
> >> >> > > > > > > > > > > user="user@mydomain.com"
> >> >> > > > > > > > > > > password="userpwd"
> >> >> > > > > > > > > > > host="imap.mydomain.com"
> >> >> > > > > > > > > > > include=""
> >> >> > > > > > > > > > > exclude=""
> >> >> > > > > > > > > > > processor="MailEntityProcessor"
> >> >> > > > > > > > > > > protocol="imap"
> >> >> > > > > > > > > > > />
> >> >> > > > > > > > > > >
> >> >> > > > > > > > > > > HTH
> >> >> > > > > > > > > > > Peter
> >> >> > > > > > > > > > >
> >> >> > > > > > > > > > >
> >> >> > > > > > > > > > >
> >> >> > > > > > > > > > > On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <
> >> >> > > > > gora@mimirtech.com
> >> >> > > > > > >
> >> >> > > > > > > > > wrote:
> >> >> > > > > > > > > > >
> >> >> > > > > > > > > > > > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <
> >> >> > > > > > > > > matiasgalonso@gmail.com
> >> >> > > > > > > > > > >
> >> >> > > > > > > > > > > > wrote:
> >> >> > > > > > > > > > > > > Hi everyone!
> >> >> > > > > > > > > > > > >
> >> >> > > > > > > > > > > > >
> >> >> > > > > > > > > > > > > I'm trying to index mails into solr through
> DHI
> >> >> > (based
> >> >> > > > on
> >> >> > > > > > the
> >> >> > > > > > > > > > > > > "example-DIH"). For this I´m using my
> personal
> >> >> email
> >> >> > > from
> >> >> > > > > > > gmail,
> >> >> > > > > > > > > but
> >> >> > > > > > > > > > I
> >> >> > > > > > > > > > > > can´t
> >> >> > > > > > > > > > > > > index.
> >> >> > > > > > > > > > > >
> >> >> > > > > > > > > > > > Have not used the MailEntityProcessor with
> Gmail,
> >> but
> >> >> > > some
> >> >> > > > > > > > > > > > points below:
> >> >> > > > > > > > > > > >
> >> >> > > > > > > > > > > > > Configuration in Data-config .xml:
> >> >> > > > > > > > > > > > >
> >> >> > > > > > > > > > > > > <dataConfig>
> >> >> > > > > > > > > > > > >
> >> >> > > > > > > > > > > > > <document>
> >> >> > > > > > > > > > > > >
> >> >> > > > > > > > > > > > > <entity name="email"
> >> >> > > > > > > > > > > > >
> >> >> > > > > > > > > > > > > user="mail@gmail.com"
> >> >> > > > > > > > > > > > ^^^^^ I presume that you have
> put
> >> in
> >> >> > your
> >> >> > > > > > actual
> >> >> > > > > > > > > > > > email address here.
> >> >> > > > > > > > > > > > [...]
> >> >> > > > > > > > > > > > > protocol="imap"/>
> >> >> > > > > > > > > > > > ^^^^^ Shouldn't this be
> >> imaps,
> >> >> at
> >> >> > > > least
> >> >> > > > > as
> >> >> > > > > > > > > > > > per
> >> http://wiki.apache.org/solr/MailEntityProcessor
> >> >> > > > > > > > > > > >
> >> >> > > > > > > > > > > > Regards,
> >> >> > > > > > > > > > > > Gora
> >> >> > > > > > > > > > > >
> >> >> > > > > > > > > > >
> >> >> > > > > > > > > >
> >> >> > > > > > > > >
> >> >> > > > > > > >
> >> >> > > > > > >
> >> >> > > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >> >
> >>
> >
>
Re: Help -DIH (mail)
Posted by Peter Sturge <pe...@gmail.com>.
Hi,
When you ran the status command, what was the output?
On Wed, Mar 9, 2011 at 2:55 PM, Matias Alonso <ma...@gmail.com> wrote:
> Peter,
>
> You´re right; may be I expose wrong because of my english.
> I done everything you told me. I think that no find the "folder" when index.
> What you thinking about?
> Below I show to you part of the log.
>
>
>
> 09/03/2011 11:52:01 org.apache.solr.core.SolrCore execute
> INFO: [mail] webapp=/solr path=/dataimport params={command=full-import}
> status=0 QTime=0
> 09/03/2011 11:52:01 org.apache.solr.handler.dataimport.DataImporter
> doFullImport
> INFO: Starting Full Import
> 09/03/2011 11:52:01 org.apache.solr.handler.dataimport.SolrWriter
> readIndexerProperties
> INFO: Read dataimport.properties
> 09/03/2011 11:52:01 org.apache.solr.update.DirectUpdateHandler2 deleteAll
> INFO: [mail] REMOVING ALL DOCUMENTS FROM INDEX
> 09/03/2011 11:52:01 org.apache.solr.core.SolrDeletionPolicy onInit
> INFO: SolrDeletionPolicy.onInit: commits:num=1
> commit{dir=D:\Search
> Plugtree\trunk\solr\example\example-DIH\solr\mail\data\index,segFN=segments_c,version=1298912662799,generation=12,filenames=[segments_c]
> 09/03/2011 11:52:01 org.apache.solr.core.SolrDeletionPolicy updateCommits
> INFO: newest commit = 1298912662799
> 09/03/2011 11:52:01 org.apache.solr.handler.dataimport.MailEntityProcessor
> logConfig
> INFO: user : myemail@gmail.com
> pwd : mypass
> protocol : imaps
> host : imap.gmail.com
> folders :
> Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo,Mail,mail,MAIL
> recurse : false
> exclude : []
> include : []
> batchSize : 100
> fetchSize : 32768
> read timeout : 60000
> conection timeout : 30000
> custom filter :
> fetch mail since : Thu Mar 03 00:00:00 GFT 2011
>
> 09/03/2011 11:52:03 org.apache.solr.handler.dataimport.MailEntityProcessor
> connectToMailBox
> INFO: Connected to mailbox
> 09/03/2011 11:52:03 org.apache.solr.handler.dataimport.DocBuilder finish
> INFO: Import completed successfully
> 09/03/2011 11:52:03 org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: start
> commit(optimize=true,waitFlush=false,waitSearcher=true,expungeDeletes=false)
> 09/03/2011 11:52:03 org.apache.solr.core.SolrDeletionPolicy onCommit
> INFO: SolrDeletionPolicy.onCommit: commits:num=2
> commit{dir=D:\Search
> Plugtree\trunk\solr\example\example-DIH\solr\mail\data\index,segFN=segments_c,version=1298912662799,generation=12,filenames=[segments_c]
> commit{dir=D:\Search
> Plugtree\trunk\solr\example\example-DIH\solr\mail\data\index,segFN=segments_d,version=1298912662800,generation=13,filenames=[segments_d]
> 09/03/2011 11:52:03 org.apache.solr.core.SolrDeletionPolicy updateCommits
> INFO: newest commit = 1298912662800
> 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@1cee792 main
> 09/03/2011 11:52:03 org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: end_commit_flush
> 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@1cee792 main from Searcher@9a18a0 main
>
> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@1cee792 main
>
> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@1cee792 main from Searcher@9a18a0 main
>
> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@1cee792 main
>
> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@1cee792 main from Searcher@9a18a0 main
>
> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@1cee792 main
>
> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming Searcher@1cee792 main from Searcher@9a18a0 main
>
> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
> INFO: autowarming result for Searcher@1cee792 main
>
> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> 09/03/2011 11:52:03 org.apache.solr.core.QuerySenderListener newSearcher
> INFO: QuerySenderListener sending requests to Searcher@1cee792 main
> 09/03/2011 11:52:03 org.apache.solr.core.SolrCore execute
> INFO: [mail] webapp=null path=null
> params={start=0&event=newSearcher&q=solr&rows=10} hits=0 status=0 QTime=0
> 09/03/2011 11:52:03 org.apache.solr.core.SolrCore execute
> INFO: [mail] webapp=null path=null
> params={start=0&event=newSearcher&q=rocks&rows=10} hits=0 status=0 QTime=0
> 09/03/2011 11:52:03 org.apache.solr.core.SolrCore execute
> INFO: [mail] webapp=null path=null
> params={event=newSearcher&q=static+newSearcher+warming+query+from+solrconfig.xml}
> hits=0 status=0 QTime=0
> 09/03/2011 11:52:03 org.apache.solr.core.QuerySenderListener newSearcher
> INFO: QuerySenderListener done.
> 09/03/2011 11:52:03 org.apache.solr.core.SolrCore registerSearcher
> INFO: [mail] Registered new searcher Searcher@1cee792 main
> 09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher close
> INFO: Closing Searcher@9a18a0 main
>
> fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>
> filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>
> queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
>
> documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
> 09/03/2011 11:52:03 org.apache.solr.handler.dataimport.SolrWriter
> readIndexerProperties
> INFO: Read dataimport.properties
> 09/03/2011 11:52:03 org.apache.solr.handler.dataimport.SolrWriter persist
> INFO: Wrote last indexed time to dataimport.properties
> 09/03/2011 11:52:03 org.apache.solr.update.processor.LogUpdateProcessor
> finish
> INFO: {deleteByQuery=*:*,optimize=} 0 0
> 09/03/2011 11:52:03 org.apache.solr.handler.dataimport.DocBuilder execute
> INFO: Time taken = 0:0:2.359
>
>
>
> 09/03/2011 11:54:58 org.apache.solr.core.SolrCore execute
> INFO: [mail] webapp=/solr path=/dataimport params={command=status} status=0
> QTime=0
>
>
>
> Thks,
>
> Matias.
>
>
>
>
>
> 2011/3/9 Peter Sturge <pe...@gmail.com>
>
>> Hi,
>>
>> You've included some output in your message, so I presume something
>> *did* happen when you ran the 'status' command (but it might not be
>> what you wanted to happen :-)
>>
>> If you run:
>> http://localhost:8983/solr/mail/dataimport?command=status
>>
>> and you get something like this back:
>> <str name="status">idle</str>
>> <str name="importResponse"/>
>> <lst name="statusMessages"/>
>>
>> It means that no full-import or delta-import has been run during the
>> life of the JVM Solr session.
>>
>> You should try running:
>> http://localhost:8983/solr/mail/dataimport?command=full-import
>>
>> Then run:
>> http://localhost:8983/solr/mail/dataimport?command=status
>>
>> to see the status of the full-import (busy, idle, error, rolled back etc.)
>>
>> You can enable java logging by editing your JRE's lib/logging.properties
>> file.
>>
>> Something like this should give you some log files:
>> handlers= java.util.logging.FileHandler
>> .level= INFO
>> java.util.logging.FileHandler.pattern = ./logs/mylogs%d.log
>> java.util.logging.FileeHandler.level = INFO
>> java.util.logging.FileHandler.limit = 500000
>> java.util.logging.FileHandler.count = 1
>> java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
>>
>> NOTE: Make sure the 'logs' folder exists (in your $cwd) before you
>> start, or you'll get an error.
>>
>> HTH
>> Peter
>>
>>
>>
>>
>> On Wed, Mar 9, 2011 at 12:47 PM, Matias Alonso <ma...@gmail.com>
>> wrote:
>> > Hi Peter,
>> >
>> > When I execute the commands you mentioned, nothing happend.
>> > Below I show you the comands executed and the answered of they.
>> > Sorry, but I don´t know how to enable the log; my jre is by default.
>> > Rememeber I´m running the example-DIH
>> (trunk\solr\example\example-DIH\solr);
>> > java -Dsolr.solr.home="./example-DIH/solr/" -jar start.jar.
>> >
>> >
>> >
>> > Import:
>> > http://localhost:8983/solr/mail/dataimport?command=status<
>> > http://localhost:8983/solr/mail/dataimport?command=full-import>
>> >
>> > <response>
>> > -
>> > <lst name="responseHeader">
>> > <int name="status">0</int>
>> > <int name="QTime">15</int>
>> > </lst>
>> > -
>> > <lst name="initArgs">
>> > -
>> > <lst name="defaults">
>> > <str name="config">data-config.xml</str>
>> > </lst>
>> > </lst>
>> > -
>> > <str name="command">
>> > full-import<
>> http://localhost:8983/solr/mail/dataimport?command=full-import>
>> > </str>
>> > <str name="status">idle</str>
>> > <str name="importResponse"/>
>> > <lst name="statusMessages"/>
>> > -
>> > <str name="WARNING">
>> > This response format is experimental. It is likely to change in the
>> future.
>> > </str>
>> > </response>
>> >
>> >
>> >
>> > Status:
>> > http://localhost:8983/solr/mail/dataimport?command=status<
>> > http://localhost:8983/solr/mail/dataimport?command=full-import>
>> >
>> >
>> > <response>
>> > -
>> > <lst name="responseHeader">
>> > <int name="status">0</int>
>> > <int name="QTime">0</int>
>> > </lst>
>> > -
>> > <lst name="initArgs">
>> > -
>> > <lst name="defaults">
>> > <str name="config">data-config.xml</str>
>> > </lst>
>> > </lst>
>> > -
>> > <str name="command">
>> > status<http://localhost:8983/solr/mail/dataimport?command=full-import>
>> > </str>
>> > <str name="status">idle</str>
>> > <str name="importResponse"/>
>> > <lst name="statusMessages"/>
>> > -
>> > <str name="WARNING">
>> > This response format is experimental. It is likely to change in the
>> future.
>> > </str>
>> > </response>
>> >
>> >
>> >
>> >
>> > Thank you for your help.
>> >
>> > Matias.
>> >
>> >
>> >
>> >
>> >
>> >
>> > 2011/3/4 Peter Sturge <pe...@gmail.com>
>> >
>> >> Can you try this:
>> >>
>> >> Issue a full import command like this:
>> >>
>> >> http://localhost:8983/solr/dataimport?command=full-import<
>> >> http://localhost:8983/solr/db/dataimport?command=full-import>
>> >>
>> >> (There is no core name here - if you're using a core name (db?), then
>> add
>> >> that in between solr/ and /dataimport)
>> >>
>> >> then, run:
>> >> http://localhost:8983/solr/dataimport?command=status<
>> >> http://localhost:8983/solr/db/dataimport?command=full-import>
>> >>
>> >> This will show the results of the previous import. Has it been
>> rolled-back?
>> >> If so, there might be something in the log if it's enabled (see your
>> jre's
>> >> lib/logging.properties file).
>> >> (you won't see any errors unless you run the status command - that's
>> where
>> >> they're stored)
>> >>
>> >> HTH
>> >> Peter
>> >>
>> >>
>> >>
>> >>
>> >> On Sat, Mar 5, 2011 at 12:46 AM, Matias Alonso <matiasgalonso@gmail.com
>> >> >wrote:
>> >>
>> >> > I´m using the trunk.
>> >> >
>> >> > Thanks Peter for your preoccupation!
>> >> >
>> >> > Matias.
>> >> >
>> >> >
>> >> >
>> >> > 2011/3/4 Peter Sturge <pe...@gmail.com>
>> >> >
>> >> > > Hi Matias,
>> >> > >
>> >> > > What version of Solr are you using? Are you running any patches
>> (maybe
>> >> > > SOLR-2245)?
>> >> > >
>> >> > > Thanks,
>> >> > > Peter
>> >> > >
>> >> > >
>> >> > >
>> >> > > On Fri, Mar 4, 2011 at 8:25 PM, Matias Alonso <
>> matiasgalonso@gmail.com
>> >> > > >wrote:
>> >> > >
>> >> > > > Hi Peter,
>> >> > > >
>> >> > > > From "DataImportHandler Development Console" I made a full-import,
>> >> but
>> >> > > > didn´t work.
>> >> > > >
>> >> > > > Now, I execute "
>> >> > > > http://localhost:8983/solr/mail/dataimport?command=full-import"
>> but
>> >> > > > nothing
>> >> > > > happends; no index; no errors.
>> >> > > >
>> >> > > > thks...
>> >> > > >
>> >> > > > Matias.
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
>> >> > > >
>> >> > > > > Hi Mataias,
>> >> > > > >
>> >> > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimportaccesses
>> >> > > > > the dataimport handler, but you need to tell it to do something
>> by
>> >> > > > > sending a command:
>> >> > > > >
>> >> > >
>> >>
>> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
>> >> > > > > ?command=full-import<
>> >> > > > > http://localhost:8983/solr/db/dataimport?command=full-import>
>> >> > > > >
>> >> > > > > If you haven't already, have a look at:
>> >> > > > >
>> >> > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> http://www.lucidimagination.com/search/out?u=http%3A%2F%2Fwiki.apache.org%2Fsolr%2FDataImportHandler
>> >> > > > >
>> >> > > > > It gives very thorough and useful advice on getting the DIH
>> >> working.
>> >> > > > >
>> >> > > > > Peter
>> >> > > > >
>> >> > > > >
>> >> > > > >
>> >> > > > > On Fri, Mar 4, 2011 at 6:59 PM, Matias Alonso <
>> >> > matiasgalonso@gmail.com
>> >> > > > > >wrote:
>> >> > > > >
>> >> > > > > > Hi Peter,
>> >> > > > > >
>> >> > > > > > I test with deltaFetch="false", but doesn´t work :(
>> >> > > > > > I'm using "DataImportHandler Development Console" to index (
>> >> > > > > >
>> >> > > >
>> >> >
>> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
>> >> > > > > );
>> >> > > > > > I'm working with "example-DIH".
>> >> > > > > >
>> >> > > > > > thks...
>> >> > > > > >
>> >> > > > > >
>> >> > > > > >
>> >> > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
>> >> > > > > >
>> >> > > > > > > Hi Matias,
>> >> > > > > > >
>> >> > > > > > > I haven't seen it in the posts, but I may have missed it --
>> >> what
>> >> > is
>> >> > > > the
>> >> > > > > > > import command you're sending?
>> >> > > > > > > Something like:
>> >> > > > > > >
>> http://localhost:8983/solr/db/dataimport?command=full-import
>> >> > > > > > >
>> >> > > > > > > Can you also test it with deltaFetch="false". I seem to
>> >> remember
>> >> > > > having
>> >> > > > > > > some
>> >> > > > > > > problems with delta in the MailEntityProcessor.
>> >> > > > > > >
>> >> > > > > > >
>> >> > > > > > >
>> >> > > > > > > On Fri, Mar 4, 2011 at 6:29 PM, Matias Alonso <
>> >> > > > matiasgalonso@gmail.com
>> >> > > > > > > >wrote:
>> >> > > > > > >
>> >> > > > > > > > <dataConfig>
>> >> > > > > > > > <document>
>> >> > > > > > > > <entity name="email"
>> >> > > > > > > > user="myemail@gmail.com"
>> >> > > > > > > > password="mypassword"
>> >> > > > > > > > host="imap.gmail.com"
>> >> > > > > > > > fetchMailsSince="2011-01-01 00:00:00"
>> >> > > > > > > > deltaFetch="true"
>> >> > > > > > > > include=""
>> >> > > > > > > > exclude=""
>> >> > > > > > > > recurse="false"
>> >> > > > > > > >
>> >> > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> folders="Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo"
>> >> > > > > > > > includeContent="true"
>> >> > > > > > > > processAttachments="false"
>> >> > > > > > > > includeOtherUserFolders="false"
>> >> > > > > > > > includeSharedFolders="false"
>> >> > > > > > > > batchSize="100"
>> >> > > > > > > > processor="MailEntityProcessor"
>> >> > > > > > > > protocol="imaps" />
>> >> > > > > > > > </document>
>> >> > > > > > > > </dataConfig>
>> >> > > > > > > >
>> >> > > > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
>> >> > > > > > > >
>> >> > > > > > > > > Hi Matias,
>> >> > > > > > > > >
>> >> > > > > > > > > Can you post your data-config.xml? (with disquised
>> >> > > > > names/credentials)
>> >> > > > > > > > >
>> >> > > > > > > > > Thanks,
>> >> > > > > > > > > Peter
>> >> > > > > > > > >
>> >> > > > > > > > >
>> >> > > > > > > > > On Fri, Mar 4, 2011 at 5:13 PM, Matias Alonso <
>> >> > > > > > matiasgalonso@gmail.com
>> >> > > > > > > > > >wrote:
>> >> > > > > > > > >
>> >> > > > > > > > > > Thks Peter,
>> >> > > > > > > > > >
>> >> > > > > > > > > > Yes, gmail gives me imaps (i understood that). So, I
>> >> tried
>> >> > > what
>> >> > > > > you
>> >> > > > > > > > > mention
>> >> > > > > > > > > > but I had get the original mesange I posted.
>> >> > > > > > > > > >
>> >> > > > > > > > > > Matias.
>> >> > > > > > > > > >
>> >> > > > > > > > > >
>> >> > > > > > > > > >
>> >> > > > > > > > > >
>> >> > > > > > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
>> >> > > > > > > > > >
>> >> > > > > > > > > > > Hi,
>> >> > > > > > > > > > >
>> >> > > > > > > > > > > You need to put your password in as well. You should
>> >> use
>> >> > > > > > > > > protocol="imap"
>> >> > > > > > > > > > > unless your gmail is set for imaps (I don't believe
>> the
>> >> > > free
>> >> > > > > > gmail
>> >> > > > > > > > > gives
>> >> > > > > > > > > > > you
>> >> > > > > > > > > > > this).
>> >> > > > > > > > > > >
>> >> > > > > > > > > > > <entity name="email"
>> >> > > > > > > > > > > user="user@mydomain.com"
>> >> > > > > > > > > > > password="userpwd"
>> >> > > > > > > > > > > host="imap.mydomain.com"
>> >> > > > > > > > > > > include=""
>> >> > > > > > > > > > > exclude=""
>> >> > > > > > > > > > > processor="MailEntityProcessor"
>> >> > > > > > > > > > > protocol="imap"
>> >> > > > > > > > > > > />
>> >> > > > > > > > > > >
>> >> > > > > > > > > > > HTH
>> >> > > > > > > > > > > Peter
>> >> > > > > > > > > > >
>> >> > > > > > > > > > >
>> >> > > > > > > > > > >
>> >> > > > > > > > > > > On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <
>> >> > > > > gora@mimirtech.com
>> >> > > > > > >
>> >> > > > > > > > > wrote:
>> >> > > > > > > > > > >
>> >> > > > > > > > > > > > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <
>> >> > > > > > > > > matiasgalonso@gmail.com
>> >> > > > > > > > > > >
>> >> > > > > > > > > > > > wrote:
>> >> > > > > > > > > > > > > Hi everyone!
>> >> > > > > > > > > > > > >
>> >> > > > > > > > > > > > >
>> >> > > > > > > > > > > > > I'm trying to index mails into solr through DHI
>> >> > (based
>> >> > > > on
>> >> > > > > > the
>> >> > > > > > > > > > > > > "example-DIH"). For this I´m using my personal
>> >> email
>> >> > > from
>> >> > > > > > > gmail,
>> >> > > > > > > > > but
>> >> > > > > > > > > > I
>> >> > > > > > > > > > > > can´t
>> >> > > > > > > > > > > > > index.
>> >> > > > > > > > > > > >
>> >> > > > > > > > > > > > Have not used the MailEntityProcessor with Gmail,
>> but
>> >> > > some
>> >> > > > > > > > > > > > points below:
>> >> > > > > > > > > > > >
>> >> > > > > > > > > > > > > Configuration in Data-config .xml:
>> >> > > > > > > > > > > > >
>> >> > > > > > > > > > > > > <dataConfig>
>> >> > > > > > > > > > > > >
>> >> > > > > > > > > > > > > <document>
>> >> > > > > > > > > > > > >
>> >> > > > > > > > > > > > > <entity name="email"
>> >> > > > > > > > > > > > >
>> >> > > > > > > > > > > > > user="mail@gmail.com"
>> >> > > > > > > > > > > > ^^^^^ I presume that you have put
>> in
>> >> > your
>> >> > > > > > actual
>> >> > > > > > > > > > > > email address here.
>> >> > > > > > > > > > > > [...]
>> >> > > > > > > > > > > > > protocol="imap"/>
>> >> > > > > > > > > > > > ^^^^^ Shouldn't this be
>> imaps,
>> >> at
>> >> > > > least
>> >> > > > > as
>> >> > > > > > > > > > > > per
>> http://wiki.apache.org/solr/MailEntityProcessor
>> >> > > > > > > > > > > >
>> >> > > > > > > > > > > > Regards,
>> >> > > > > > > > > > > > Gora
>> >> > > > > > > > > > > >
>> >> > > > > > > > > > >
>> >> > > > > > > > > >
>> >> > > > > > > > >
>> >> > > > > > > >
>> >> > > > > > >
>> >> > > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >
>>
>
Re: Help -DIH (mail)
Posted by Matias Alonso <ma...@gmail.com>.
Peter,
You´re right; may be I expose wrong because of my english.
I done everything you told me. I think that no find the "folder" when index.
What you thinking about?
Below I show to you part of the log.
09/03/2011 11:52:01 org.apache.solr.core.SolrCore execute
INFO: [mail] webapp=/solr path=/dataimport params={command=full-import}
status=0 QTime=0
09/03/2011 11:52:01 org.apache.solr.handler.dataimport.DataImporter
doFullImport
INFO: Starting Full Import
09/03/2011 11:52:01 org.apache.solr.handler.dataimport.SolrWriter
readIndexerProperties
INFO: Read dataimport.properties
09/03/2011 11:52:01 org.apache.solr.update.DirectUpdateHandler2 deleteAll
INFO: [mail] REMOVING ALL DOCUMENTS FROM INDEX
09/03/2011 11:52:01 org.apache.solr.core.SolrDeletionPolicy onInit
INFO: SolrDeletionPolicy.onInit: commits:num=1
commit{dir=D:\Search
Plugtree\trunk\solr\example\example-DIH\solr\mail\data\index,segFN=segments_c,version=1298912662799,generation=12,filenames=[segments_c]
09/03/2011 11:52:01 org.apache.solr.core.SolrDeletionPolicy updateCommits
INFO: newest commit = 1298912662799
09/03/2011 11:52:01 org.apache.solr.handler.dataimport.MailEntityProcessor
logConfig
INFO: user : myemail@gmail.com
pwd : mypass
protocol : imaps
host : imap.gmail.com
folders :
Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo,Mail,mail,MAIL
recurse : false
exclude : []
include : []
batchSize : 100
fetchSize : 32768
read timeout : 60000
conection timeout : 30000
custom filter :
fetch mail since : Thu Mar 03 00:00:00 GFT 2011
09/03/2011 11:52:03 org.apache.solr.handler.dataimport.MailEntityProcessor
connectToMailBox
INFO: Connected to mailbox
09/03/2011 11:52:03 org.apache.solr.handler.dataimport.DocBuilder finish
INFO: Import completed successfully
09/03/2011 11:52:03 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start
commit(optimize=true,waitFlush=false,waitSearcher=true,expungeDeletes=false)
09/03/2011 11:52:03 org.apache.solr.core.SolrDeletionPolicy onCommit
INFO: SolrDeletionPolicy.onCommit: commits:num=2
commit{dir=D:\Search
Plugtree\trunk\solr\example\example-DIH\solr\mail\data\index,segFN=segments_c,version=1298912662799,generation=12,filenames=[segments_c]
commit{dir=D:\Search
Plugtree\trunk\solr\example\example-DIH\solr\mail\data\index,segFN=segments_d,version=1298912662800,generation=13,filenames=[segments_d]
09/03/2011 11:52:03 org.apache.solr.core.SolrDeletionPolicy updateCommits
INFO: newest commit = 1298912662800
09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@1cee792 main
09/03/2011 11:52:03 org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@1cee792 main from Searcher@9a18a0 main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@1cee792 main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@1cee792 main from Searcher@9a18a0 main
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@1cee792 main
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@1cee792 main from Searcher@9a18a0 main
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@1cee792 main
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@1cee792 main from Searcher@9a18a0 main
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@1cee792 main
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
09/03/2011 11:52:03 org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener sending requests to Searcher@1cee792 main
09/03/2011 11:52:03 org.apache.solr.core.SolrCore execute
INFO: [mail] webapp=null path=null
params={start=0&event=newSearcher&q=solr&rows=10} hits=0 status=0 QTime=0
09/03/2011 11:52:03 org.apache.solr.core.SolrCore execute
INFO: [mail] webapp=null path=null
params={start=0&event=newSearcher&q=rocks&rows=10} hits=0 status=0 QTime=0
09/03/2011 11:52:03 org.apache.solr.core.SolrCore execute
INFO: [mail] webapp=null path=null
params={event=newSearcher&q=static+newSearcher+warming+query+from+solrconfig.xml}
hits=0 status=0 QTime=0
09/03/2011 11:52:03 org.apache.solr.core.QuerySenderListener newSearcher
INFO: QuerySenderListener done.
09/03/2011 11:52:03 org.apache.solr.core.SolrCore registerSearcher
INFO: [mail] Registered new searcher Searcher@1cee792 main
09/03/2011 11:52:03 org.apache.solr.search.SolrIndexSearcher close
INFO: Closing Searcher@9a18a0 main
fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=5,evictions=0,size=5,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=0,cumulative_hits=0,cumulative_hitratio=0.00,cumulative_inserts=0,cumulative_evictions=0}
09/03/2011 11:52:03 org.apache.solr.handler.dataimport.SolrWriter
readIndexerProperties
INFO: Read dataimport.properties
09/03/2011 11:52:03 org.apache.solr.handler.dataimport.SolrWriter persist
INFO: Wrote last indexed time to dataimport.properties
09/03/2011 11:52:03 org.apache.solr.update.processor.LogUpdateProcessor
finish
INFO: {deleteByQuery=*:*,optimize=} 0 0
09/03/2011 11:52:03 org.apache.solr.handler.dataimport.DocBuilder execute
INFO: Time taken = 0:0:2.359
09/03/2011 11:54:58 org.apache.solr.core.SolrCore execute
INFO: [mail] webapp=/solr path=/dataimport params={command=status} status=0
QTime=0
Thks,
Matias.
2011/3/9 Peter Sturge <pe...@gmail.com>
> Hi,
>
> You've included some output in your message, so I presume something
> *did* happen when you ran the 'status' command (but it might not be
> what you wanted to happen :-)
>
> If you run:
> http://localhost:8983/solr/mail/dataimport?command=status
>
> and you get something like this back:
> <str name="status">idle</str>
> <str name="importResponse"/>
> <lst name="statusMessages"/>
>
> It means that no full-import or delta-import has been run during the
> life of the JVM Solr session.
>
> You should try running:
> http://localhost:8983/solr/mail/dataimport?command=full-import
>
> Then run:
> http://localhost:8983/solr/mail/dataimport?command=status
>
> to see the status of the full-import (busy, idle, error, rolled back etc.)
>
> You can enable java logging by editing your JRE's lib/logging.properties
> file.
>
> Something like this should give you some log files:
> handlers= java.util.logging.FileHandler
> .level= INFO
> java.util.logging.FileHandler.pattern = ./logs/mylogs%d.log
> java.util.logging.FileeHandler.level = INFO
> java.util.logging.FileHandler.limit = 500000
> java.util.logging.FileHandler.count = 1
> java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
>
> NOTE: Make sure the 'logs' folder exists (in your $cwd) before you
> start, or you'll get an error.
>
> HTH
> Peter
>
>
>
>
> On Wed, Mar 9, 2011 at 12:47 PM, Matias Alonso <ma...@gmail.com>
> wrote:
> > Hi Peter,
> >
> > When I execute the commands you mentioned, nothing happend.
> > Below I show you the comands executed and the answered of they.
> > Sorry, but I don´t know how to enable the log; my jre is by default.
> > Rememeber I´m running the example-DIH
> (trunk\solr\example\example-DIH\solr);
> > java -Dsolr.solr.home="./example-DIH/solr/" -jar start.jar.
> >
> >
> >
> > Import:
> > http://localhost:8983/solr/mail/dataimport?command=status<
> > http://localhost:8983/solr/mail/dataimport?command=full-import>
> >
> > <response>
> > -
> > <lst name="responseHeader">
> > <int name="status">0</int>
> > <int name="QTime">15</int>
> > </lst>
> > -
> > <lst name="initArgs">
> > -
> > <lst name="defaults">
> > <str name="config">data-config.xml</str>
> > </lst>
> > </lst>
> > -
> > <str name="command">
> > full-import<
> http://localhost:8983/solr/mail/dataimport?command=full-import>
> > </str>
> > <str name="status">idle</str>
> > <str name="importResponse"/>
> > <lst name="statusMessages"/>
> > -
> > <str name="WARNING">
> > This response format is experimental. It is likely to change in the
> future.
> > </str>
> > </response>
> >
> >
> >
> > Status:
> > http://localhost:8983/solr/mail/dataimport?command=status<
> > http://localhost:8983/solr/mail/dataimport?command=full-import>
> >
> >
> > <response>
> > -
> > <lst name="responseHeader">
> > <int name="status">0</int>
> > <int name="QTime">0</int>
> > </lst>
> > -
> > <lst name="initArgs">
> > -
> > <lst name="defaults">
> > <str name="config">data-config.xml</str>
> > </lst>
> > </lst>
> > -
> > <str name="command">
> > status<http://localhost:8983/solr/mail/dataimport?command=full-import>
> > </str>
> > <str name="status">idle</str>
> > <str name="importResponse"/>
> > <lst name="statusMessages"/>
> > -
> > <str name="WARNING">
> > This response format is experimental. It is likely to change in the
> future.
> > </str>
> > </response>
> >
> >
> >
> >
> > Thank you for your help.
> >
> > Matias.
> >
> >
> >
> >
> >
> >
> > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >
> >> Can you try this:
> >>
> >> Issue a full import command like this:
> >>
> >> http://localhost:8983/solr/dataimport?command=full-import<
> >> http://localhost:8983/solr/db/dataimport?command=full-import>
> >>
> >> (There is no core name here - if you're using a core name (db?), then
> add
> >> that in between solr/ and /dataimport)
> >>
> >> then, run:
> >> http://localhost:8983/solr/dataimport?command=status<
> >> http://localhost:8983/solr/db/dataimport?command=full-import>
> >>
> >> This will show the results of the previous import. Has it been
> rolled-back?
> >> If so, there might be something in the log if it's enabled (see your
> jre's
> >> lib/logging.properties file).
> >> (you won't see any errors unless you run the status command - that's
> where
> >> they're stored)
> >>
> >> HTH
> >> Peter
> >>
> >>
> >>
> >>
> >> On Sat, Mar 5, 2011 at 12:46 AM, Matias Alonso <matiasgalonso@gmail.com
> >> >wrote:
> >>
> >> > I´m using the trunk.
> >> >
> >> > Thanks Peter for your preoccupation!
> >> >
> >> > Matias.
> >> >
> >> >
> >> >
> >> > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >> >
> >> > > Hi Matias,
> >> > >
> >> > > What version of Solr are you using? Are you running any patches
> (maybe
> >> > > SOLR-2245)?
> >> > >
> >> > > Thanks,
> >> > > Peter
> >> > >
> >> > >
> >> > >
> >> > > On Fri, Mar 4, 2011 at 8:25 PM, Matias Alonso <
> matiasgalonso@gmail.com
> >> > > >wrote:
> >> > >
> >> > > > Hi Peter,
> >> > > >
> >> > > > From "DataImportHandler Development Console" I made a full-import,
> >> but
> >> > > > didn´t work.
> >> > > >
> >> > > > Now, I execute "
> >> > > > http://localhost:8983/solr/mail/dataimport?command=full-import"
> but
> >> > > > nothing
> >> > > > happends; no index; no errors.
> >> > > >
> >> > > > thks...
> >> > > >
> >> > > > Matias.
> >> > > >
> >> > > >
> >> > > >
> >> > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >> > > >
> >> > > > > Hi Mataias,
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimportaccesses
> >> > > > > the dataimport handler, but you need to tell it to do something
> by
> >> > > > > sending a command:
> >> > > > >
> >> > >
> >>
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
> >> > > > > ?command=full-import<
> >> > > > > http://localhost:8983/solr/db/dataimport?command=full-import>
> >> > > > >
> >> > > > > If you haven't already, have a look at:
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://www.lucidimagination.com/search/out?u=http%3A%2F%2Fwiki.apache.org%2Fsolr%2FDataImportHandler
> >> > > > >
> >> > > > > It gives very thorough and useful advice on getting the DIH
> >> working.
> >> > > > >
> >> > > > > Peter
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > On Fri, Mar 4, 2011 at 6:59 PM, Matias Alonso <
> >> > matiasgalonso@gmail.com
> >> > > > > >wrote:
> >> > > > >
> >> > > > > > Hi Peter,
> >> > > > > >
> >> > > > > > I test with deltaFetch="false", but doesn´t work :(
> >> > > > > > I'm using "DataImportHandler Development Console" to index (
> >> > > > > >
> >> > > >
> >> >
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
> >> > > > > );
> >> > > > > > I'm working with "example-DIH".
> >> > > > > >
> >> > > > > > thks...
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >> > > > > >
> >> > > > > > > Hi Matias,
> >> > > > > > >
> >> > > > > > > I haven't seen it in the posts, but I may have missed it --
> >> what
> >> > is
> >> > > > the
> >> > > > > > > import command you're sending?
> >> > > > > > > Something like:
> >> > > > > > >
> http://localhost:8983/solr/db/dataimport?command=full-import
> >> > > > > > >
> >> > > > > > > Can you also test it with deltaFetch="false". I seem to
> >> remember
> >> > > > having
> >> > > > > > > some
> >> > > > > > > problems with delta in the MailEntityProcessor.
> >> > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Fri, Mar 4, 2011 at 6:29 PM, Matias Alonso <
> >> > > > matiasgalonso@gmail.com
> >> > > > > > > >wrote:
> >> > > > > > >
> >> > > > > > > > <dataConfig>
> >> > > > > > > > <document>
> >> > > > > > > > <entity name="email"
> >> > > > > > > > user="myemail@gmail.com"
> >> > > > > > > > password="mypassword"
> >> > > > > > > > host="imap.gmail.com"
> >> > > > > > > > fetchMailsSince="2011-01-01 00:00:00"
> >> > > > > > > > deltaFetch="true"
> >> > > > > > > > include=""
> >> > > > > > > > exclude=""
> >> > > > > > > > recurse="false"
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> folders="Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo"
> >> > > > > > > > includeContent="true"
> >> > > > > > > > processAttachments="false"
> >> > > > > > > > includeOtherUserFolders="false"
> >> > > > > > > > includeSharedFolders="false"
> >> > > > > > > > batchSize="100"
> >> > > > > > > > processor="MailEntityProcessor"
> >> > > > > > > > protocol="imaps" />
> >> > > > > > > > </document>
> >> > > > > > > > </dataConfig>
> >> > > > > > > >
> >> > > > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >> > > > > > > >
> >> > > > > > > > > Hi Matias,
> >> > > > > > > > >
> >> > > > > > > > > Can you post your data-config.xml? (with disquised
> >> > > > > names/credentials)
> >> > > > > > > > >
> >> > > > > > > > > Thanks,
> >> > > > > > > > > Peter
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > On Fri, Mar 4, 2011 at 5:13 PM, Matias Alonso <
> >> > > > > > matiasgalonso@gmail.com
> >> > > > > > > > > >wrote:
> >> > > > > > > > >
> >> > > > > > > > > > Thks Peter,
> >> > > > > > > > > >
> >> > > > > > > > > > Yes, gmail gives me imaps (i understood that). So, I
> >> tried
> >> > > what
> >> > > > > you
> >> > > > > > > > > mention
> >> > > > > > > > > > but I had get the original mesange I posted.
> >> > > > > > > > > >
> >> > > > > > > > > > Matias.
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >> > > > > > > > > >
> >> > > > > > > > > > > Hi,
> >> > > > > > > > > > >
> >> > > > > > > > > > > You need to put your password in as well. You should
> >> use
> >> > > > > > > > > protocol="imap"
> >> > > > > > > > > > > unless your gmail is set for imaps (I don't believe
> the
> >> > > free
> >> > > > > > gmail
> >> > > > > > > > > gives
> >> > > > > > > > > > > you
> >> > > > > > > > > > > this).
> >> > > > > > > > > > >
> >> > > > > > > > > > > <entity name="email"
> >> > > > > > > > > > > user="user@mydomain.com"
> >> > > > > > > > > > > password="userpwd"
> >> > > > > > > > > > > host="imap.mydomain.com"
> >> > > > > > > > > > > include=""
> >> > > > > > > > > > > exclude=""
> >> > > > > > > > > > > processor="MailEntityProcessor"
> >> > > > > > > > > > > protocol="imap"
> >> > > > > > > > > > > />
> >> > > > > > > > > > >
> >> > > > > > > > > > > HTH
> >> > > > > > > > > > > Peter
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <
> >> > > > > gora@mimirtech.com
> >> > > > > > >
> >> > > > > > > > > wrote:
> >> > > > > > > > > > >
> >> > > > > > > > > > > > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <
> >> > > > > > > > > matiasgalonso@gmail.com
> >> > > > > > > > > > >
> >> > > > > > > > > > > > wrote:
> >> > > > > > > > > > > > > Hi everyone!
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > I'm trying to index mails into solr through DHI
> >> > (based
> >> > > > on
> >> > > > > > the
> >> > > > > > > > > > > > > "example-DIH"). For this I´m using my personal
> >> email
> >> > > from
> >> > > > > > > gmail,
> >> > > > > > > > > but
> >> > > > > > > > > > I
> >> > > > > > > > > > > > can´t
> >> > > > > > > > > > > > > index.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Have not used the MailEntityProcessor with Gmail,
> but
> >> > > some
> >> > > > > > > > > > > > points below:
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > > Configuration in Data-config .xml:
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > <dataConfig>
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > <document>
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > <entity name="email"
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > user="mail@gmail.com"
> >> > > > > > > > > > > > ^^^^^ I presume that you have put
> in
> >> > your
> >> > > > > > actual
> >> > > > > > > > > > > > email address here.
> >> > > > > > > > > > > > [...]
> >> > > > > > > > > > > > > protocol="imap"/>
> >> > > > > > > > > > > > ^^^^^ Shouldn't this be
> imaps,
> >> at
> >> > > > least
> >> > > > > as
> >> > > > > > > > > > > > per
> http://wiki.apache.org/solr/MailEntityProcessor
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Regards,
> >> > > > > > > > > > > > Gora
> >> > > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
Re: Help -DIH (mail)
Posted by Peter Sturge <pe...@gmail.com>.
Hi,
You've included some output in your message, so I presume something
*did* happen when you ran the 'status' command (but it might not be
what you wanted to happen :-)
If you run:
http://localhost:8983/solr/mail/dataimport?command=status
and you get something like this back:
<str name="status">idle</str>
<str name="importResponse"/>
<lst name="statusMessages"/>
It means that no full-import or delta-import has been run during the
life of the JVM Solr session.
You should try running:
http://localhost:8983/solr/mail/dataimport?command=full-import
Then run:
http://localhost:8983/solr/mail/dataimport?command=status
to see the status of the full-import (busy, idle, error, rolled back etc.)
You can enable java logging by editing your JRE's lib/logging.properties file.
Something like this should give you some log files:
handlers= java.util.logging.FileHandler
.level= INFO
java.util.logging.FileHandler.pattern = ./logs/mylogs%d.log
java.util.logging.FileeHandler.level = INFO
java.util.logging.FileHandler.limit = 500000
java.util.logging.FileHandler.count = 1
java.util.logging.FileHandler.formatter = java.util.logging.SimpleFormatter
NOTE: Make sure the 'logs' folder exists (in your $cwd) before you
start, or you'll get an error.
HTH
Peter
On Wed, Mar 9, 2011 at 12:47 PM, Matias Alonso <ma...@gmail.com> wrote:
> Hi Peter,
>
> When I execute the commands you mentioned, nothing happend.
> Below I show you the comands executed and the answered of they.
> Sorry, but I don´t know how to enable the log; my jre is by default.
> Rememeber I´m running the example-DIH (trunk\solr\example\example-DIH\solr);
> java -Dsolr.solr.home="./example-DIH/solr/" -jar start.jar.
>
>
>
> Import:
> http://localhost:8983/solr/mail/dataimport?command=status<
> http://localhost:8983/solr/mail/dataimport?command=full-import>
>
> <response>
> -
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">15</int>
> </lst>
> -
> <lst name="initArgs">
> -
> <lst name="defaults">
> <str name="config">data-config.xml</str>
> </lst>
> </lst>
> -
> <str name="command">
> full-import<http://localhost:8983/solr/mail/dataimport?command=full-import>
> </str>
> <str name="status">idle</str>
> <str name="importResponse"/>
> <lst name="statusMessages"/>
> -
> <str name="WARNING">
> This response format is experimental. It is likely to change in the future.
> </str>
> </response>
>
>
>
> Status:
> http://localhost:8983/solr/mail/dataimport?command=status<
> http://localhost:8983/solr/mail/dataimport?command=full-import>
>
>
> <response>
> -
> <lst name="responseHeader">
> <int name="status">0</int>
> <int name="QTime">0</int>
> </lst>
> -
> <lst name="initArgs">
> -
> <lst name="defaults">
> <str name="config">data-config.xml</str>
> </lst>
> </lst>
> -
> <str name="command">
> status<http://localhost:8983/solr/mail/dataimport?command=full-import>
> </str>
> <str name="status">idle</str>
> <str name="importResponse"/>
> <lst name="statusMessages"/>
> -
> <str name="WARNING">
> This response format is experimental. It is likely to change in the future.
> </str>
> </response>
>
>
>
>
> Thank you for your help.
>
> Matias.
>
>
>
>
>
>
> 2011/3/4 Peter Sturge <pe...@gmail.com>
>
>> Can you try this:
>>
>> Issue a full import command like this:
>>
>> http://localhost:8983/solr/dataimport?command=full-import<
>> http://localhost:8983/solr/db/dataimport?command=full-import>
>>
>> (There is no core name here - if you're using a core name (db?), then add
>> that in between solr/ and /dataimport)
>>
>> then, run:
>> http://localhost:8983/solr/dataimport?command=status<
>> http://localhost:8983/solr/db/dataimport?command=full-import>
>>
>> This will show the results of the previous import. Has it been rolled-back?
>> If so, there might be something in the log if it's enabled (see your jre's
>> lib/logging.properties file).
>> (you won't see any errors unless you run the status command - that's where
>> they're stored)
>>
>> HTH
>> Peter
>>
>>
>>
>>
>> On Sat, Mar 5, 2011 at 12:46 AM, Matias Alonso <matiasgalonso@gmail.com
>> >wrote:
>>
>> > I´m using the trunk.
>> >
>> > Thanks Peter for your preoccupation!
>> >
>> > Matias.
>> >
>> >
>> >
>> > 2011/3/4 Peter Sturge <pe...@gmail.com>
>> >
>> > > Hi Matias,
>> > >
>> > > What version of Solr are you using? Are you running any patches (maybe
>> > > SOLR-2245)?
>> > >
>> > > Thanks,
>> > > Peter
>> > >
>> > >
>> > >
>> > > On Fri, Mar 4, 2011 at 8:25 PM, Matias Alonso <matiasgalonso@gmail.com
>> > > >wrote:
>> > >
>> > > > Hi Peter,
>> > > >
>> > > > From "DataImportHandler Development Console" I made a full-import,
>> but
>> > > > didn´t work.
>> > > >
>> > > > Now, I execute "
>> > > > http://localhost:8983/solr/mail/dataimport?command=full-import" but
>> > > > nothing
>> > > > happends; no index; no errors.
>> > > >
>> > > > thks...
>> > > >
>> > > > Matias.
>> > > >
>> > > >
>> > > >
>> > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
>> > > >
>> > > > > Hi Mataias,
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimportaccesses
>> > > > > the dataimport handler, but you need to tell it to do something by
>> > > > > sending a command:
>> > > > >
>> > >
>> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
>> > > > > ?command=full-import<
>> > > > > http://localhost:8983/solr/db/dataimport?command=full-import>
>> > > > >
>> > > > > If you haven't already, have a look at:
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://www.lucidimagination.com/search/out?u=http%3A%2F%2Fwiki.apache.org%2Fsolr%2FDataImportHandler
>> > > > >
>> > > > > It gives very thorough and useful advice on getting the DIH
>> working.
>> > > > >
>> > > > > Peter
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Fri, Mar 4, 2011 at 6:59 PM, Matias Alonso <
>> > matiasgalonso@gmail.com
>> > > > > >wrote:
>> > > > >
>> > > > > > Hi Peter,
>> > > > > >
>> > > > > > I test with deltaFetch="false", but doesn´t work :(
>> > > > > > I'm using "DataImportHandler Development Console" to index (
>> > > > > >
>> > > >
>> > http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
>> > > > > );
>> > > > > > I'm working with "example-DIH".
>> > > > > >
>> > > > > > thks...
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
>> > > > > >
>> > > > > > > Hi Matias,
>> > > > > > >
>> > > > > > > I haven't seen it in the posts, but I may have missed it --
>> what
>> > is
>> > > > the
>> > > > > > > import command you're sending?
>> > > > > > > Something like:
>> > > > > > > http://localhost:8983/solr/db/dataimport?command=full-import
>> > > > > > >
>> > > > > > > Can you also test it with deltaFetch="false". I seem to
>> remember
>> > > > having
>> > > > > > > some
>> > > > > > > problems with delta in the MailEntityProcessor.
>> > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > > > On Fri, Mar 4, 2011 at 6:29 PM, Matias Alonso <
>> > > > matiasgalonso@gmail.com
>> > > > > > > >wrote:
>> > > > > > >
>> > > > > > > > <dataConfig>
>> > > > > > > > <document>
>> > > > > > > > <entity name="email"
>> > > > > > > > user="myemail@gmail.com"
>> > > > > > > > password="mypassword"
>> > > > > > > > host="imap.gmail.com"
>> > > > > > > > fetchMailsSince="2011-01-01 00:00:00"
>> > > > > > > > deltaFetch="true"
>> > > > > > > > include=""
>> > > > > > > > exclude=""
>> > > > > > > > recurse="false"
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> folders="Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo"
>> > > > > > > > includeContent="true"
>> > > > > > > > processAttachments="false"
>> > > > > > > > includeOtherUserFolders="false"
>> > > > > > > > includeSharedFolders="false"
>> > > > > > > > batchSize="100"
>> > > > > > > > processor="MailEntityProcessor"
>> > > > > > > > protocol="imaps" />
>> > > > > > > > </document>
>> > > > > > > > </dataConfig>
>> > > > > > > >
>> > > > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
>> > > > > > > >
>> > > > > > > > > Hi Matias,
>> > > > > > > > >
>> > > > > > > > > Can you post your data-config.xml? (with disquised
>> > > > > names/credentials)
>> > > > > > > > >
>> > > > > > > > > Thanks,
>> > > > > > > > > Peter
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Fri, Mar 4, 2011 at 5:13 PM, Matias Alonso <
>> > > > > > matiasgalonso@gmail.com
>> > > > > > > > > >wrote:
>> > > > > > > > >
>> > > > > > > > > > Thks Peter,
>> > > > > > > > > >
>> > > > > > > > > > Yes, gmail gives me imaps (i understood that). So, I
>> tried
>> > > what
>> > > > > you
>> > > > > > > > > mention
>> > > > > > > > > > but I had get the original mesange I posted.
>> > > > > > > > > >
>> > > > > > > > > > Matias.
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
>> > > > > > > > > >
>> > > > > > > > > > > Hi,
>> > > > > > > > > > >
>> > > > > > > > > > > You need to put your password in as well. You should
>> use
>> > > > > > > > > protocol="imap"
>> > > > > > > > > > > unless your gmail is set for imaps (I don't believe the
>> > > free
>> > > > > > gmail
>> > > > > > > > > gives
>> > > > > > > > > > > you
>> > > > > > > > > > > this).
>> > > > > > > > > > >
>> > > > > > > > > > > <entity name="email"
>> > > > > > > > > > > user="user@mydomain.com"
>> > > > > > > > > > > password="userpwd"
>> > > > > > > > > > > host="imap.mydomain.com"
>> > > > > > > > > > > include=""
>> > > > > > > > > > > exclude=""
>> > > > > > > > > > > processor="MailEntityProcessor"
>> > > > > > > > > > > protocol="imap"
>> > > > > > > > > > > />
>> > > > > > > > > > >
>> > > > > > > > > > > HTH
>> > > > > > > > > > > Peter
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <
>> > > > > gora@mimirtech.com
>> > > > > > >
>> > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <
>> > > > > > > > > matiasgalonso@gmail.com
>> > > > > > > > > > >
>> > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > Hi everyone!
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > I'm trying to index mails into solr through DHI
>> > (based
>> > > > on
>> > > > > > the
>> > > > > > > > > > > > > "example-DIH"). For this I´m using my personal
>> email
>> > > from
>> > > > > > > gmail,
>> > > > > > > > > but
>> > > > > > > > > > I
>> > > > > > > > > > > > can´t
>> > > > > > > > > > > > > index.
>> > > > > > > > > > > >
>> > > > > > > > > > > > Have not used the MailEntityProcessor with Gmail, but
>> > > some
>> > > > > > > > > > > > points below:
>> > > > > > > > > > > >
>> > > > > > > > > > > > > Configuration in Data-config .xml:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > <dataConfig>
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > <document>
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > <entity name="email"
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > user="mail@gmail.com"
>> > > > > > > > > > > > ^^^^^ I presume that you have put in
>> > your
>> > > > > > actual
>> > > > > > > > > > > > email address here.
>> > > > > > > > > > > > [...]
>> > > > > > > > > > > > > protocol="imap"/>
>> > > > > > > > > > > > ^^^^^ Shouldn't this be imaps,
>> at
>> > > > least
>> > > > > as
>> > > > > > > > > > > > per http://wiki.apache.org/solr/MailEntityProcessor
>> > > > > > > > > > > >
>> > > > > > > > > > > > Regards,
>> > > > > > > > > > > > Gora
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: Help -DIH (mail)
Posted by Matias Alonso <ma...@gmail.com>.
Hi Peter,
When I execute the commands you mentioned, nothing happend.
Below I show you the comands executed and the answered of they.
Sorry, but I don´t know how to enable the log; my jre is by default.
Rememeber I´m running the example-DIH (trunk\solr\example\example-DIH\solr);
java -Dsolr.solr.home="./example-DIH/solr/" -jar start.jar.
Import:
http://localhost:8983/solr/mail/dataimport?command=status<
http://localhost:8983/solr/mail/dataimport?command=full-import>
<response>
-
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">15</int>
</lst>
-
<lst name="initArgs">
-
<lst name="defaults">
<str name="config">data-config.xml</str>
</lst>
</lst>
-
<str name="command">
full-import<http://localhost:8983/solr/mail/dataimport?command=full-import>
</str>
<str name="status">idle</str>
<str name="importResponse"/>
<lst name="statusMessages"/>
-
<str name="WARNING">
This response format is experimental. It is likely to change in the future.
</str>
</response>
Status:
http://localhost:8983/solr/mail/dataimport?command=status<
http://localhost:8983/solr/mail/dataimport?command=full-import>
<response>
-
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">0</int>
</lst>
-
<lst name="initArgs">
-
<lst name="defaults">
<str name="config">data-config.xml</str>
</lst>
</lst>
-
<str name="command">
status<http://localhost:8983/solr/mail/dataimport?command=full-import>
</str>
<str name="status">idle</str>
<str name="importResponse"/>
<lst name="statusMessages"/>
-
<str name="WARNING">
This response format is experimental. It is likely to change in the future.
</str>
</response>
Thank you for your help.
Matias.
2011/3/4 Peter Sturge <pe...@gmail.com>
> Can you try this:
>
> Issue a full import command like this:
>
> http://localhost:8983/solr/dataimport?command=full-import<
> http://localhost:8983/solr/db/dataimport?command=full-import>
>
> (There is no core name here - if you're using a core name (db?), then add
> that in between solr/ and /dataimport)
>
> then, run:
> http://localhost:8983/solr/dataimport?command=status<
> http://localhost:8983/solr/db/dataimport?command=full-import>
>
> This will show the results of the previous import. Has it been rolled-back?
> If so, there might be something in the log if it's enabled (see your jre's
> lib/logging.properties file).
> (you won't see any errors unless you run the status command - that's where
> they're stored)
>
> HTH
> Peter
>
>
>
>
> On Sat, Mar 5, 2011 at 12:46 AM, Matias Alonso <matiasgalonso@gmail.com
> >wrote:
>
> > I´m using the trunk.
> >
> > Thanks Peter for your preoccupation!
> >
> > Matias.
> >
> >
> >
> > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >
> > > Hi Matias,
> > >
> > > What version of Solr are you using? Are you running any patches (maybe
> > > SOLR-2245)?
> > >
> > > Thanks,
> > > Peter
> > >
> > >
> > >
> > > On Fri, Mar 4, 2011 at 8:25 PM, Matias Alonso <matiasgalonso@gmail.com
> > > >wrote:
> > >
> > > > Hi Peter,
> > > >
> > > > From "DataImportHandler Development Console" I made a full-import,
> but
> > > > didn´t work.
> > > >
> > > > Now, I execute "
> > > > http://localhost:8983/solr/mail/dataimport?command=full-import" but
> > > > nothing
> > > > happends; no index; no errors.
> > > >
> > > > thks...
> > > >
> > > > Matias.
> > > >
> > > >
> > > >
> > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > >
> > > > > Hi Mataias,
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimportaccesses
> > > > > the dataimport handler, but you need to tell it to do something by
> > > > > sending a command:
> > > > >
> > >
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
> > > > > ?command=full-import<
> > > > > http://localhost:8983/solr/db/dataimport?command=full-import>
> > > > >
> > > > > If you haven't already, have a look at:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> http://www.lucidimagination.com/search/out?u=http%3A%2F%2Fwiki.apache.org%2Fsolr%2FDataImportHandler
> > > > >
> > > > > It gives very thorough and useful advice on getting the DIH
> working.
> > > > >
> > > > > Peter
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Mar 4, 2011 at 6:59 PM, Matias Alonso <
> > matiasgalonso@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Hi Peter,
> > > > > >
> > > > > > I test with deltaFetch="false", but doesn´t work :(
> > > > > > I'm using "DataImportHandler Development Console" to index (
> > > > > >
> > > >
> > http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
> > > > > );
> > > > > > I'm working with "example-DIH".
> > > > > >
> > > > > > thks...
> > > > > >
> > > > > >
> > > > > >
> > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > > > >
> > > > > > > Hi Matias,
> > > > > > >
> > > > > > > I haven't seen it in the posts, but I may have missed it --
> what
> > is
> > > > the
> > > > > > > import command you're sending?
> > > > > > > Something like:
> > > > > > > http://localhost:8983/solr/db/dataimport?command=full-import
> > > > > > >
> > > > > > > Can you also test it with deltaFetch="false". I seem to
> remember
> > > > having
> > > > > > > some
> > > > > > > problems with delta in the MailEntityProcessor.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Mar 4, 2011 at 6:29 PM, Matias Alonso <
> > > > matiasgalonso@gmail.com
> > > > > > > >wrote:
> > > > > > >
> > > > > > > > <dataConfig>
> > > > > > > > <document>
> > > > > > > > <entity name="email"
> > > > > > > > user="myemail@gmail.com"
> > > > > > > > password="mypassword"
> > > > > > > > host="imap.gmail.com"
> > > > > > > > fetchMailsSince="2011-01-01 00:00:00"
> > > > > > > > deltaFetch="true"
> > > > > > > > include=""
> > > > > > > > exclude=""
> > > > > > > > recurse="false"
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> folders="Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo"
> > > > > > > > includeContent="true"
> > > > > > > > processAttachments="false"
> > > > > > > > includeOtherUserFolders="false"
> > > > > > > > includeSharedFolders="false"
> > > > > > > > batchSize="100"
> > > > > > > > processor="MailEntityProcessor"
> > > > > > > > protocol="imaps" />
> > > > > > > > </document>
> > > > > > > > </dataConfig>
> > > > > > > >
> > > > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > > > > > >
> > > > > > > > > Hi Matias,
> > > > > > > > >
> > > > > > > > > Can you post your data-config.xml? (with disquised
> > > > > names/credentials)
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Peter
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Mar 4, 2011 at 5:13 PM, Matias Alonso <
> > > > > > matiasgalonso@gmail.com
> > > > > > > > > >wrote:
> > > > > > > > >
> > > > > > > > > > Thks Peter,
> > > > > > > > > >
> > > > > > > > > > Yes, gmail gives me imaps (i understood that). So, I
> tried
> > > what
> > > > > you
> > > > > > > > > mention
> > > > > > > > > > but I had get the original mesange I posted.
> > > > > > > > > >
> > > > > > > > > > Matias.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > > > > > > > >
> > > > > > > > > > > Hi,
> > > > > > > > > > >
> > > > > > > > > > > You need to put your password in as well. You should
> use
> > > > > > > > > protocol="imap"
> > > > > > > > > > > unless your gmail is set for imaps (I don't believe the
> > > free
> > > > > > gmail
> > > > > > > > > gives
> > > > > > > > > > > you
> > > > > > > > > > > this).
> > > > > > > > > > >
> > > > > > > > > > > <entity name="email"
> > > > > > > > > > > user="user@mydomain.com"
> > > > > > > > > > > password="userpwd"
> > > > > > > > > > > host="imap.mydomain.com"
> > > > > > > > > > > include=""
> > > > > > > > > > > exclude=""
> > > > > > > > > > > processor="MailEntityProcessor"
> > > > > > > > > > > protocol="imap"
> > > > > > > > > > > />
> > > > > > > > > > >
> > > > > > > > > > > HTH
> > > > > > > > > > > Peter
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <
> > > > > gora@mimirtech.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <
> > > > > > > > > matiasgalonso@gmail.com
> > > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > > Hi everyone!
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > I'm trying to index mails into solr through DHI
> > (based
> > > > on
> > > > > > the
> > > > > > > > > > > > > "example-DIH"). For this I´m using my personal
> email
> > > from
> > > > > > > gmail,
> > > > > > > > > but
> > > > > > > > > > I
> > > > > > > > > > > > can´t
> > > > > > > > > > > > > index.
> > > > > > > > > > > >
> > > > > > > > > > > > Have not used the MailEntityProcessor with Gmail, but
> > > some
> > > > > > > > > > > > points below:
> > > > > > > > > > > >
> > > > > > > > > > > > > Configuration in Data-config .xml:
> > > > > > > > > > > > >
> > > > > > > > > > > > > <dataConfig>
> > > > > > > > > > > > >
> > > > > > > > > > > > > <document>
> > > > > > > > > > > > >
> > > > > > > > > > > > > <entity name="email"
> > > > > > > > > > > > >
> > > > > > > > > > > > > user="mail@gmail.com"
> > > > > > > > > > > > ^^^^^ I presume that you have put in
> > your
> > > > > > actual
> > > > > > > > > > > > email address here.
> > > > > > > > > > > > [...]
> > > > > > > > > > > > > protocol="imap"/>
> > > > > > > > > > > > ^^^^^ Shouldn't this be imaps,
> at
> > > > least
> > > > > as
> > > > > > > > > > > > per http://wiki.apache.org/solr/MailEntityProcessor
> > > > > > > > > > > >
> > > > > > > > > > > > Regards,
> > > > > > > > > > > > Gora
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Help -DIH (mail)
Posted by Peter Sturge <pe...@gmail.com>.
Can you try this:
Issue a full import command like this:
http://localhost:8983/solr/dataimport?command=full-import<http://localhost:8983/solr/db/dataimport?command=full-import>
(There is no core name here - if you're using a core name (db?), then add
that in between solr/ and /dataimport)
then, run:
http://localhost:8983/solr/dataimport?command=status<http://localhost:8983/solr/db/dataimport?command=full-import>
This will show the results of the previous import. Has it been rolled-back?
If so, there might be something in the log if it's enabled (see your jre's
lib/logging.properties file).
(you won't see any errors unless you run the status command - that's where
they're stored)
HTH
Peter
On Sat, Mar 5, 2011 at 12:46 AM, Matias Alonso <ma...@gmail.com>wrote:
> I´m using the trunk.
>
> Thanks Peter for your preoccupation!
>
> Matias.
>
>
>
> 2011/3/4 Peter Sturge <pe...@gmail.com>
>
> > Hi Matias,
> >
> > What version of Solr are you using? Are you running any patches (maybe
> > SOLR-2245)?
> >
> > Thanks,
> > Peter
> >
> >
> >
> > On Fri, Mar 4, 2011 at 8:25 PM, Matias Alonso <matiasgalonso@gmail.com
> > >wrote:
> >
> > > Hi Peter,
> > >
> > > From "DataImportHandler Development Console" I made a full-import, but
> > > didn´t work.
> > >
> > > Now, I execute "
> > > http://localhost:8983/solr/mail/dataimport?command=full-import" but
> > > nothing
> > > happends; no index; no errors.
> > >
> > > thks...
> > >
> > > Matias.
> > >
> > >
> > >
> > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > >
> > > > Hi Mataias,
> > > >
> > > >
> > > >
> > >
> >
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimportaccesses
> > > > the dataimport handler, but you need to tell it to do something by
> > > > sending a command:
> > > >
> > http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
> > > > ?command=full-import<
> > > > http://localhost:8983/solr/db/dataimport?command=full-import>
> > > >
> > > > If you haven't already, have a look at:
> > > >
> > > >
> > > >
> > >
> >
> http://www.lucidimagination.com/search/out?u=http%3A%2F%2Fwiki.apache.org%2Fsolr%2FDataImportHandler
> > > >
> > > > It gives very thorough and useful advice on getting the DIH working.
> > > >
> > > > Peter
> > > >
> > > >
> > > >
> > > > On Fri, Mar 4, 2011 at 6:59 PM, Matias Alonso <
> matiasgalonso@gmail.com
> > > > >wrote:
> > > >
> > > > > Hi Peter,
> > > > >
> > > > > I test with deltaFetch="false", but doesn´t work :(
> > > > > I'm using "DataImportHandler Development Console" to index (
> > > > >
> > >
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
> > > > );
> > > > > I'm working with "example-DIH".
> > > > >
> > > > > thks...
> > > > >
> > > > >
> > > > >
> > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > > >
> > > > > > Hi Matias,
> > > > > >
> > > > > > I haven't seen it in the posts, but I may have missed it -- what
> is
> > > the
> > > > > > import command you're sending?
> > > > > > Something like:
> > > > > > http://localhost:8983/solr/db/dataimport?command=full-import
> > > > > >
> > > > > > Can you also test it with deltaFetch="false". I seem to remember
> > > having
> > > > > > some
> > > > > > problems with delta in the MailEntityProcessor.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Fri, Mar 4, 2011 at 6:29 PM, Matias Alonso <
> > > matiasgalonso@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > <dataConfig>
> > > > > > > <document>
> > > > > > > <entity name="email"
> > > > > > > user="myemail@gmail.com"
> > > > > > > password="mypassword"
> > > > > > > host="imap.gmail.com"
> > > > > > > fetchMailsSince="2011-01-01 00:00:00"
> > > > > > > deltaFetch="true"
> > > > > > > include=""
> > > > > > > exclude=""
> > > > > > > recurse="false"
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> folders="Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo"
> > > > > > > includeContent="true"
> > > > > > > processAttachments="false"
> > > > > > > includeOtherUserFolders="false"
> > > > > > > includeSharedFolders="false"
> > > > > > > batchSize="100"
> > > > > > > processor="MailEntityProcessor"
> > > > > > > protocol="imaps" />
> > > > > > > </document>
> > > > > > > </dataConfig>
> > > > > > >
> > > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > > > > >
> > > > > > > > Hi Matias,
> > > > > > > >
> > > > > > > > Can you post your data-config.xml? (with disquised
> > > > names/credentials)
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Peter
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Mar 4, 2011 at 5:13 PM, Matias Alonso <
> > > > > matiasgalonso@gmail.com
> > > > > > > > >wrote:
> > > > > > > >
> > > > > > > > > Thks Peter,
> > > > > > > > >
> > > > > > > > > Yes, gmail gives me imaps (i understood that). So, I tried
> > what
> > > > you
> > > > > > > > mention
> > > > > > > > > but I had get the original mesange I posted.
> > > > > > > > >
> > > > > > > > > Matias.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > You need to put your password in as well. You should use
> > > > > > > > protocol="imap"
> > > > > > > > > > unless your gmail is set for imaps (I don't believe the
> > free
> > > > > gmail
> > > > > > > > gives
> > > > > > > > > > you
> > > > > > > > > > this).
> > > > > > > > > >
> > > > > > > > > > <entity name="email"
> > > > > > > > > > user="user@mydomain.com"
> > > > > > > > > > password="userpwd"
> > > > > > > > > > host="imap.mydomain.com"
> > > > > > > > > > include=""
> > > > > > > > > > exclude=""
> > > > > > > > > > processor="MailEntityProcessor"
> > > > > > > > > > protocol="imap"
> > > > > > > > > > />
> > > > > > > > > >
> > > > > > > > > > HTH
> > > > > > > > > > Peter
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <
> > > > gora@mimirtech.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <
> > > > > > > > matiasgalonso@gmail.com
> > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > > > Hi everyone!
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > I’m trying to index mails into solr through DHI
> (based
> > > on
> > > > > the
> > > > > > > > > > > > “example-DIH”). For this I´m using my personal email
> > from
> > > > > > gmail,
> > > > > > > > but
> > > > > > > > > I
> > > > > > > > > > > can´t
> > > > > > > > > > > > index.
> > > > > > > > > > >
> > > > > > > > > > > Have not used the MailEntityProcessor with Gmail, but
> > some
> > > > > > > > > > > points below:
> > > > > > > > > > >
> > > > > > > > > > > > Configuration in Data-config .xml:
> > > > > > > > > > > >
> > > > > > > > > > > > <dataConfig>
> > > > > > > > > > > >
> > > > > > > > > > > > <document>
> > > > > > > > > > > >
> > > > > > > > > > > > <entity name="email"
> > > > > > > > > > > >
> > > > > > > > > > > > user="mail@gmail.com"
> > > > > > > > > > > ^^^^^ I presume that you have put in
> your
> > > > > actual
> > > > > > > > > > > email address here.
> > > > > > > > > > > [...]
> > > > > > > > > > > > protocol="imap"/>
> > > > > > > > > > > ^^^^^ Shouldn't this be imaps, at
> > > least
> > > > as
> > > > > > > > > > > per http://wiki.apache.org/solr/MailEntityProcessor
> > > > > > > > > > >
> > > > > > > > > > > Regards,
> > > > > > > > > > > Gora
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Help -DIH (mail)
Posted by Matias Alonso <ma...@gmail.com>.
I´m using the trunk.
Thanks Peter for your preoccupation!
Matias.
2011/3/4 Peter Sturge <pe...@gmail.com>
> Hi Matias,
>
> What version of Solr are you using? Are you running any patches (maybe
> SOLR-2245)?
>
> Thanks,
> Peter
>
>
>
> On Fri, Mar 4, 2011 at 8:25 PM, Matias Alonso <matiasgalonso@gmail.com
> >wrote:
>
> > Hi Peter,
> >
> > From "DataImportHandler Development Console" I made a full-import, but
> > didn´t work.
> >
> > Now, I execute "
> > http://localhost:8983/solr/mail/dataimport?command=full-import" but
> > nothing
> > happends; no index; no errors.
> >
> > thks...
> >
> > Matias.
> >
> >
> >
> > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >
> > > Hi Mataias,
> > >
> > >
> > >
> >
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimportaccesses
> > > the dataimport handler, but you need to tell it to do something by
> > > sending a command:
> > >
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
> > > ?command=full-import<
> > > http://localhost:8983/solr/db/dataimport?command=full-import>
> > >
> > > If you haven't already, have a look at:
> > >
> > >
> > >
> >
> http://www.lucidimagination.com/search/out?u=http%3A%2F%2Fwiki.apache.org%2Fsolr%2FDataImportHandler
> > >
> > > It gives very thorough and useful advice on getting the DIH working.
> > >
> > > Peter
> > >
> > >
> > >
> > > On Fri, Mar 4, 2011 at 6:59 PM, Matias Alonso <matiasgalonso@gmail.com
> > > >wrote:
> > >
> > > > Hi Peter,
> > > >
> > > > I test with deltaFetch="false", but doesn´t work :(
> > > > I'm using "DataImportHandler Development Console" to index (
> > > >
> > http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
> > > );
> > > > I'm working with "example-DIH".
> > > >
> > > > thks...
> > > >
> > > >
> > > >
> > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > >
> > > > > Hi Matias,
> > > > >
> > > > > I haven't seen it in the posts, but I may have missed it -- what is
> > the
> > > > > import command you're sending?
> > > > > Something like:
> > > > > http://localhost:8983/solr/db/dataimport?command=full-import
> > > > >
> > > > > Can you also test it with deltaFetch="false". I seem to remember
> > having
> > > > > some
> > > > > problems with delta in the MailEntityProcessor.
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Mar 4, 2011 at 6:29 PM, Matias Alonso <
> > matiasgalonso@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > <dataConfig>
> > > > > > <document>
> > > > > > <entity name="email"
> > > > > > user="myemail@gmail.com"
> > > > > > password="mypassword"
> > > > > > host="imap.gmail.com"
> > > > > > fetchMailsSince="2011-01-01 00:00:00"
> > > > > > deltaFetch="true"
> > > > > > include=""
> > > > > > exclude=""
> > > > > > recurse="false"
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> folders="Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo"
> > > > > > includeContent="true"
> > > > > > processAttachments="false"
> > > > > > includeOtherUserFolders="false"
> > > > > > includeSharedFolders="false"
> > > > > > batchSize="100"
> > > > > > processor="MailEntityProcessor"
> > > > > > protocol="imaps" />
> > > > > > </document>
> > > > > > </dataConfig>
> > > > > >
> > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > > > >
> > > > > > > Hi Matias,
> > > > > > >
> > > > > > > Can you post your data-config.xml? (with disquised
> > > names/credentials)
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Peter
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Mar 4, 2011 at 5:13 PM, Matias Alonso <
> > > > matiasgalonso@gmail.com
> > > > > > > >wrote:
> > > > > > >
> > > > > > > > Thks Peter,
> > > > > > > >
> > > > > > > > Yes, gmail gives me imaps (i understood that). So, I tried
> what
> > > you
> > > > > > > mention
> > > > > > > > but I had get the original mesange I posted.
> > > > > > > >
> > > > > > > > Matias.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > You need to put your password in as well. You should use
> > > > > > > protocol="imap"
> > > > > > > > > unless your gmail is set for imaps (I don't believe the
> free
> > > > gmail
> > > > > > > gives
> > > > > > > > > you
> > > > > > > > > this).
> > > > > > > > >
> > > > > > > > > <entity name="email"
> > > > > > > > > user="user@mydomain.com"
> > > > > > > > > password="userpwd"
> > > > > > > > > host="imap.mydomain.com"
> > > > > > > > > include=""
> > > > > > > > > exclude=""
> > > > > > > > > processor="MailEntityProcessor"
> > > > > > > > > protocol="imap"
> > > > > > > > > />
> > > > > > > > >
> > > > > > > > > HTH
> > > > > > > > > Peter
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <
> > > gora@mimirtech.com
> > > > >
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <
> > > > > > > matiasgalonso@gmail.com
> > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > Hi everyone!
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > I’m trying to index mails into solr through DHI (based
> > on
> > > > the
> > > > > > > > > > > “example-DIH”). For this I´m using my personal email
> from
> > > > > gmail,
> > > > > > > but
> > > > > > > > I
> > > > > > > > > > can´t
> > > > > > > > > > > index.
> > > > > > > > > >
> > > > > > > > > > Have not used the MailEntityProcessor with Gmail, but
> some
> > > > > > > > > > points below:
> > > > > > > > > >
> > > > > > > > > > > Configuration in Data-config .xml:
> > > > > > > > > > >
> > > > > > > > > > > <dataConfig>
> > > > > > > > > > >
> > > > > > > > > > > <document>
> > > > > > > > > > >
> > > > > > > > > > > <entity name="email"
> > > > > > > > > > >
> > > > > > > > > > > user="mail@gmail.com"
> > > > > > > > > > ^^^^^ I presume that you have put in your
> > > > actual
> > > > > > > > > > email address here.
> > > > > > > > > > [...]
> > > > > > > > > > > protocol="imap"/>
> > > > > > > > > > ^^^^^ Shouldn't this be imaps, at
> > least
> > > as
> > > > > > > > > > per http://wiki.apache.org/solr/MailEntityProcessor
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > > Gora
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Help -DIH (mail)
Posted by Peter Sturge <pe...@gmail.com>.
Hi Matias,
What version of Solr are you using? Are you running any patches (maybe
SOLR-2245)?
Thanks,
Peter
On Fri, Mar 4, 2011 at 8:25 PM, Matias Alonso <ma...@gmail.com>wrote:
> Hi Peter,
>
> From "DataImportHandler Development Console" I made a full-import, but
> didn´t work.
>
> Now, I execute "
> http://localhost:8983/solr/mail/dataimport?command=full-import" but
> nothing
> happends; no index; no errors.
>
> thks...
>
> Matias.
>
>
>
> 2011/3/4 Peter Sturge <pe...@gmail.com>
>
> > Hi Mataias,
> >
> >
> >
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimportaccesses
> > the dataimport handler, but you need to tell it to do something by
> > sending a command:
> > http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
> > ?command=full-import<
> > http://localhost:8983/solr/db/dataimport?command=full-import>
> >
> > If you haven't already, have a look at:
> >
> >
> >
> http://www.lucidimagination.com/search/out?u=http%3A%2F%2Fwiki.apache.org%2Fsolr%2FDataImportHandler
> >
> > It gives very thorough and useful advice on getting the DIH working.
> >
> > Peter
> >
> >
> >
> > On Fri, Mar 4, 2011 at 6:59 PM, Matias Alonso <matiasgalonso@gmail.com
> > >wrote:
> >
> > > Hi Peter,
> > >
> > > I test with deltaFetch="false", but doesn´t work :(
> > > I'm using "DataImportHandler Development Console" to index (
> > >
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
> > );
> > > I'm working with "example-DIH".
> > >
> > > thks...
> > >
> > >
> > >
> > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > >
> > > > Hi Matias,
> > > >
> > > > I haven't seen it in the posts, but I may have missed it -- what is
> the
> > > > import command you're sending?
> > > > Something like:
> > > > http://localhost:8983/solr/db/dataimport?command=full-import
> > > >
> > > > Can you also test it with deltaFetch="false". I seem to remember
> having
> > > > some
> > > > problems with delta in the MailEntityProcessor.
> > > >
> > > >
> > > >
> > > > On Fri, Mar 4, 2011 at 6:29 PM, Matias Alonso <
> matiasgalonso@gmail.com
> > > > >wrote:
> > > >
> > > > > <dataConfig>
> > > > > <document>
> > > > > <entity name="email"
> > > > > user="myemail@gmail.com"
> > > > > password="mypassword"
> > > > > host="imap.gmail.com"
> > > > > fetchMailsSince="2011-01-01 00:00:00"
> > > > > deltaFetch="true"
> > > > > include=""
> > > > > exclude=""
> > > > > recurse="false"
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> folders="Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo"
> > > > > includeContent="true"
> > > > > processAttachments="false"
> > > > > includeOtherUserFolders="false"
> > > > > includeSharedFolders="false"
> > > > > batchSize="100"
> > > > > processor="MailEntityProcessor"
> > > > > protocol="imaps" />
> > > > > </document>
> > > > > </dataConfig>
> > > > >
> > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > > >
> > > > > > Hi Matias,
> > > > > >
> > > > > > Can you post your data-config.xml? (with disquised
> > names/credentials)
> > > > > >
> > > > > > Thanks,
> > > > > > Peter
> > > > > >
> > > > > >
> > > > > > On Fri, Mar 4, 2011 at 5:13 PM, Matias Alonso <
> > > matiasgalonso@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > Thks Peter,
> > > > > > >
> > > > > > > Yes, gmail gives me imaps (i understood that). So, I tried what
> > you
> > > > > > mention
> > > > > > > but I had get the original mesange I posted.
> > > > > > >
> > > > > > > Matias.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > You need to put your password in as well. You should use
> > > > > > protocol="imap"
> > > > > > > > unless your gmail is set for imaps (I don't believe the free
> > > gmail
> > > > > > gives
> > > > > > > > you
> > > > > > > > this).
> > > > > > > >
> > > > > > > > <entity name="email"
> > > > > > > > user="user@mydomain.com"
> > > > > > > > password="userpwd"
> > > > > > > > host="imap.mydomain.com"
> > > > > > > > include=""
> > > > > > > > exclude=""
> > > > > > > > processor="MailEntityProcessor"
> > > > > > > > protocol="imap"
> > > > > > > > />
> > > > > > > >
> > > > > > > > HTH
> > > > > > > > Peter
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <
> > gora@mimirtech.com
> > > >
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <
> > > > > > matiasgalonso@gmail.com
> > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > Hi everyone!
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > I’m trying to index mails into solr through DHI (based
> on
> > > the
> > > > > > > > > > “example-DIH”). For this I´m using my personal email from
> > > > gmail,
> > > > > > but
> > > > > > > I
> > > > > > > > > can´t
> > > > > > > > > > index.
> > > > > > > > >
> > > > > > > > > Have not used the MailEntityProcessor with Gmail, but some
> > > > > > > > > points below:
> > > > > > > > >
> > > > > > > > > > Configuration in Data-config .xml:
> > > > > > > > > >
> > > > > > > > > > <dataConfig>
> > > > > > > > > >
> > > > > > > > > > <document>
> > > > > > > > > >
> > > > > > > > > > <entity name="email"
> > > > > > > > > >
> > > > > > > > > > user="mail@gmail.com"
> > > > > > > > > ^^^^^ I presume that you have put in your
> > > actual
> > > > > > > > > email address here.
> > > > > > > > > [...]
> > > > > > > > > > protocol="imap"/>
> > > > > > > > > ^^^^^ Shouldn't this be imaps, at
> least
> > as
> > > > > > > > > per http://wiki.apache.org/solr/MailEntityProcessor
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > > Gora
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Help -DIH (mail)
Posted by Matias Alonso <ma...@gmail.com>.
Hi Peter,
>From "DataImportHandler Development Console" I made a full-import, but
didn´t work.
Now, I execute "
http://localhost:8983/solr/mail/dataimport?command=full-import" but nothing
happends; no index; no errors.
thks...
Matias.
2011/3/4 Peter Sturge <pe...@gmail.com>
> Hi Mataias,
>
>
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimportaccesses
> the dataimport handler, but you need to tell it to do something by
> sending a command:
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
> ?command=full-import<
> http://localhost:8983/solr/db/dataimport?command=full-import>
>
> If you haven't already, have a look at:
>
>
> http://www.lucidimagination.com/search/out?u=http%3A%2F%2Fwiki.apache.org%2Fsolr%2FDataImportHandler
>
> It gives very thorough and useful advice on getting the DIH working.
>
> Peter
>
>
>
> On Fri, Mar 4, 2011 at 6:59 PM, Matias Alonso <matiasgalonso@gmail.com
> >wrote:
>
> > Hi Peter,
> >
> > I test with deltaFetch="false", but doesn´t work :(
> > I'm using "DataImportHandler Development Console" to index (
> > http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
> );
> > I'm working with "example-DIH".
> >
> > thks...
> >
> >
> >
> > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >
> > > Hi Matias,
> > >
> > > I haven't seen it in the posts, but I may have missed it -- what is the
> > > import command you're sending?
> > > Something like:
> > > http://localhost:8983/solr/db/dataimport?command=full-import
> > >
> > > Can you also test it with deltaFetch="false". I seem to remember having
> > > some
> > > problems with delta in the MailEntityProcessor.
> > >
> > >
> > >
> > > On Fri, Mar 4, 2011 at 6:29 PM, Matias Alonso <matiasgalonso@gmail.com
> > > >wrote:
> > >
> > > > <dataConfig>
> > > > <document>
> > > > <entity name="email"
> > > > user="myemail@gmail.com"
> > > > password="mypassword"
> > > > host="imap.gmail.com"
> > > > fetchMailsSince="2011-01-01 00:00:00"
> > > > deltaFetch="true"
> > > > include=""
> > > > exclude=""
> > > > recurse="false"
> > > >
> > > >
> > > >
> > >
> >
> folders="Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo"
> > > > includeContent="true"
> > > > processAttachments="false"
> > > > includeOtherUserFolders="false"
> > > > includeSharedFolders="false"
> > > > batchSize="100"
> > > > processor="MailEntityProcessor"
> > > > protocol="imaps" />
> > > > </document>
> > > > </dataConfig>
> > > >
> > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > >
> > > > > Hi Matias,
> > > > >
> > > > > Can you post your data-config.xml? (with disquised
> names/credentials)
> > > > >
> > > > > Thanks,
> > > > > Peter
> > > > >
> > > > >
> > > > > On Fri, Mar 4, 2011 at 5:13 PM, Matias Alonso <
> > matiasgalonso@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > Thks Peter,
> > > > > >
> > > > > > Yes, gmail gives me imaps (i understood that). So, I tried what
> you
> > > > > mention
> > > > > > but I had get the original mesange I posted.
> > > > > >
> > > > > > Matias.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > You need to put your password in as well. You should use
> > > > > protocol="imap"
> > > > > > > unless your gmail is set for imaps (I don't believe the free
> > gmail
> > > > > gives
> > > > > > > you
> > > > > > > this).
> > > > > > >
> > > > > > > <entity name="email"
> > > > > > > user="user@mydomain.com"
> > > > > > > password="userpwd"
> > > > > > > host="imap.mydomain.com"
> > > > > > > include=""
> > > > > > > exclude=""
> > > > > > > processor="MailEntityProcessor"
> > > > > > > protocol="imap"
> > > > > > > />
> > > > > > >
> > > > > > > HTH
> > > > > > > Peter
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <
> gora@mimirtech.com
> > >
> > > > > wrote:
> > > > > > >
> > > > > > > > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <
> > > > > matiasgalonso@gmail.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > > > Hi everyone!
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I’m trying to index mails into solr through DHI (based on
> > the
> > > > > > > > > “example-DIH”). For this I´m using my personal email from
> > > gmail,
> > > > > but
> > > > > > I
> > > > > > > > can´t
> > > > > > > > > index.
> > > > > > > >
> > > > > > > > Have not used the MailEntityProcessor with Gmail, but some
> > > > > > > > points below:
> > > > > > > >
> > > > > > > > > Configuration in Data-config .xml:
> > > > > > > > >
> > > > > > > > > <dataConfig>
> > > > > > > > >
> > > > > > > > > <document>
> > > > > > > > >
> > > > > > > > > <entity name="email"
> > > > > > > > >
> > > > > > > > > user="mail@gmail.com"
> > > > > > > > ^^^^^ I presume that you have put in your
> > actual
> > > > > > > > email address here.
> > > > > > > > [...]
> > > > > > > > > protocol="imap"/>
> > > > > > > > ^^^^^ Shouldn't this be imaps, at least
> as
> > > > > > > > per http://wiki.apache.org/solr/MailEntityProcessor
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Gora
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Help -DIH (mail)
Posted by Peter Sturge <pe...@gmail.com>.
Hi Mataias,
http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimportaccesses
the dataimport handler, but you need to tell it to do something by
sending a command:
http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport
?command=full-import<http://localhost:8983/solr/db/dataimport?command=full-import>
If you haven't already, have a look at:
http://www.lucidimagination.com/search/out?u=http%3A%2F%2Fwiki.apache.org%2Fsolr%2FDataImportHandler
It gives very thorough and useful advice on getting the DIH working.
Peter
On Fri, Mar 4, 2011 at 6:59 PM, Matias Alonso <ma...@gmail.com>wrote:
> Hi Peter,
>
> I test with deltaFetch="false", but doesn´t work :(
> I'm using "DataImportHandler Development Console" to index (
> http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport);
> I'm working with "example-DIH".
>
> thks...
>
>
>
> 2011/3/4 Peter Sturge <pe...@gmail.com>
>
> > Hi Matias,
> >
> > I haven't seen it in the posts, but I may have missed it -- what is the
> > import command you're sending?
> > Something like:
> > http://localhost:8983/solr/db/dataimport?command=full-import
> >
> > Can you also test it with deltaFetch="false". I seem to remember having
> > some
> > problems with delta in the MailEntityProcessor.
> >
> >
> >
> > On Fri, Mar 4, 2011 at 6:29 PM, Matias Alonso <matiasgalonso@gmail.com
> > >wrote:
> >
> > > <dataConfig>
> > > <document>
> > > <entity name="email"
> > > user="myemail@gmail.com"
> > > password="mypassword"
> > > host="imap.gmail.com"
> > > fetchMailsSince="2011-01-01 00:00:00"
> > > deltaFetch="true"
> > > include=""
> > > exclude=""
> > > recurse="false"
> > >
> > >
> > >
> >
> folders="Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo"
> > > includeContent="true"
> > > processAttachments="false"
> > > includeOtherUserFolders="false"
> > > includeSharedFolders="false"
> > > batchSize="100"
> > > processor="MailEntityProcessor"
> > > protocol="imaps" />
> > > </document>
> > > </dataConfig>
> > >
> > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > >
> > > > Hi Matias,
> > > >
> > > > Can you post your data-config.xml? (with disquised names/credentials)
> > > >
> > > > Thanks,
> > > > Peter
> > > >
> > > >
> > > > On Fri, Mar 4, 2011 at 5:13 PM, Matias Alonso <
> matiasgalonso@gmail.com
> > > > >wrote:
> > > >
> > > > > Thks Peter,
> > > > >
> > > > > Yes, gmail gives me imaps (i understood that). So, I tried what you
> > > > mention
> > > > > but I had get the original mesange I posted.
> > > > >
> > > > > Matias.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > You need to put your password in as well. You should use
> > > > protocol="imap"
> > > > > > unless your gmail is set for imaps (I don't believe the free
> gmail
> > > > gives
> > > > > > you
> > > > > > this).
> > > > > >
> > > > > > <entity name="email"
> > > > > > user="user@mydomain.com"
> > > > > > password="userpwd"
> > > > > > host="imap.mydomain.com"
> > > > > > include=""
> > > > > > exclude=""
> > > > > > processor="MailEntityProcessor"
> > > > > > protocol="imap"
> > > > > > />
> > > > > >
> > > > > > HTH
> > > > > > Peter
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <gora@mimirtech.com
> >
> > > > wrote:
> > > > > >
> > > > > > > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <
> > > > matiasgalonso@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > > > Hi everyone!
> > > > > > > >
> > > > > > > >
> > > > > > > > I’m trying to index mails into solr through DHI (based on
> the
> > > > > > > > “example-DIH”). For this I´m using my personal email from
> > gmail,
> > > > but
> > > > > I
> > > > > > > can´t
> > > > > > > > index.
> > > > > > >
> > > > > > > Have not used the MailEntityProcessor with Gmail, but some
> > > > > > > points below:
> > > > > > >
> > > > > > > > Configuration in Data-config .xml:
> > > > > > > >
> > > > > > > > <dataConfig>
> > > > > > > >
> > > > > > > > <document>
> > > > > > > >
> > > > > > > > <entity name="email"
> > > > > > > >
> > > > > > > > user="mail@gmail.com"
> > > > > > > ^^^^^ I presume that you have put in your
> actual
> > > > > > > email address here.
> > > > > > > [...]
> > > > > > > > protocol="imap"/>
> > > > > > > ^^^^^ Shouldn't this be imaps, at least as
> > > > > > > per http://wiki.apache.org/solr/MailEntityProcessor
> > > > > > >
> > > > > > > Regards,
> > > > > > > Gora
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Help -DIH (mail)
Posted by Matias Alonso <ma...@gmail.com>.
Hi Peter,
I test with deltaFetch="false", but doesn´t work :(
I'm using "DataImportHandler Development Console" to index (
http://localhost:8983/solr/mail/admin/dataimport.jsp?handler=/dataimport);
I'm working with "example-DIH".
thks...
2011/3/4 Peter Sturge <pe...@gmail.com>
> Hi Matias,
>
> I haven't seen it in the posts, but I may have missed it -- what is the
> import command you're sending?
> Something like:
> http://localhost:8983/solr/db/dataimport?command=full-import
>
> Can you also test it with deltaFetch="false". I seem to remember having
> some
> problems with delta in the MailEntityProcessor.
>
>
>
> On Fri, Mar 4, 2011 at 6:29 PM, Matias Alonso <matiasgalonso@gmail.com
> >wrote:
>
> > <dataConfig>
> > <document>
> > <entity name="email"
> > user="myemail@gmail.com"
> > password="mypassword"
> > host="imap.gmail.com"
> > fetchMailsSince="2011-01-01 00:00:00"
> > deltaFetch="true"
> > include=""
> > exclude=""
> > recurse="false"
> >
> >
> >
> folders="Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo"
> > includeContent="true"
> > processAttachments="false"
> > includeOtherUserFolders="false"
> > includeSharedFolders="false"
> > batchSize="100"
> > processor="MailEntityProcessor"
> > protocol="imaps" />
> > </document>
> > </dataConfig>
> >
> > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >
> > > Hi Matias,
> > >
> > > Can you post your data-config.xml? (with disquised names/credentials)
> > >
> > > Thanks,
> > > Peter
> > >
> > >
> > > On Fri, Mar 4, 2011 at 5:13 PM, Matias Alonso <matiasgalonso@gmail.com
> > > >wrote:
> > >
> > > > Thks Peter,
> > > >
> > > > Yes, gmail gives me imaps (i understood that). So, I tried what you
> > > mention
> > > > but I had get the original mesange I posted.
> > > >
> > > > Matias.
> > > >
> > > >
> > > >
> > > >
> > > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > > >
> > > > > Hi,
> > > > >
> > > > > You need to put your password in as well. You should use
> > > protocol="imap"
> > > > > unless your gmail is set for imaps (I don't believe the free gmail
> > > gives
> > > > > you
> > > > > this).
> > > > >
> > > > > <entity name="email"
> > > > > user="user@mydomain.com"
> > > > > password="userpwd"
> > > > > host="imap.mydomain.com"
> > > > > include=""
> > > > > exclude=""
> > > > > processor="MailEntityProcessor"
> > > > > protocol="imap"
> > > > > />
> > > > >
> > > > > HTH
> > > > > Peter
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <go...@mimirtech.com>
> > > wrote:
> > > > >
> > > > > > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <
> > > matiasgalonso@gmail.com
> > > > >
> > > > > > wrote:
> > > > > > > Hi everyone!
> > > > > > >
> > > > > > >
> > > > > > > I’m trying to index mails into solr through DHI (based on the
> > > > > > > “example-DIH”). For this I´m using my personal email from
> gmail,
> > > but
> > > > I
> > > > > > can´t
> > > > > > > index.
> > > > > >
> > > > > > Have not used the MailEntityProcessor with Gmail, but some
> > > > > > points below:
> > > > > >
> > > > > > > Configuration in Data-config .xml:
> > > > > > >
> > > > > > > <dataConfig>
> > > > > > >
> > > > > > > <document>
> > > > > > >
> > > > > > > <entity name="email"
> > > > > > >
> > > > > > > user="mail@gmail.com"
> > > > > > ^^^^^ I presume that you have put in your actual
> > > > > > email address here.
> > > > > > [...]
> > > > > > > protocol="imap"/>
> > > > > > ^^^^^ Shouldn't this be imaps, at least as
> > > > > > per http://wiki.apache.org/solr/MailEntityProcessor
> > > > > >
> > > > > > Regards,
> > > > > > Gora
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Help -DIH (mail)
Posted by Peter Sturge <pe...@gmail.com>.
Hi Matias,
I haven't seen it in the posts, but I may have missed it -- what is the
import command you're sending?
Something like: http://localhost:8983/solr/db/dataimport?command=full-import
Can you also test it with deltaFetch="false". I seem to remember having some
problems with delta in the MailEntityProcessor.
On Fri, Mar 4, 2011 at 6:29 PM, Matias Alonso <ma...@gmail.com>wrote:
> <dataConfig>
> <document>
> <entity name="email"
> user="myemail@gmail.com"
> password="mypassword"
> host="imap.gmail.com"
> fetchMailsSince="2011-01-01 00:00:00"
> deltaFetch="true"
> include=""
> exclude=""
> recurse="false"
>
>
> folders="Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo"
> includeContent="true"
> processAttachments="false"
> includeOtherUserFolders="false"
> includeSharedFolders="false"
> batchSize="100"
> processor="MailEntityProcessor"
> protocol="imaps" />
> </document>
> </dataConfig>
>
> 2011/3/4 Peter Sturge <pe...@gmail.com>
>
> > Hi Matias,
> >
> > Can you post your data-config.xml? (with disquised names/credentials)
> >
> > Thanks,
> > Peter
> >
> >
> > On Fri, Mar 4, 2011 at 5:13 PM, Matias Alonso <matiasgalonso@gmail.com
> > >wrote:
> >
> > > Thks Peter,
> > >
> > > Yes, gmail gives me imaps (i understood that). So, I tried what you
> > mention
> > > but I had get the original mesange I posted.
> > >
> > > Matias.
> > >
> > >
> > >
> > >
> > > 2011/3/4 Peter Sturge <pe...@gmail.com>
> > >
> > > > Hi,
> > > >
> > > > You need to put your password in as well. You should use
> > protocol="imap"
> > > > unless your gmail is set for imaps (I don't believe the free gmail
> > gives
> > > > you
> > > > this).
> > > >
> > > > <entity name="email"
> > > > user="user@mydomain.com"
> > > > password="userpwd"
> > > > host="imap.mydomain.com"
> > > > include=""
> > > > exclude=""
> > > > processor="MailEntityProcessor"
> > > > protocol="imap"
> > > > />
> > > >
> > > > HTH
> > > > Peter
> > > >
> > > >
> > > >
> > > > On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <go...@mimirtech.com>
> > wrote:
> > > >
> > > > > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <
> > matiasgalonso@gmail.com
> > > >
> > > > > wrote:
> > > > > > Hi everyone!
> > > > > >
> > > > > >
> > > > > > I’m trying to index mails into solr through DHI (based on the
> > > > > > “example-DIH”). For this I´m using my personal email from gmail,
> > but
> > > I
> > > > > can´t
> > > > > > index.
> > > > >
> > > > > Have not used the MailEntityProcessor with Gmail, but some
> > > > > points below:
> > > > >
> > > > > > Configuration in Data-config .xml:
> > > > > >
> > > > > > <dataConfig>
> > > > > >
> > > > > > <document>
> > > > > >
> > > > > > <entity name="email"
> > > > > >
> > > > > > user="mail@gmail.com"
> > > > > ^^^^^ I presume that you have put in your actual
> > > > > email address here.
> > > > > [...]
> > > > > > protocol="imap"/>
> > > > > ^^^^^ Shouldn't this be imaps, at least as
> > > > > per http://wiki.apache.org/solr/MailEntityProcessor
> > > > >
> > > > > Regards,
> > > > > Gora
> > > > >
> > > >
> > >
> >
>
Re: Help -DIH (mail)
Posted by Matias Alonso <ma...@gmail.com>.
<dataConfig>
<document>
<entity name="email"
user="myemail@gmail.com"
password="mypassword"
host="imap.gmail.com"
fetchMailsSince="2011-01-01 00:00:00"
deltaFetch="true"
include=""
exclude=""
recurse="false"
folders="Recibidos,recibidos,RECIBIDOS,inbox.InBox,INBOX,Mail,MAIL,mail,CORREO,correo,Correo"
includeContent="true"
processAttachments="false"
includeOtherUserFolders="false"
includeSharedFolders="false"
batchSize="100"
processor="MailEntityProcessor"
protocol="imaps" />
</document>
</dataConfig>
2011/3/4 Peter Sturge <pe...@gmail.com>
> Hi Matias,
>
> Can you post your data-config.xml? (with disquised names/credentials)
>
> Thanks,
> Peter
>
>
> On Fri, Mar 4, 2011 at 5:13 PM, Matias Alonso <matiasgalonso@gmail.com
> >wrote:
>
> > Thks Peter,
> >
> > Yes, gmail gives me imaps (i understood that). So, I tried what you
> mention
> > but I had get the original mesange I posted.
> >
> > Matias.
> >
> >
> >
> >
> > 2011/3/4 Peter Sturge <pe...@gmail.com>
> >
> > > Hi,
> > >
> > > You need to put your password in as well. You should use
> protocol="imap"
> > > unless your gmail is set for imaps (I don't believe the free gmail
> gives
> > > you
> > > this).
> > >
> > > <entity name="email"
> > > user="user@mydomain.com"
> > > password="userpwd"
> > > host="imap.mydomain.com"
> > > include=""
> > > exclude=""
> > > processor="MailEntityProcessor"
> > > protocol="imap"
> > > />
> > >
> > > HTH
> > > Peter
> > >
> > >
> > >
> > > On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <go...@mimirtech.com>
> wrote:
> > >
> > > > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <
> matiasgalonso@gmail.com
> > >
> > > > wrote:
> > > > > Hi everyone!
> > > > >
> > > > >
> > > > > I’m trying to index mails into solr through DHI (based on the
> > > > > “example-DIH”). For this I´m using my personal email from gmail,
> but
> > I
> > > > can´t
> > > > > index.
> > > >
> > > > Have not used the MailEntityProcessor with Gmail, but some
> > > > points below:
> > > >
> > > > > Configuration in Data-config .xml:
> > > > >
> > > > > <dataConfig>
> > > > >
> > > > > <document>
> > > > >
> > > > > <entity name="email"
> > > > >
> > > > > user="mail@gmail.com"
> > > > ^^^^^ I presume that you have put in your actual
> > > > email address here.
> > > > [...]
> > > > > protocol="imap"/>
> > > > ^^^^^ Shouldn't this be imaps, at least as
> > > > per http://wiki.apache.org/solr/MailEntityProcessor
> > > >
> > > > Regards,
> > > > Gora
> > > >
> > >
> >
>
Re: Help -DIH (mail)
Posted by Peter Sturge <pe...@gmail.com>.
Hi Matias,
Can you post your data-config.xml? (with disquised names/credentials)
Thanks,
Peter
On Fri, Mar 4, 2011 at 5:13 PM, Matias Alonso <ma...@gmail.com>wrote:
> Thks Peter,
>
> Yes, gmail gives me imaps (i understood that). So, I tried what you mention
> but I had get the original mesange I posted.
>
> Matias.
>
>
>
>
> 2011/3/4 Peter Sturge <pe...@gmail.com>
>
> > Hi,
> >
> > You need to put your password in as well. You should use protocol="imap"
> > unless your gmail is set for imaps (I don't believe the free gmail gives
> > you
> > this).
> >
> > <entity name="email"
> > user="user@mydomain.com"
> > password="userpwd"
> > host="imap.mydomain.com"
> > include=""
> > exclude=""
> > processor="MailEntityProcessor"
> > protocol="imap"
> > />
> >
> > HTH
> > Peter
> >
> >
> >
> > On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <go...@mimirtech.com> wrote:
> >
> > > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <matiasgalonso@gmail.com
> >
> > > wrote:
> > > > Hi everyone!
> > > >
> > > >
> > > > I’m trying to index mails into solr through DHI (based on the
> > > > “example-DIH”). For this I´m using my personal email from gmail, but
> I
> > > can´t
> > > > index.
> > >
> > > Have not used the MailEntityProcessor with Gmail, but some
> > > points below:
> > >
> > > > Configuration in Data-config .xml:
> > > >
> > > > <dataConfig>
> > > >
> > > > <document>
> > > >
> > > > <entity name="email"
> > > >
> > > > user="mail@gmail.com"
> > > ^^^^^ I presume that you have put in your actual
> > > email address here.
> > > [...]
> > > > protocol="imap"/>
> > > ^^^^^ Shouldn't this be imaps, at least as
> > > per http://wiki.apache.org/solr/MailEntityProcessor
> > >
> > > Regards,
> > > Gora
> > >
> >
>
Re: Help -DIH (mail)
Posted by Matias Alonso <ma...@gmail.com>.
Thks Peter,
Yes, gmail gives me imaps (i understood that). So, I tried what you mention
but I had get the original mesange I posted.
Matias.
2011/3/4 Peter Sturge <pe...@gmail.com>
> Hi,
>
> You need to put your password in as well. You should use protocol="imap"
> unless your gmail is set for imaps (I don't believe the free gmail gives
> you
> this).
>
> <entity name="email"
> user="user@mydomain.com"
> password="userpwd"
> host="imap.mydomain.com"
> include=""
> exclude=""
> processor="MailEntityProcessor"
> protocol="imap"
> />
>
> HTH
> Peter
>
>
>
> On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <go...@mimirtech.com> wrote:
>
> > On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <ma...@gmail.com>
> > wrote:
> > > Hi everyone!
> > >
> > >
> > > I’m trying to index mails into solr through DHI (based on the
> > > “example-DIH”). For this I´m using my personal email from gmail, but I
> > can´t
> > > index.
> >
> > Have not used the MailEntityProcessor with Gmail, but some
> > points below:
> >
> > > Configuration in Data-config .xml:
> > >
> > > <dataConfig>
> > >
> > > <document>
> > >
> > > <entity name="email"
> > >
> > > user="mail@gmail.com"
> > ^^^^^ I presume that you have put in your actual
> > email address here.
> > [...]
> > > protocol="imap"/>
> > ^^^^^ Shouldn't this be imaps, at least as
> > per http://wiki.apache.org/solr/MailEntityProcessor
> >
> > Regards,
> > Gora
> >
>
Re: Help -DIH (mail)
Posted by Peter Sturge <pe...@gmail.com>.
Hi,
You need to put your password in as well. You should use protocol="imap"
unless your gmail is set for imaps (I don't believe the free gmail gives you
this).
<entity name="email"
user="user@mydomain.com"
password="userpwd"
host="imap.mydomain.com"
include=""
exclude=""
processor="MailEntityProcessor"
protocol="imap"
/>
HTH
Peter
On Fri, Mar 4, 2011 at 4:42 PM, Gora Mohanty <go...@mimirtech.com> wrote:
> On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <ma...@gmail.com>
> wrote:
> > Hi everyone!
> >
> >
> > I’m trying to index mails into solr through DHI (based on the
> > “example-DIH”). For this I´m using my personal email from gmail, but I
> can´t
> > index.
>
> Have not used the MailEntityProcessor with Gmail, but some
> points below:
>
> > Configuration in Data-config .xml:
> >
> > <dataConfig>
> >
> > <document>
> >
> > <entity name="email"
> >
> > user="mail@gmail.com"
> ^^^^^ I presume that you have put in your actual
> email address here.
> [...]
> > protocol="imap"/>
> ^^^^^ Shouldn't this be imaps, at least as
> per http://wiki.apache.org/solr/MailEntityProcessor
>
> Regards,
> Gora
>
Re: Help -DIH (mail)
Posted by Gora Mohanty <go...@mimirtech.com>.
On Fri, Mar 4, 2011 at 9:20 PM, Matias Alonso <ma...@gmail.com> wrote:
> Hi everyone!
>
>
> I’m trying to index mails into solr through DHI (based on the
> “example-DIH”). For this I´m using my personal email from gmail, but I can´t
> index.
Have not used the MailEntityProcessor with Gmail, but some
points below:
> Configuration in Data-config .xml:
>
> <dataConfig>
>
> <document>
>
> <entity name="email"
>
> user="mail@gmail.com"
^^^^^ I presume that you have put in your actual
email address here.
[...]
> protocol="imap"/>
^^^^^ Shouldn't this be imaps, at least as
per http://wiki.apache.org/solr/MailEntityProcessor
Regards,
Gora