You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Shinichiro Abe (JIRA)" <ji...@apache.org> on 2011/04/28 08:03:03 UTC
[jira] [Created] (SOLR-2480) Text extraction of password protected
files
Text extraction of password protected files
-------------------------------------------
Key: SOLR-2480
URL: https://issues.apache.org/jira/browse/SOLR-2480
Project: Solr
Issue Type: Improvement
Components: contrib - Solr Cell (Tika extraction)
Affects Versions: 3.1
Reporter: Shinichiro Abe
Priority: Minor
Proposal:
There are password-protected files. PDF, Office documents in 2007 format/97 format.
These files are posted using SolrCell.
We do not have to read these files if we do not know the reading password of files.
So, these files may not be extracted text.
My requirement is that these files should be processed normally without extracting text, and without throwing exception.
This background:
Now, when you post a password-protected file, solr returns 500 server error.
Solr catches the error in ExtractingDocumentLoader and throws TikException.
I use ManifoldCF.
If the solr server responds 500, ManifoldCF judge is that "this
document should be retried because I have absolutely no idea what
happened".
And it attempts to retry posting many times without getting the password.
In the other case, my customer posts the files with embedded images.
Sometimes it seems that solr throws TikaException of unknown cause.
He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: [jira] [Commented] (SOLR-2480) Text extraction of password
protected files
Posted by Erick Erickson <er...@gmail.com>.
Hmmm, I'm not sure this fits into Solr-445 or not, could you add this
comment to that
patch discussion so we at least look?
Thanks,
Erick
On Thu, Apr 28, 2011 at 2:03 AM, Shinichiro Abe (JIRA) <ji...@apache.org> wrote:
>
> [ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026137#comment-13026137 ]
>
> Shinichiro Abe commented on SOLR-2480:
> --------------------------------------
>
> Improvement ideas:
> 1, TikaException is always ignored, and index only the metadata.
> 2, Parameter "ignoreTikaException" is provided newly.
> If it is true then it returns 200 response, if it is false then it throws TikaException.
> 3, If Solr can catch internal exception about encrypting error, it changes return code each exception.
> If it can judge poi.EncryptedDocumentException, pdfbox.exceptions.CryptographyException. etc. then it returns 200 or another code response, if it judges the other exception then it throws TikaException.
>
>> Text extraction of password protected files
>> -------------------------------------------
>>
>> Key: SOLR-2480
>> URL: https://issues.apache.org/jira/browse/SOLR-2480
>> Project: Solr
>> Issue Type: Improvement
>> Components: contrib - Solr Cell (Tika extraction)
>> Affects Versions: 3.1
>> Reporter: Shinichiro Abe
>> Priority: Minor
>>
>> Proposal:
>> There are password-protected files. PDF, Office documents in 2007 format/97 format.
>> These files are posted using SolrCell.
>> We do not have to read these files if we do not know the reading password of files.
>> So, these files may not be extracted text.
>> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
>> This background:
>> Now, when you post a password-protected file, solr returns 500 server error.
>> Solr catches the error in ExtractingDocumentLoader and throws TikException.
>> I use ManifoldCF.
>> If the solr server responds 500, ManifoldCF judge is that "this
>> document should be retried because I have absolutely no idea what
>> happened".
>> And it attempts to retry posting many times without getting the password.
>> In the other case, my customer posts the files with embedded images.
>> Sometimes it seems that solr throws TikaException of unknown cause.
>> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
>
> --
> This message is automatically generated by JIRA.
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2480) Text extraction of password
protected files
Posted by "Shinichiro Abe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026137#comment-13026137 ]
Shinichiro Abe commented on SOLR-2480:
--------------------------------------
Improvement ideas:
1, TikaException is always ignored, and index only the metadata.
2, Parameter "ignoreTikaException" is provided newly.
If it is true then it returns 200 response, if it is false then it throws TikaException.
3, If Solr can catch internal exception about encrypting error, it changes return code each exception.
If it can judge poi.EncryptedDocumentException, pdfbox.exceptions.CryptographyException. etc. then it returns 200 or another code response, if it judges the other exception then it throws TikaException.
> Text extraction of password protected files
> -------------------------------------------
>
> Key: SOLR-2480
> URL: https://issues.apache.org/jira/browse/SOLR-2480
> Project: Solr
> Issue Type: Improvement
> Components: contrib - Solr Cell (Tika extraction)
> Affects Versions: 3.1
> Reporter: Shinichiro Abe
> Priority: Minor
>
> Proposal:
> There are password-protected files. PDF, Office documents in 2007 format/97 format.
> These files are posted using SolrCell.
> We do not have to read these files if we do not know the reading password of files.
> So, these files may not be extracted text.
> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
> This background:
> Now, when you post a password-protected file, solr returns 500 server error.
> Solr catches the error in ExtractingDocumentLoader and throws TikException.
> I use ManifoldCF.
> If the solr server responds 500, ManifoldCF judge is that "this
> document should be retried because I have absolutely no idea what
> happened".
> And it attempts to retry posting many times without getting the password.
> In the other case, my customer posts the files with embedded images.
> Sometimes it seems that solr throws TikaException of unknown cause.
> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (SOLR-2480) Text extraction of password protected
files
Posted by "Koji Sekiguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Sekiguchi resolved SOLR-2480.
----------------------------------
Resolution: Fixed
trunk: Committed revision 1103120.
3x: Committed revision 1103124.
> Text extraction of password protected files
> -------------------------------------------
>
> Key: SOLR-2480
> URL: https://issues.apache.org/jira/browse/SOLR-2480
> Project: Solr
> Issue Type: Improvement
> Components: contrib - Solr Cell (Tika extraction)
> Affects Versions: 1.4.1, 3.1
> Reporter: Shinichiro Abe
> Assignee: Koji Sekiguchi
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2480-idea1.patch, SOLR-2480.patch, SOLR-2480.patch, SOLR-2480.patch, password-is-solrcell.docx
>
>
> Proposal:
> There are password-protected files. PDF, Office documents in 2007 format/97 format.
> These files are posted using SolrCell.
> We do not have to read these files if we do not know the reading password of files.
> So, these files may not be extracted text.
> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
> This background:
> Now, when you post a password-protected file, solr returns 500 server error.
> Solr catches the error in ExtractingDocumentLoader and throws TikException.
> I use ManifoldCF.
> If the solr server responds 500, ManifoldCF judge is that "this
> document should be retried because I have absolutely no idea what
> happened".
> And it attempts to retry posting many times without getting the password.
> In the other case, my customer posts the files with embedded images.
> Sometimes it seems that solr throws TikaException of unknown cause.
> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2480) Text extraction of password protected
files
Posted by "Koji Sekiguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Sekiguchi updated SOLR-2480:
---------------------------------
Attachment: SOLR-2480.patch
A patch that introduces ignoreTikaException flag.
> Text extraction of password protected files
> -------------------------------------------
>
> Key: SOLR-2480
> URL: https://issues.apache.org/jira/browse/SOLR-2480
> Project: Solr
> Issue Type: Improvement
> Components: contrib - Solr Cell (Tika extraction)
> Affects Versions: 1.4.1, 3.1
> Reporter: Shinichiro Abe
> Assignee: Koji Sekiguchi
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2480-idea1.patch, SOLR-2480.patch
>
>
> Proposal:
> There are password-protected files. PDF, Office documents in 2007 format/97 format.
> These files are posted using SolrCell.
> We do not have to read these files if we do not know the reading password of files.
> So, these files may not be extracted text.
> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
> This background:
> Now, when you post a password-protected file, solr returns 500 server error.
> Solr catches the error in ExtractingDocumentLoader and throws TikException.
> I use ManifoldCF.
> If the solr server responds 500, ManifoldCF judge is that "this
> document should be retried because I have absolutely no idea what
> happened".
> And it attempts to retry posting many times without getting the password.
> In the other case, my customer posts the files with embedded images.
> Sometimes it seems that solr throws TikaException of unknown cause.
> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2480) Text extraction of password
protected files
Posted by "Shinichiro Abe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027544#comment-13027544 ]
Shinichiro Abe commented on SOLR-2480:
--------------------------------------
There is a same issue.
https://issues.apache.org/jira/browse/SOLR-445
If it be able to applied by that same policy, this issue is duplicate.
> Text extraction of password protected files
> -------------------------------------------
>
> Key: SOLR-2480
> URL: https://issues.apache.org/jira/browse/SOLR-2480
> Project: Solr
> Issue Type: Improvement
> Components: contrib - Solr Cell (Tika extraction)
> Affects Versions: 3.1
> Reporter: Shinichiro Abe
> Priority: Minor
>
> Proposal:
> There are password-protected files. PDF, Office documents in 2007 format/97 format.
> These files are posted using SolrCell.
> We do not have to read these files if we do not know the reading password of files.
> So, these files may not be extracted text.
> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
> This background:
> Now, when you post a password-protected file, solr returns 500 server error.
> Solr catches the error in ExtractingDocumentLoader and throws TikException.
> I use ManifoldCF.
> If the solr server responds 500, ManifoldCF judge is that "this
> document should be retried because I have absolutely no idea what
> happened".
> And it attempts to retry posting many times without getting the password.
> In the other case, my customer posts the files with embedded images.
> Sometimes it seems that solr throws TikaException of unknown cause.
> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2480) Text extraction of password protected
files
Posted by "Shinichiro Abe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shinichiro Abe updated SOLR-2480:
---------------------------------
Attachment: SOLR-2480-idea1.patch
> Text extraction of password protected files
> -------------------------------------------
>
> Key: SOLR-2480
> URL: https://issues.apache.org/jira/browse/SOLR-2480
> Project: Solr
> Issue Type: Improvement
> Components: contrib - Solr Cell (Tika extraction)
> Affects Versions: 3.1
> Reporter: Shinichiro Abe
> Priority: Minor
> Attachments: SOLR-2480-idea1.patch
>
>
> Proposal:
> There are password-protected files. PDF, Office documents in 2007 format/97 format.
> These files are posted using SolrCell.
> We do not have to read these files if we do not know the reading password of files.
> So, these files may not be extracted text.
> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
> This background:
> Now, when you post a password-protected file, solr returns 500 server error.
> Solr catches the error in ExtractingDocumentLoader and throws TikException.
> I use ManifoldCF.
> If the solr server responds 500, ManifoldCF judge is that "this
> document should be retried because I have absolutely no idea what
> happened".
> And it attempts to retry posting many times without getting the password.
> In the other case, my customer posts the files with embedded images.
> Sometimes it seems that solr throws TikaException of unknown cause.
> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2480) Text extraction of password
protected files
Posted by "Koji Sekiguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027553#comment-13027553 ]
Koji Sekiguchi commented on SOLR-2480:
--------------------------------------
BTW, I have a similar issue when using UIMA update processor, as sometimes UIMA annotators fail to extract meta data for some reason (eg Alchemy Web services stop). I'll open a separate ticket for it.
> Text extraction of password protected files
> -------------------------------------------
>
> Key: SOLR-2480
> URL: https://issues.apache.org/jira/browse/SOLR-2480
> Project: Solr
> Issue Type: Improvement
> Components: contrib - Solr Cell (Tika extraction)
> Affects Versions: 3.1
> Reporter: Shinichiro Abe
> Priority: Minor
>
> Proposal:
> There are password-protected files. PDF, Office documents in 2007 format/97 format.
> These files are posted using SolrCell.
> We do not have to read these files if we do not know the reading password of files.
> So, these files may not be extracted text.
> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
> This background:
> Now, when you post a password-protected file, solr returns 500 server error.
> Solr catches the error in ExtractingDocumentLoader and throws TikException.
> I use ManifoldCF.
> If the solr server responds 500, ManifoldCF judge is that "this
> document should be retried because I have absolutely no idea what
> happened".
> And it attempts to retry posting many times without getting the password.
> In the other case, my customer posts the files with embedded images.
> Sometimes it seems that solr throws TikaException of unknown cause.
> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2480) Text extraction of password
protected files
Posted by "Koji Sekiguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027547#comment-13027547 ]
Koji Sekiguchi commented on SOLR-2480:
--------------------------------------
Though I've not yet read entire comment SOLR-445, I don't think your requirement is same.
According to description of SOLR-445, the reporter wants Solr to skip the error <doc/> and continue adding the rest of <doc/> in <add>...</add>. But I think you want Solr to skip the content *field* because tika cannot extract it for some reasons but add meta data fields, right?
> Text extraction of password protected files
> -------------------------------------------
>
> Key: SOLR-2480
> URL: https://issues.apache.org/jira/browse/SOLR-2480
> Project: Solr
> Issue Type: Improvement
> Components: contrib - Solr Cell (Tika extraction)
> Affects Versions: 3.1
> Reporter: Shinichiro Abe
> Priority: Minor
>
> Proposal:
> There are password-protected files. PDF, Office documents in 2007 format/97 format.
> These files are posted using SolrCell.
> We do not have to read these files if we do not know the reading password of files.
> So, these files may not be extracted text.
> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
> This background:
> Now, when you post a password-protected file, solr returns 500 server error.
> Solr catches the error in ExtractingDocumentLoader and throws TikException.
> I use ManifoldCF.
> If the solr server responds 500, ManifoldCF judge is that "this
> document should be retried because I have absolutely no idea what
> happened".
> And it attempts to retry posting many times without getting the password.
> In the other case, my customer posts the files with embedded images.
> Sometimes it seems that solr throws TikaException of unknown cause.
> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2480) Text extraction of password
protected files
Posted by "Shinichiro Abe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027579#comment-13027579 ]
Shinichiro Abe commented on SOLR-2480:
--------------------------------------
{quote}
But I think you want Solr to skip the content field because tika cannot extract it for some reasons but add meta data fields, right?
{quote}
Yes, I want to post the metadate without contents that throw parse-error.
ExtractingDocumentLoader also should be fixed.
This patch expresses improvement ideas(1).
And I think SOLR-445 can resolve improvement ideas(2).
> Text extraction of password protected files
> -------------------------------------------
>
> Key: SOLR-2480
> URL: https://issues.apache.org/jira/browse/SOLR-2480
> Project: Solr
> Issue Type: Improvement
> Components: contrib - Solr Cell (Tika extraction)
> Affects Versions: 3.1
> Reporter: Shinichiro Abe
> Priority: Minor
> Attachments: SOLR-2480-idea1.patch
>
>
> Proposal:
> There are password-protected files. PDF, Office documents in 2007 format/97 format.
> These files are posted using SolrCell.
> We do not have to read these files if we do not know the reading password of files.
> So, these files may not be extracted text.
> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
> This background:
> Now, when you post a password-protected file, solr returns 500 server error.
> Solr catches the error in ExtractingDocumentLoader and throws TikException.
> I use ManifoldCF.
> If the solr server responds 500, ManifoldCF judge is that "this
> document should be retried because I have absolutely no idea what
> happened".
> And it attempts to retry posting many times without getting the password.
> In the other case, my customer posts the files with embedded images.
> Sometimes it seems that solr throws TikaException of unknown cause.
> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2480) Text extraction of password
protected files
Posted by "Koji Sekiguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13033429#comment-13033429 ]
Koji Sekiguchi commented on SOLR-2480:
--------------------------------------
bq. And I think SOLR-445 can resolve improvement ideas(2).
No. You should consider the difference between this issue and SOLR-445. (see my comment above)
As I understand your requirement that was described in Description, and it is quite similar SOLR-2512 that has been resolved, I'll try a patch that has ignoreErrors flag for TikaException.
I added an ability to ignore exceptions when trying to extract mata data from text in SOLR-2512, i.g. Solr indexed the text but gave up meta data. On the other hand, the ignore flag in this ticket is for giving up text but indexing meta data. It cannot be resolved by SOLR-445.
> Text extraction of password protected files
> -------------------------------------------
>
> Key: SOLR-2480
> URL: https://issues.apache.org/jira/browse/SOLR-2480
> Project: Solr
> Issue Type: Improvement
> Components: contrib - Solr Cell (Tika extraction)
> Affects Versions: 1.4.1, 3.1
> Reporter: Shinichiro Abe
> Assignee: Koji Sekiguchi
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2480-idea1.patch
>
>
> Proposal:
> There are password-protected files. PDF, Office documents in 2007 format/97 format.
> These files are posted using SolrCell.
> We do not have to read these files if we do not know the reading password of files.
> So, these files may not be extracted text.
> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
> This background:
> Now, when you post a password-protected file, solr returns 500 server error.
> Solr catches the error in ExtractingDocumentLoader and throws TikException.
> I use ManifoldCF.
> If the solr server responds 500, ManifoldCF judge is that "this
> document should be retried because I have absolutely no idea what
> happened".
> And it attempts to retry posting many times without getting the password.
> In the other case, my customer posts the files with embedded images.
> Sometimes it seems that solr throws TikaException of unknown cause.
> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2480) Text extraction of password
protected files
Posted by "Koji Sekiguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032247#comment-13032247 ]
Koji Sekiguchi commented on SOLR-2480:
--------------------------------------
bq. BTW, I have a similar issue when using UIMA update processor, as sometimes UIMA annotators fail to extract meta data for some reason (eg Alchemy Web services stop). I'll open a separate ticket for it.
Opened SOLR-2512.
> Text extraction of password protected files
> -------------------------------------------
>
> Key: SOLR-2480
> URL: https://issues.apache.org/jira/browse/SOLR-2480
> Project: Solr
> Issue Type: Improvement
> Components: contrib - Solr Cell (Tika extraction)
> Affects Versions: 3.1
> Reporter: Shinichiro Abe
> Priority: Minor
> Attachments: SOLR-2480-idea1.patch
>
>
> Proposal:
> There are password-protected files. PDF, Office documents in 2007 format/97 format.
> These files are posted using SolrCell.
> We do not have to read these files if we do not know the reading password of files.
> So, these files may not be extracted text.
> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
> This background:
> Now, when you post a password-protected file, solr returns 500 server error.
> Solr catches the error in ExtractingDocumentLoader and throws TikException.
> I use ManifoldCF.
> If the solr server responds 500, ManifoldCF judge is that "this
> document should be retried because I have absolutely no idea what
> happened".
> And it attempts to retry posting many times without getting the password.
> In the other case, my customer posts the files with embedded images.
> Sometimes it seems that solr throws TikaException of unknown cause.
> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2480) Text extraction of password protected
files
Posted by "Koji Sekiguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Sekiguchi updated SOLR-2480:
---------------------------------
Attachment: SOLR-2480.patch
New patch.
According to custom, ExtractingRequestHandlerTest class should be at o.a.s.handler.extraction, but curiously it was o.a.s.handler. I corrected it in this patch.
> Text extraction of password protected files
> -------------------------------------------
>
> Key: SOLR-2480
> URL: https://issues.apache.org/jira/browse/SOLR-2480
> Project: Solr
> Issue Type: Improvement
> Components: contrib - Solr Cell (Tika extraction)
> Affects Versions: 1.4.1, 3.1
> Reporter: Shinichiro Abe
> Assignee: Koji Sekiguchi
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2480-idea1.patch, SOLR-2480.patch, SOLR-2480.patch, SOLR-2480.patch, password-is-solrcell.docx
>
>
> Proposal:
> There are password-protected files. PDF, Office documents in 2007 format/97 format.
> These files are posted using SolrCell.
> We do not have to read these files if we do not know the reading password of files.
> So, these files may not be extracted text.
> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
> This background:
> Now, when you post a password-protected file, solr returns 500 server error.
> Solr catches the error in ExtractingDocumentLoader and throws TikException.
> I use ManifoldCF.
> If the solr server responds 500, ManifoldCF judge is that "this
> document should be retried because I have absolutely no idea what
> happened".
> And it attempts to retry posting many times without getting the password.
> In the other case, my customer posts the files with embedded images.
> Sometimes it seems that solr throws TikaException of unknown cause.
> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2480) Text extraction of password protected
files
Posted by "Koji Sekiguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Sekiguchi updated SOLR-2480:
---------------------------------
Affects Version/s: 1.4.1
Fix Version/s: 4.0
3.2
> Text extraction of password protected files
> -------------------------------------------
>
> Key: SOLR-2480
> URL: https://issues.apache.org/jira/browse/SOLR-2480
> Project: Solr
> Issue Type: Improvement
> Components: contrib - Solr Cell (Tika extraction)
> Affects Versions: 1.4.1, 3.1
> Reporter: Shinichiro Abe
> Assignee: Koji Sekiguchi
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2480-idea1.patch
>
>
> Proposal:
> There are password-protected files. PDF, Office documents in 2007 format/97 format.
> These files are posted using SolrCell.
> We do not have to read these files if we do not know the reading password of files.
> So, these files may not be extracted text.
> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
> This background:
> Now, when you post a password-protected file, solr returns 500 server error.
> Solr catches the error in ExtractingDocumentLoader and throws TikException.
> I use ManifoldCF.
> If the solr server responds 500, ManifoldCF judge is that "this
> document should be retried because I have absolutely no idea what
> happened".
> And it attempts to retry posting many times without getting the password.
> In the other case, my customer posts the files with embedded images.
> Sometimes it seems that solr throws TikaException of unknown cause.
> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Assigned] (SOLR-2480) Text extraction of password protected
files
Posted by "Koji Sekiguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Sekiguchi reassigned SOLR-2480:
------------------------------------
Assignee: Koji Sekiguchi
> Text extraction of password protected files
> -------------------------------------------
>
> Key: SOLR-2480
> URL: https://issues.apache.org/jira/browse/SOLR-2480
> Project: Solr
> Issue Type: Improvement
> Components: contrib - Solr Cell (Tika extraction)
> Affects Versions: 3.1
> Reporter: Shinichiro Abe
> Assignee: Koji Sekiguchi
> Priority: Minor
> Attachments: SOLR-2480-idea1.patch
>
>
> Proposal:
> There are password-protected files. PDF, Office documents in 2007 format/97 format.
> These files are posted using SolrCell.
> We do not have to read these files if we do not know the reading password of files.
> So, these files may not be extracted text.
> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
> This background:
> Now, when you post a password-protected file, solr returns 500 server error.
> Solr catches the error in ExtractingDocumentLoader and throws TikException.
> I use ManifoldCF.
> If the solr server responds 500, ManifoldCF judge is that "this
> document should be retried because I have absolutely no idea what
> happened".
> And it attempts to retry posting many times without getting the password.
> In the other case, my customer posts the files with embedded images.
> Sometimes it seems that solr throws TikaException of unknown cause.
> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2480) Text extraction of password protected
files
Posted by "Koji Sekiguchi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Koji Sekiguchi updated SOLR-2480:
---------------------------------
Attachment: password-is-solrcell.docx
SOLR-2480.patch
Attached the next patch and password protected word file that is used for test.
I added test cases for ignoreTikaException=true|false cases.
I think this is ready to commit.
> Text extraction of password protected files
> -------------------------------------------
>
> Key: SOLR-2480
> URL: https://issues.apache.org/jira/browse/SOLR-2480
> Project: Solr
> Issue Type: Improvement
> Components: contrib - Solr Cell (Tika extraction)
> Affects Versions: 1.4.1, 3.1
> Reporter: Shinichiro Abe
> Assignee: Koji Sekiguchi
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: SOLR-2480-idea1.patch, SOLR-2480.patch, SOLR-2480.patch, password-is-solrcell.docx
>
>
> Proposal:
> There are password-protected files. PDF, Office documents in 2007 format/97 format.
> These files are posted using SolrCell.
> We do not have to read these files if we do not know the reading password of files.
> So, these files may not be extracted text.
> My requirement is that these files should be processed normally without extracting text, and without throwing exception.
> This background:
> Now, when you post a password-protected file, solr returns 500 server error.
> Solr catches the error in ExtractingDocumentLoader and throws TikException.
> I use ManifoldCF.
> If the solr server responds 500, ManifoldCF judge is that "this
> document should be retried because I have absolutely no idea what
> happened".
> And it attempts to retry posting many times without getting the password.
> In the other case, my customer posts the files with embedded images.
> Sometimes it seems that solr throws TikaException of unknown cause.
> He wants to post just metadata without extracting text, but makes him stop posting by the exception.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org