You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stormcrawler.apache.org by "rzo1 (via GitHub)" <gi...@apache.org> on 2026/09/23 17:35:26 UTC

[PR] #2097 Tika ParserBolt: optional timeout for the parse with parser.tika.timeout [stormcrawler]

rzo1 opened a new pull request, #2182:
URL: https://github.com/apache/stormcrawler/pull/2182

   Fixes #2097 (the remaining part after #2174).
   
   Adds `parser.tika.timeout` in milliseconds, default -1 (no limit). When set, the parse runs on a separate thread and a document which takes longer is sent to the status stream as an ERROR with `parse timeout`. The thread is interrupted and the parse stops at its next SAX event; a parser stuck without output keeps its thread and the following documents use a new one.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] #2097 Tika ParserBolt: optional timeout for the parse with parser.tika.timeout [stormcrawler]

Posted by "rzo1 (via GitHub)" <gi...@apache.org>.
rzo1 commented on PR #2182:
URL: https://github.com/apache/stormcrawler/pull/2182#issuecomment-5800383878

   @tballison As my go-to-TIKA expert: would you recommend PipesFork for SC? ;-)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] #2097 Tika ParserBolt: optional timeout for the parse with parser.tika.timeout [stormcrawler]

Posted by "rzo1 (via GitHub)" <gi...@apache.org>.
rzo1 commented on PR #2182:
URL: https://github.com/apache/stormcrawler/pull/2182#issuecomment-5800494254

   > I'm having an agent draft a PR.
   
   Would be great :) closing here !


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] #2097 Tika ParserBolt: optional timeout for the parse with parser.tika.timeout [stormcrawler]

Posted by "tballison (via GitHub)" <gi...@apache.org>.
tballison commented on PR #2182:
URL: https://github.com/apache/stormcrawler/pull/2182#issuecomment-5800005990

   I may misunderstand how Tika is used in StormCrawler, but interrupting a Tika thread has no guarantees that it will stop parsing unless you're using Tika pipes parser?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] #2097 Tika ParserBolt: optional timeout for the parse with parser.tika.timeout [stormcrawler]

Posted by "tballison (via GitHub)" <gi...@apache.org>.
tballison commented on PR #2182:
URL: https://github.com/apache/stormcrawler/pull/2182#issuecomment-5800121010

   Ref: https://tika.apache.org/docs/4.0.x/using-tika/java-api/index.html
   
   I'm drafting a new page on how to call tika pipes within Java. Have your :robot: see if the PipesForkParser is a good fit.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] #2097 Tika ParserBolt: optional timeout for the parse with parser.tika.timeout [stormcrawler]

Posted by "tballison (via GitHub)" <gi...@apache.org>.
tballison commented on PR #2182:
URL: https://github.com/apache/stormcrawler/pull/2182#issuecomment-5800474954

   I'm having an agent draft a PR. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] #2097 Tika ParserBolt: optional timeout for the parse with parser.tika.timeout [stormcrawler]

Posted by "tballison (via GitHub)" <gi...@apache.org>.
tballison commented on PR #2182:
URL: https://github.com/apache/stormcrawler/pull/2182#issuecomment-5800018083

   If you want to test this rigorously, a) I can send you a fun PDF or b) you can use our MockParser from tika-core test jar to instrument a timing out parser.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] #2097 Tika ParserBolt: optional timeout for the parse with parser.tika.timeout [stormcrawler]

Posted by "rzo1 (via GitHub)" <gi...@apache.org>.
rzo1 commented on PR #2182:
URL: https://github.com/apache/stormcrawler/pull/2182#issuecomment-5800346127

   Thanks Tim. Yes, the interrupt only works if the parser still writes to the handler, no? If it loops without output the thread keeps running, the bolt just stops waiting for it. The docs do overclaim :D
   
   Please send me the PDF, I'll try it against this branch.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] #2097 Tika ParserBolt: optional timeout for the parse with parser.tika.timeout [stormcrawler]

Posted by "rzo1 (via GitHub)" <gi...@apache.org>.
rzo1 commented on PR #2182:
URL: https://github.com/apache/stormcrawler/pull/2182#issuecomment-5800440462

   I convert this to draft and have my agents take a look at the PDF you send me + PipesFork for SC. Once this is in and tested, I will re-open this for ready to review ;-) - happy to get your feedback on it.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] #2097 Tika ParserBolt: optional timeout for the parse with parser.tika.timeout [stormcrawler]

Posted by "tballison (via GitHub)" <gi...@apache.org>.
tballison commented on PR #2182:
URL: https://github.com/apache/stormcrawler/pull/2182#issuecomment-5800403990

   Yes! Please! Then you'd actually be in Tika's security model!
   
   There are caveats about sizing/cpus etc. That's something that stormcrawler has to figure out. I can help.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] #2097 Tika ParserBolt: optional timeout for the parse with parser.tika.timeout [stormcrawler]

Posted by "rzo1 (via GitHub)" <gi...@apache.org>.
rzo1 closed pull request #2182: #2097 Tika ParserBolt: optional timeout for the parse with parser.tika.timeout
URL: https://github.com/apache/stormcrawler/pull/2182


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] #2097 Tika ParserBolt: optional timeout for the parse with parser.tika.timeout [stormcrawler]

Posted by "rzo1 (via GitHub)" <gi...@apache.org>.
rzo1 commented on PR #2182:
URL: https://github.com/apache/stormcrawler/pull/2182#issuecomment-5800496458

   Thx @tballison 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org