You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tika.apache.org by "Tyler Palsulich (JIRA)" <ji...@apache.org> on 2015/03/20 21:40:38 UTC

[jira] [Updated] (TIKA-1308) Support in memory parse mode(don't create temp file): to support run Tika in GAE

     [ https://issues.apache.org/jira/browse/TIKA-1308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tyler Palsulich updated TIKA-1308:
----------------------------------
    Description: 
I am trying to use Tika in GAE and write a simple servlet to extract meta data info from jpeg:
{code}
String urlStr = req.getParameter("imageUrl");
byte[] oldImageData = IOUtils.toByteArray(new URL(urlStr));

ByteArrayInputStream bais = new ByteArrayInputStream(oldImageData);
Metadata metadata = new Metadata();
BodyContentHandler ch = new BodyContentHandler();
AutoDetectParser parser = new AutoDetectParser();
parser.parse(bais, ch, metadata, new ParseContext());
bais.close();
{code}
This fails with exception:
{code}
Caused by: java.lang.SecurityException: Unable to create temporary file
	at java.io.File.createTempFile(File.java:1986)
	at org.apache.tika.io.TemporaryResources.createTemporaryFile(TemporaryResources.java:66)
	at org.apache.tika.io.TikaInputStream.getFile(TikaInputStream.java:533)
	at org.apache.tika.parser.jpeg.JpegParser.parse(JpegParser.java:56)
	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242
{code}
Checked the code, in org.apache.tika.parser.jpeg.JpegParser.parse(InputStream, ContentHandler, Metadata, ParseContext), it creates a temp file from the input stream.

I can understand why tika create temp file from the stream: so tika can parse it multiple times.

But as GAE and other cloud servers are getting more popular, is it possible to avoid create temp file: instead we can copy the origin stream to a byteArray stream, so tika can also parse it multiple times.
-- This will have a limit on the file size, as tika keeps the whole file in memory, but this can make tika work in GAE and maybe other cloud server.

We can add a parameter in parser.parse to indicate whether do in memory parse only.
 

  was:
I am trying to use Tika in GAE and write a simple servlet to extract meta data info from jpeg:

String urlStr = req.getParameter("imageUrl");
byte[] oldImageData = IOUtils.toByteArray(new URL(urlStr));

ByteArrayInputStream bais = new ByteArrayInputStream(oldImageData);
Metadata metadata = new Metadata();
BodyContentHandler ch = new BodyContentHandler();
AutoDetectParser parser = new AutoDetectParser();
parser.parse(bais, ch, metadata, new ParseContext());
bais.close();

This fails with exception:
Caused by: java.lang.SecurityException: Unable to create temporary file
	at java.io.File.createTempFile(File.java:1986)
	at org.apache.tika.io.TemporaryResources.createTemporaryFile(TemporaryResources.java:66)
	at org.apache.tika.io.TikaInputStream.getFile(TikaInputStream.java:533)
	at org.apache.tika.parser.jpeg.JpegParser.parse(JpegParser.java:56)
	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242

Checked the code, in org.apache.tika.parser.jpeg.JpegParser.parse(InputStream, ContentHandler, Metadata, ParseContext), it creates a temp file from the input stream.

I can understand why tika create temp file from the stream: so tika can parse it multiple times.

But as GAE and other cloud servers are getting more popular, is it possible to avoid create temp file: instead we can copy the origin stream to a byteArray stream, so tika can also parse it multiple times.
-- This will have a limit on the file size, as tika keeps the whole file in memory, but this can make tika work in GAE and maybe other cloud server.

We can add a parameter in parser.parse to indicate whether do in memory parse only.
 


> Support in memory parse mode(don't create temp file): to support run Tika in GAE
> --------------------------------------------------------------------------------
>
>                 Key: TIKA-1308
>                 URL: https://issues.apache.org/jira/browse/TIKA-1308
>             Project: Tika
>          Issue Type: Improvement
>          Components: parser
>    Affects Versions: 1.5
>            Reporter: jefferyyuan
>              Labels: gae
>             Fix For: 1.8
>
>
> I am trying to use Tika in GAE and write a simple servlet to extract meta data info from jpeg:
> {code}
> String urlStr = req.getParameter("imageUrl");
> byte[] oldImageData = IOUtils.toByteArray(new URL(urlStr));
> ByteArrayInputStream bais = new ByteArrayInputStream(oldImageData);
> Metadata metadata = new Metadata();
> BodyContentHandler ch = new BodyContentHandler();
> AutoDetectParser parser = new AutoDetectParser();
> parser.parse(bais, ch, metadata, new ParseContext());
> bais.close();
> {code}
> This fails with exception:
> {code}
> Caused by: java.lang.SecurityException: Unable to create temporary file
> 	at java.io.File.createTempFile(File.java:1986)
> 	at org.apache.tika.io.TemporaryResources.createTemporaryFile(TemporaryResources.java:66)
> 	at org.apache.tika.io.TikaInputStream.getFile(TikaInputStream.java:533)
> 	at org.apache.tika.parser.jpeg.JpegParser.parse(JpegParser.java:56)
> 	at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:242
> {code}
> Checked the code, in org.apache.tika.parser.jpeg.JpegParser.parse(InputStream, ContentHandler, Metadata, ParseContext), it creates a temp file from the input stream.
> I can understand why tika create temp file from the stream: so tika can parse it multiple times.
> But as GAE and other cloud servers are getting more popular, is it possible to avoid create temp file: instead we can copy the origin stream to a byteArray stream, so tika can also parse it multiple times.
> -- This will have a limit on the file size, as tika keeps the whole file in memory, but this can make tika work in GAE and maybe other cloud server.
> We can add a parameter in parser.parse to indicate whether do in memory parse only.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)