You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-dev@lucene.apache.org by "Peter Wolanin (JIRA)" <ji...@apache.org> on 2009/07/13 21:29:14 UTC

[jira] Created: (SOLR-1274) Provide multiple output formats in extract-only mode for tika handler

Provide multiple output formats in extract-only mode for tika handler
---------------------------------------------------------------------

                 Key: SOLR-1274
                 URL: https://issues.apache.org/jira/browse/SOLR-1274
             Project: Solr
          Issue Type: New Feature
    Affects Versions: 1.4
            Reporter: Peter Wolanin
            Priority: Minor
             Fix For: 1.4


The proposed feature is to accept a URL parameter when using extract-only mode to specify an output format.  This parameter might just overload the existing "ext.extract.only" so that one can optionally specify a format, e.g. false|true|xml|text  where true and xml give the same response (i.e. xml remains the default)

I had been assuming that I could choose among possible tika output
formats when using the extracting request handler in extract-only mode
as if from the CLI with the tika jar:

   -x or --xml        Output XHTML content (default)
   -h or --html       Output HTML content
   -t or --text       Output plain text content
   -m or --metadata   Output only metadata

However, looking at the docs and source, it seems that only the xml
option is available (hard-coded) in ExtractingDocumentLoader.java
{code}
serializer = new XMLSerializer(writer, new OutputFormat("XML", "UTF-8", true));
{code}

Providing at least a plain-text response seems to work if you change the serializer to a TextSerializer (org.apache.xml.serialize.TextSerializer).





-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Resolved: (SOLR-1274) Provide multiple output formats in extract-only mode for tika handler

Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Grant Ingersoll resolved SOLR-1274.
-----------------------------------

    Resolution: Fixed

I committed this patch, plus a test for it.

> Provide multiple output formats in extract-only mode for tika handler
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1274
>                 URL: https://issues.apache.org/jira/browse/SOLR-1274
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.4
>            Reporter: Peter Wolanin
>            Assignee: Grant Ingersoll
>            Priority: Minor
>             Fix For: 1.4
>
>         Attachments: SOLR-1274.patch, SOLR-1274.patch
>
>
> The proposed feature is to accept a URL parameter when using extract-only mode to specify an output format.  This parameter might just overload the existing "ext.extract.only" so that one can optionally specify a format, e.g. false|true|xml|text  where true and xml give the same response (i.e. xml remains the default)
> I had been assuming that I could choose among possible tika output
> formats when using the extracting request handler in extract-only mode
> as if from the CLI with the tika jar:
>    -x or --xml        Output XHTML content (default)
>    -h or --html       Output HTML content
>    -t or --text       Output plain text content
>    -m or --metadata   Output only metadata
> However, looking at the docs and source, it seems that only the xml
> option is available (hard-coded) in ExtractingDocumentLoader.java
> {code}
> serializer = new XMLSerializer(writer, new OutputFormat("XML", "UTF-8", true));
> {code}
> Providing at least a plain-text response seems to work if you change the serializer to a TextSerializer (org.apache.xml.serialize.TextSerializer).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (SOLR-1274) Provide multiple output formats in extract-only mode for tika handler

Posted by "Peter Wolanin (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SOLR-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731437#action_12731437 ] 

Peter Wolanin commented on SOLR-1274:
-------------------------------------

A minimal version of this would be pretty trivial as far as features go, and I'd thought Yonik was indicating on the e-mail list that it would be a reasonable follow on to his last patch in the linked issue.

> Provide multiple output formats in extract-only mode for tika handler
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1274
>                 URL: https://issues.apache.org/jira/browse/SOLR-1274
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.4
>            Reporter: Peter Wolanin
>            Priority: Minor
>             Fix For: 1.4
>
>
> The proposed feature is to accept a URL parameter when using extract-only mode to specify an output format.  This parameter might just overload the existing "ext.extract.only" so that one can optionally specify a format, e.g. false|true|xml|text  where true and xml give the same response (i.e. xml remains the default)
> I had been assuming that I could choose among possible tika output
> formats when using the extracting request handler in extract-only mode
> as if from the CLI with the tika jar:
>    -x or --xml        Output XHTML content (default)
>    -h or --html       Output HTML content
>    -t or --text       Output plain text content
>    -m or --metadata   Output only metadata
> However, looking at the docs and source, it seems that only the xml
> option is available (hard-coded) in ExtractingDocumentLoader.java
> {code}
> serializer = new XMLSerializer(writer, new OutputFormat("XML", "UTF-8", true));
> {code}
> Providing at least a plain-text response seems to work if you change the serializer to a TextSerializer (org.apache.xml.serialize.TextSerializer).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (SOLR-1274) Provide multiple output formats in extract-only mode for tika handler

Posted by "Peter Wolanin (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Peter Wolanin updated SOLR-1274:
--------------------------------

    Attachment: SOLR-1274.patch

Here's a patch that's nearly there, but somehow I'm missing something in how java behaves.  The param is getting picked up, but this line never evals as true, even when the param is parsed right:

{code}
  if (extractFormat == "text") {
{code}


If I set it to
{code}
  if (true) {
{code}

I get the desired text-only output.

> Provide multiple output formats in extract-only mode for tika handler
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1274
>                 URL: https://issues.apache.org/jira/browse/SOLR-1274
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.4
>            Reporter: Peter Wolanin
>            Priority: Minor
>             Fix For: 1.4
>
>         Attachments: SOLR-1274.patch
>
>
> The proposed feature is to accept a URL parameter when using extract-only mode to specify an output format.  This parameter might just overload the existing "ext.extract.only" so that one can optionally specify a format, e.g. false|true|xml|text  where true and xml give the same response (i.e. xml remains the default)
> I had been assuming that I could choose among possible tika output
> formats when using the extracting request handler in extract-only mode
> as if from the CLI with the tika jar:
>    -x or --xml        Output XHTML content (default)
>    -h or --html       Output HTML content
>    -t or --text       Output plain text content
>    -m or --metadata   Output only metadata
> However, looking at the docs and source, it seems that only the xml
> option is available (hard-coded) in ExtractingDocumentLoader.java
> {code}
> serializer = new XMLSerializer(writer, new OutputFormat("XML", "UTF-8", true));
> {code}
> Providing at least a plain-text response seems to work if you change the serializer to a TextSerializer (org.apache.xml.serialize.TextSerializer).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (SOLR-1274) Provide multiple output formats in extract-only mode for tika handler

Posted by "Otis Gospodnetic (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SOLR-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738793#action_12738793 ] 

Otis Gospodnetic commented on SOLR-1274:
----------------------------------------

Try:
{code}
if ("text".equals(extractFormat)) {
{code}

:)


> Provide multiple output formats in extract-only mode for tika handler
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1274
>                 URL: https://issues.apache.org/jira/browse/SOLR-1274
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.4
>            Reporter: Peter Wolanin
>            Priority: Minor
>             Fix For: 1.4
>
>         Attachments: SOLR-1274.patch
>
>
> The proposed feature is to accept a URL parameter when using extract-only mode to specify an output format.  This parameter might just overload the existing "ext.extract.only" so that one can optionally specify a format, e.g. false|true|xml|text  where true and xml give the same response (i.e. xml remains the default)
> I had been assuming that I could choose among possible tika output
> formats when using the extracting request handler in extract-only mode
> as if from the CLI with the tika jar:
>    -x or --xml        Output XHTML content (default)
>    -h or --html       Output HTML content
>    -t or --text       Output plain text content
>    -m or --metadata   Output only metadata
> However, looking at the docs and source, it seems that only the xml
> option is available (hard-coded) in ExtractingDocumentLoader.java
> {code}
> serializer = new XMLSerializer(writer, new OutputFormat("XML", "UTF-8", true));
> {code}
> Providing at least a plain-text response seems to work if you change the serializer to a TextSerializer (org.apache.xml.serialize.TextSerializer).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Assigned: (SOLR-1274) Provide multiple output formats in extract-only mode for tika handler

Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Grant Ingersoll reassigned SOLR-1274:
-------------------------------------

    Assignee: Grant Ingersoll

> Provide multiple output formats in extract-only mode for tika handler
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1274
>                 URL: https://issues.apache.org/jira/browse/SOLR-1274
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.4
>            Reporter: Peter Wolanin
>            Assignee: Grant Ingersoll
>            Priority: Minor
>             Fix For: 1.4
>
>         Attachments: SOLR-1274.patch, SOLR-1274.patch
>
>
> The proposed feature is to accept a URL parameter when using extract-only mode to specify an output format.  This parameter might just overload the existing "ext.extract.only" so that one can optionally specify a format, e.g. false|true|xml|text  where true and xml give the same response (i.e. xml remains the default)
> I had been assuming that I could choose among possible tika output
> formats when using the extracting request handler in extract-only mode
> as if from the CLI with the tika jar:
>    -x or --xml        Output XHTML content (default)
>    -h or --html       Output HTML content
>    -t or --text       Output plain text content
>    -m or --metadata   Output only metadata
> However, looking at the docs and source, it seems that only the xml
> option is available (hard-coded) in ExtractingDocumentLoader.java
> {code}
> serializer = new XMLSerializer(writer, new OutputFormat("XML", "UTF-8", true));
> {code}
> Providing at least a plain-text response seems to work if you change the serializer to a TextSerializer (org.apache.xml.serialize.TextSerializer).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (SOLR-1274) Provide multiple output formats in extract-only mode for tika handler

Posted by "Grant Ingersoll (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SOLR-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738648#action_12738648 ] 

Grant Ingersoll commented on SOLR-1274:
---------------------------------------

Peter, if you have a patch, please add, otherwise I will mark as 1.5.

FWIW, I think the Serializer approach is likely to only work for XML and Text.  If you want HTML, etc., then we need to use a Transformer, which is what Tika CLI appears to be doing.

> Provide multiple output formats in extract-only mode for tika handler
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1274
>                 URL: https://issues.apache.org/jira/browse/SOLR-1274
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.4
>            Reporter: Peter Wolanin
>            Priority: Minor
>             Fix For: 1.4
>
>
> The proposed feature is to accept a URL parameter when using extract-only mode to specify an output format.  This parameter might just overload the existing "ext.extract.only" so that one can optionally specify a format, e.g. false|true|xml|text  where true and xml give the same response (i.e. xml remains the default)
> I had been assuming that I could choose among possible tika output
> formats when using the extracting request handler in extract-only mode
> as if from the CLI with the tika jar:
>    -x or --xml        Output XHTML content (default)
>    -h or --html       Output HTML content
>    -t or --text       Output plain text content
>    -m or --metadata   Output only metadata
> However, looking at the docs and source, it seems that only the xml
> option is available (hard-coded) in ExtractingDocumentLoader.java
> {code}
> serializer = new XMLSerializer(writer, new OutputFormat("XML", "UTF-8", true));
> {code}
> Providing at least a plain-text response seems to work if you change the serializer to a TextSerializer (org.apache.xml.serialize.TextSerializer).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Commented: (SOLR-1274) Provide multiple output formats in extract-only mode for tika handler

Posted by "Noble Paul (JIRA)" <ji...@apache.org>.
    [ https://issues.apache.org/jira/browse/SOLR-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12731291#action_12731291 ] 

Noble Paul commented on SOLR-1274:
----------------------------------

We are in the process of a release . New feature requests are not generally entertained. Shall we move it to 1.5 ?

> Provide multiple output formats in extract-only mode for tika handler
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1274
>                 URL: https://issues.apache.org/jira/browse/SOLR-1274
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.4
>            Reporter: Peter Wolanin
>            Priority: Minor
>             Fix For: 1.4
>
>
> The proposed feature is to accept a URL parameter when using extract-only mode to specify an output format.  This parameter might just overload the existing "ext.extract.only" so that one can optionally specify a format, e.g. false|true|xml|text  where true and xml give the same response (i.e. xml remains the default)
> I had been assuming that I could choose among possible tika output
> formats when using the extracting request handler in extract-only mode
> as if from the CLI with the tika jar:
>    -x or --xml        Output XHTML content (default)
>    -h or --html       Output HTML content
>    -t or --text       Output plain text content
>    -m or --metadata   Output only metadata
> However, looking at the docs and source, it seems that only the xml
> option is available (hard-coded) in ExtractingDocumentLoader.java
> {code}
> serializer = new XMLSerializer(writer, new OutputFormat("XML", "UTF-8", true));
> {code}
> Providing at least a plain-text response seems to work if you change the serializer to a TextSerializer (org.apache.xml.serialize.TextSerializer).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


[jira] Updated: (SOLR-1274) Provide multiple output formats in extract-only mode for tika handler

Posted by "Peter Wolanin (JIRA)" <ji...@apache.org>.
     [ https://issues.apache.org/jira/browse/SOLR-1274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Peter Wolanin updated SOLR-1274:
--------------------------------

    Attachment: SOLR-1274.patch

Well, indeed - something like that works better.


> Provide multiple output formats in extract-only mode for tika handler
> ---------------------------------------------------------------------
>
>                 Key: SOLR-1274
>                 URL: https://issues.apache.org/jira/browse/SOLR-1274
>             Project: Solr
>          Issue Type: New Feature
>    Affects Versions: 1.4
>            Reporter: Peter Wolanin
>            Priority: Minor
>             Fix For: 1.4
>
>         Attachments: SOLR-1274.patch, SOLR-1274.patch
>
>
> The proposed feature is to accept a URL parameter when using extract-only mode to specify an output format.  This parameter might just overload the existing "ext.extract.only" so that one can optionally specify a format, e.g. false|true|xml|text  where true and xml give the same response (i.e. xml remains the default)
> I had been assuming that I could choose among possible tika output
> formats when using the extracting request handler in extract-only mode
> as if from the CLI with the tika jar:
>    -x or --xml        Output XHTML content (default)
>    -h or --html       Output HTML content
>    -t or --text       Output plain text content
>    -m or --metadata   Output only metadata
> However, looking at the docs and source, it seems that only the xml
> option is available (hard-coded) in ExtractingDocumentLoader.java
> {code}
> serializer = new XMLSerializer(writer, new OutputFormat("XML", "UTF-8", true));
> {code}
> Providing at least a plain-text response seems to work if you change the serializer to a TextSerializer (org.apache.xml.serialize.TextSerializer).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.