[jira] [Commented] (TIKA-3282) OneNote Parser breaks non-ASCII Characters

["Adrian Diemer (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TIKA-3282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17276123#comment-17276123 ] 

Adrian Diemer commented on TIKA-3282:
-------------------------------------

The Patch is not working.

The reason for that is that any non ASCII character is filtered out by [line 123|https://github.com/apache/tika/blob/12effe10215d6903ba6c1fafaaa9b578d13b17c9/tika-parsers/tika-parsers-classic/tika-parsers-classic-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/onenote/OneNoteLegacyDumpStrings.java#L123] which means the character is skipped and processing of the previous characters is started instead.

I've tried to fix it like this: [^TIKA-3282-2.patch] but that didn't work either. There was a lot of clutter in the output.

> OneNote Parser breaks non-ASCII Characters
> ------------------------------------------
>
>                 Key: TIKA-3282
>                 URL: https://issues.apache.org/jira/browse/TIKA-3282
>             Project: Tika
>          Issue Type: Bug
>          Components: parser
>    Affects Versions: 1.24, 1.25, 1.24.1
>            Reporter: Adrian Diemer
>            Priority: Major
>         Attachments: TIKA-3282-2.patch, TIKA-3282.patch
>
>
> The OneNote parser doesn't seem to handle non-ASCII characters correctly.
> The reason for that seems to be that the characterset US_ASCII is used instead of ISO_8859_1 [here|https://github.com/apache/tika/blob/91ef4693363d31dba05de9eec1ab2d52a298c6b5/tika-parsers/tika-parsers-classic/tika-parsers-classic-modules/tika-parser-microsoft-module/src/main/java/org/apache/tika/parser/microsoft/onenote/OneNoteTreeWalker.java#L448].



--
This message was sent by Atlassian Jira
(v8.3.4#803005)