You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tika.apache.org by "Tilman Hausherr (Jira)" <ji...@apache.org> on 2024/07/13 14:22:00 UTC
[jira] [Updated] (TIKA-4278) TextAndCSVParser doesn't detect semicolon separated file
[ https://issues.apache.org/jira/browse/TIKA-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tilman Hausherr updated TIKA-4278:
----------------------------------
Description:
I ran the code from the attached SO issue and yes it doesn't detect semicolon separated files. The reason is this line in {{TextAndCSVParser.java}}:
{code:java}
private static final char[] DEFAULT_DELIMITERS = new char[]\{',', '\t'};
{code}
This is later uses by {{CSVSniffer}}. For some reason the other delimiters (pipe, colon and semicolon) aren't in that array, although they are in {{CHAR_TO_STRING_DELIMITER_MAP}}. I modified {{DEFAULT_DELIMITERS}} and now it works for semicolon.
Can I change this by adding the missing delimiters or was there a reason that I missed? Proposed change would change CSVSniffer so that delimiters is a set and then pass {{CHAR_TO_STRING_DELIMITER_MAP.keySet()}}.
was:
I ran the code from the attached SO issue and yes it doesn't detect semicolon separated files. The reason is this line in {{TextAndCSVParser.java}}:
{code:java}
private static final char[] DEFAULT_DELIMITERS = new char[]\{',', '\t'};
{code}
This is later uses by {{CSVSniffer}}. For some reason the other delimiters (pipe, colon and semicolon) aren't in that array, although they are in {{CHAR_TO_STRING_DELIMITER_MAP}}. I modified {{DEFAULT_DELIMITERS}} and now it works for semicolon.
Can I change this by adding the missing delimiters or was there a reason that I missed?
> TextAndCSVParser doesn't detect semicolon separated file
> --------------------------------------------------------
>
> Key: TIKA-4278
> URL: https://issues.apache.org/jira/browse/TIKA-4278
> Project: Tika
> Issue Type: Bug
> Components: parser
> Affects Versions: 2.9.2
> Reporter: Tilman Hausherr
> Priority: Major
>
> I ran the code from the attached SO issue and yes it doesn't detect semicolon separated files. The reason is this line in {{TextAndCSVParser.java}}:
> {code:java}
> private static final char[] DEFAULT_DELIMITERS = new char[]\{',', '\t'};
> {code}
> This is later uses by {{CSVSniffer}}. For some reason the other delimiters (pipe, colon and semicolon) aren't in that array, although they are in {{CHAR_TO_STRING_DELIMITER_MAP}}. I modified {{DEFAULT_DELIMITERS}} and now it works for semicolon.
> Can I change this by adding the missing delimiters or was there a reason that I missed? Proposed change would change CSVSniffer so that delimiters is a set and then pass {{CHAR_TO_STRING_DELIMITER_MAP.keySet()}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)