[jira] [Commented] (TIKA-4977) Rust .rs source is detected as application/rls-services+xml

["Philipp Ottlinger (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/TIKA-4977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18128541#comment-18128541 ] 

Philipp Ottlinger commented on TIKA-4977:
-----------------------------------------

Did you have a chance to see if the behaviour is the same with Tika 4.0.x as RAT already migrated to this version - thanks for reporting.

In Glasgow I had similar feedback that bash/shell scripts are recognized as binary as well ..... not sure if that is worth reporting as well.

> Rust .rs source is detected as application/rls-services+xml
> -----------------------------------------------------------
>
>                 Key: TIKA-4977
>                 URL: https://issues.apache.org/jira/browse/TIKA-4977
>             Project: Tika
>          Issue Type: Bug
>          Components: mime
>    Affects Versions: 3.2.3
>            Reporter: Zili Chen
>            Priority: Major
>
> A plain Rust source file is detected as application/rls-services+xml when its .rs filename is supplied, although content-only detection correctly returns text/plain.
> Reproduced with tika-core 3.2.3. The same *.rs mapping remains in the current main branch (fe5ba58bc1bee4f37217c32a05a9b74db7fa51ba).
> {code:java}
> import java.io.ByteArrayInputStream;
> import java.nio.charset.StandardCharsets;
> import org.apache.tika.Tika;
> import org.apache.tika.metadata.Metadata;
> import org.apache.tika.metadata.TikaCoreProperties;
> byte[] source = "fn main() {}\n".getBytes(StandardCharsets.UTF_8);
> Tika tika = new Tika();
> System.out.println(tika.detect(source));
> Metadata metadata = new Metadata();
> metadata.set(TikaCoreProperties.RESOURCE_NAME_KEY, "main.rs");
> System.out.println(tika.detect(new ByteArrayInputStream(source), metadata));
> {code}
> Actual output:
> {noformat}
> text/plain
> application/rls-services+xml
> {noformat}
> Changing only the resource name to main.txt returns text/plain. Rust files with an Apache license comment behave the same way. PNG and PDF bytes named main.rs retain their binary MIME types, so this concerns the filename hint applied to otherwise plain text.
> The MIME database maps *.rs to application/rls-services+xml:
> https://github.com/apache/tika/blob/3.2.3/tika-core/src/main/resources/org/apache/tika/mime/tika-mimetypes.xml#L1010
> That extension is legitimately registered for RLS XML by RFC 4826:
> https://www.iana.org/assignments/media-types/application/rls-services%2Bxml
> Please distinguish Rust source from RLS XML when the .rs extension is supplied, while preserving genuine RLS XML detection and binary content detection. A Rust text MIME type, or a text/plain fallback for non-XML content, would avoid reporting Rust code as XML. Simply adding another unrelated *.rs glob in a custom MIME file raises a Conflicting extension pattern exception in 3.2.3.
> This was found while investigating Apache RAT 0.18 skipping Rust license checks. RAT separately maps this XML subtype to BINARY instead of scanning it; a fix for that downstream classification is being prepared in RAT. This ticket concerns the inaccurate Rust MIME result itself.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)