You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tika.apache.org by "dschmidt (via GitHub)" <gi...@apache.org> on 2026/09/01 21:46:30 UTC
[PR] TIKA-4869: Emit the video of a motion photo as an embedded document [tika]
dschmidt opened a new pull request, #3115:
URL: https://github.com/apache/tika/pull/3115
A motion photo carries a short video after the image, and Tika already reads the XMP that says where it is, but not the video itself.
Two formats, one arithmetic: a Motion Photo lists its parts in `Container:Directory`, the primary image first and the rest tightly packed after it, each with an `Item:Length` and an optional `Item:Padding`; the older MicroVideo gives `Camera:MicroVideoOffset`, the number of bytes from the end of the file to the start of the video. Either way the video ends at the end of the file, so its start follows from its length.
The image parsers emit it as an ATTACHMENT embedded document named `motion-photo.<ext>`, parsed like any other embedded document. Since the same XMP goes through the same extractor for HEIC and AVIF motion photos, and their video sits in a trailing `mpvd` box whose 8 byte header is the primary item's padding, the same computation covers those too; I have no sample file to prove it with.
What is at the computed offset is typed by content, without the declared `Item:Mime` as a hint: a hint would let a wrong length pass as a video, since the mime magic returns the hint when it recognizes nothing. Nothing is emitted when the length does not fit the file or when detection recognizes nothing there, and the image records why. That case is not exotic: sharing a motion photo out of the Android gallery leaves the flag and takes the video, which the tests cover with a file that Google Photos would produce.
The name comes from the declared `Item:Mime` where the format has one. A MicroVideo does not, so its video is named from detection, which today makes it `motion-photo.qt`: Tika's magic for video/mp4 only matches the brands mp41 and mp42, and a phone writes isom. #3114 fixes that detection, and with it in place both formats name the video `motion-photo.mp4`. The two changes are independent otherwise.
The two test files declared a video in their XMP without carrying one; they now hold a small real MP4, and a third one covers the shared-without-video case.
Verified on a server built from this branch: both files answer with the video as an embedded document, `/unpack` returns its bytes unchanged, a plain JPEG is untouched, and a truncated file yields no embedded document but a warning.
https://issues.apache.org/jira/browse/TIKA-4869
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tika.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] TIKA-4869: Emit the video of a motion photo as an embedded document [tika]
Posted by "tballison (via GitHub)" <gi...@apache.org>.
tballison merged PR #3115:
URL: https://github.com/apache/tika/pull/3115
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tika.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] TIKA-4869: Emit the video of a motion photo as an embedded document [tika]
Posted by "tballison (via GitHub)" <gi...@apache.org>.
tballison commented on PR #3115:
URL: https://github.com/apache/tika/pull/3115#issuecomment-5508249776
This is really nice.
My bot has maybe one or two things worth changing. The hygiene thing with MediaType.parse should be fixed now in main(?).
Let me know what you think, and thank you!
```
1. edge-case — the offset gate accepts any recognized type, not a video; that makes a JPEG recursive
MotionPhoto.java:222 — if (type == null || MediaType.OCTET_STREAM.equals(type)) { return; }
Two measured consequences:
- Replace the trailing 1583 bytes of testJPEG_MotionPhoto.jpg with ASCII text → a second document is emitted, text/plain, named motion-photo.mp4. CHANGES.txt and the PR description both say nothing is emitted "when the bytes there are not a video". That is
not what the code does.
- outer || inner, where outer is the fixture with Item:Length rewritten to |inner| and inner is a motion photo → 3 documents: depth 1 image/jpeg named motion-photo.mp4, depth 2 its own video/mp4. Chained 1500 deep (~1.4 KB per link, 2.19 MB input), a plain
AutoDetectParser.parse → StackOverflowError.
The depth cap that saves the /rmeta path is AbstractRecursiveParserWrapperHandler.MAX_DEPTH = 100, which only exists on the RecursiveParserWrapper path. ParseRecord.maxEmbeddedDepth defaults to -1, so /tika, tika-app text output and the Tika facade have no
guard. Even at the capped 100 levels each level re-spools the whole remaining suffix via tis.getPath(), so a 50 MB input writes gigabytes of temp files.
Fix is one line and is what the PR already claims: require "video".equals(type.getType()). It matches both specs (the item's semantic is the video), and it closes the recursion for good, since a video parser never re-enters this code. Add the missing test —
a trailer that is recognized but not a video; testDeclaredVideoIsNotOne only covers unrecognized zeros.
2. edge-case — extension from the declaration, content type from detection
MotionPhoto.java:221-233. The two can disagree (motion-photo.mp4 on text/plain, above). The PR's rationale — "the parse that follows knows the format for certain, but by then the name is fixed" — is about the parse; the detected type is already in hand at
line 221, before the name is set at 229. Also RESOURCE_NAME_EXTENSION_INFERRED=true is documented as "inferred by Tika (e.g. from content type detection) rather than provided by the original document", and here it is the document's declaration. If the
declaration is deliberately preferred, worth saying why detection-at-221 was rejected.
3. edge-case (low) — IOException on the video path fails the whole image
file(tis) maps getPath()'s IOException to null, but Files.size (:217), detect (:221) and region (:239) propagate out of the image parser. That contradicts the stated contract ("a video out of reach is no reason to fail an image that parsed"). Cheap fix:
extend the guard over the rest of the body.
Hygiene
- MediaType.parse(metadata.get(... "Item:Mime")) (:289) interns an attacker-controlled string into the static, never-evicted MediaType.SIMPLE_TYPES — the same class of thing TIKA-4826/4862 are removing. The value is only used for an extension, and
EmbeddedDocumentUtil.getExtensionForMediaType(String) already takes a String and normalizes itself; dropping the parse removes the interning and a field.
- CHANGES.txt "the bytes there are not a video" — fixed by finding 1.
- parseEmbedded(..., outputHtml=false) with no <div class="embedded"> of its own; EMFParser, the closest precedent, passes true.
- 1583 is hardcoded in four places across two tests; derive it from the fixture.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tika.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] TIKA-4869: Emit the video of a motion photo as an embedded document [tika]
Posted by "tballison (via GitHub)" <gi...@apache.org>.
tballison commented on PR #3115:
URL: https://github.com/apache/tika/pull/3115#issuecomment-5508387040
On 1., I'm frankly not certain that we need to require video. If someone sticks a PDF after an image, we should extract it. I do want to avoid the Stackoverflow, though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tika.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] TIKA-4869: Emit the video of a motion photo as an embedded document [tika]
Posted by "dschmidt (via GitHub)" <gi...@apache.org>.
dschmidt commented on code in PR #3115:
URL: https://github.com/apache/tika/pull/3115#discussion_r3915916614
##########
tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/main/java/org/apache/tika/parser/image/MotionPhoto.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.parser.image;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+
+import org.xml.sax.SAXException;
+
+import org.apache.tika.extractor.EmbeddedDocumentExtractor;
+import org.apache.tika.extractor.EmbeddedDocumentUtil;
+import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.Google;
+import org.apache.tika.metadata.HttpHeaders;
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.metadata.TikaCoreProperties;
+import org.apache.tika.mime.MediaType;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.sax.EmbeddedContentHandler;
+import org.apache.tika.sax.XHTMLContentHandler;
+
+/**
+ * The video of a motion photo, appended after the image and described by the
+ * XMP the image parsers already extract (TIKA-4869):
+ * <ul>
+ * <li>a Motion Photo lists its parts in {@code Container:Directory}: the
+ * primary image first, the video last with nothing after it, each with
+ * an {@code Item:Length};</li>
+ * <li>the older MicroVideo gives {@code Camera:MicroVideoOffset}, the
+ * number of bytes from the end of the file to the start of the video.</li>
+ * </ul>
+ * Either way the video ends at the end of the file, so its start follows from
+ * its length. What is found there is typed by content: the declared
+ * {@code Item:Mime} is not used as a detection hint, because a hint would
+ * make a wrong length pass as a video, and nothing is emitted when detection
+ * recognizes nothing.
+ * <p>
+ * The same holds for HEIC motion photos, whose video sits in a trailing
+ * {@code mpvd} box; its 8 byte header is the primary item's padding, so the
+ * video still ends at the end of the file.
+ */
+final class MotionPhoto {
+
+ /**
+ * The name the video is emitted under, with the extension of whatever it
+ * turns out to be.
+ */
+ private static final String NAME = "motion-photo";
+
+ private static final String ITEM = "]/Container:Item/";
+ private static final String DIRECTORY = "xmp-raw:Container:Directory[";
+
+ /**
+ * The {@code Item:Semantic} of the video.
+ */
+ private static final String MOTION_PHOTO = "MotionPhoto";
+
+ /**
+ * A directory holds a handful of items; this only bounds the walk.
+ */
+ private static final int MAX_ITEMS = 64;
+
+ /**
+ * Enough of the video for the detectors to recognize it.
+ */
+ private static final int DETECTION_PREFIX = 8 * 1024;
+
+ /**
+ * The branch below an emitted trailer, which is not searched for a trailer
+ * of its own: what is appended to an image may be an image again, and a
+ * crafted file can nest that as deep as it likes.
+ */
+ private static final class Nested {
+ }
+
+ private static final Nested NESTED = new Nested();
+
+ private MotionPhoto() {
+ }
+
+ /**
+ * Emits the trailer as an embedded document, or nothing when the image
+ * declares none, when the declared length does not fit the file, or when
+ * the bytes there are not recognized. The last two are what sharing a
+ * motion photo out of a gallery leaves behind, a common enough thing that
+ * it is not worth an exception on a file that is otherwise fine; the XMP
+ * that promised the video is in the metadata for a client to see.
+ */
+ static void extract(TikaInputStream tis, Metadata metadata, XHTMLContentHandler xhtml,
+ ParseContext context) throws IOException, SAXException {
+ if (context.get(Nested.class) != null) {
+ return;
+ }
+ Declaration declared = declaration(metadata);
+ if (declared == null) {
+ return;
+ }
+ //a length the file cannot hold is settled from what the stream already
+ //knows, before an image gets spilled to disk on the strength of it
+ if (tis.hasLength() && declared.length >= tis.getLength()) {
+ return;
+ }
+ Trailer trailer = locate(tis, declared, context);
+ if (trailer == null) {
+ return;
+ }
+ Metadata trailerMetadata = Metadata.newInstance(context);
+ //the name has to be set before the parse, and the declaration names the
+ //format the file was written with, which detection cannot always tell
+ //apart: an MP4 with the isom brand types as quicktime (TIKA-3646), and
+ //the MicroVideo format declares nothing at all. Where the two disagree
+ //about the kind of file it is, the bytes win.
+ boolean fromDetection = declared.mime != null
+ && !declared.mime.startsWith(trailer.type.getType() + "/");
+ String extension = declared.mime == null ? "" : EmbeddedDocumentUtil
+ .getExtensionForMediaType(
+ fromDetection ? trailer.type.toString() : declared.mime);
+ trailerMetadata.set(TikaCoreProperties.RESOURCE_NAME_KEY, NAME + extension);
+ if (fromDetection && !extension.isEmpty()) {
+ trailerMetadata.set(TikaCoreProperties.RESOURCE_NAME_EXTENSION_INFERRED, true);
+ }
+ trailerMetadata.set(HttpHeaders.CONTENT_TYPE, trailer.type.toString());
+ trailerMetadata.set(TikaCoreProperties.EMBEDDED_RESOURCE_TYPE,
+ TikaCoreProperties.EmbeddedResourceType.ATTACHMENT.name());
+ EmbeddedDocumentExtractor extractor =
+ EmbeddedDocumentUtil.getEmbeddedDocumentExtractor(context);
+ if (!extractor.shouldParseEmbedded(trailerMetadata, context)) {
+ return;
+ }
+ InputStream bytes = open(trailer);
+ if (bytes == null) {
+ return;
+ }
+ context.set(Nested.class, NESTED);
+ try (TikaInputStream embedded = TikaInputStream.get(bytes)) {
+ extractor.parseEmbedded(embedded, new EmbeddedContentHandler(xhtml), trailerMetadata,
+ context, true);
+ } finally {
+ context.set(Nested.class, null);
+ }
+ }
+
+ /**
+ * What the declaration points at: where the bytes start and what they turn
+ * out to be, or null when they are not there, are not recognized, or
+ * cannot be read. An image that parsed is not failed over a trailer that
+ * is out of reach.
+ */
+ private static Trailer locate(TikaInputStream tis, Declaration declared,
+ ParseContext context) {
+ try {
+ Path file = tis.getPath();
+ long start = Files.size(file) - declared.length;
+ if (start <= 0) {
+ return null;
+ }
+ MediaType type = detect(file, start, context);
+ if (type == null || MediaType.OCTET_STREAM.equals(type)) {
+ return null;
+ }
Review Comment:
`getPath()` does not return null. Every `TikaInputSource` either returns a path or throws: FileSource hands back its own path, ByteArraySource and CachingSource spill to a temp file first, and CachingSource throws when the stream has already been read past position 0. That IOException is the case that actually happens here, and it is caught in this same method, which returns null and leaves the image alone.
##########
tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-xmp-commons/src/main/java/org/apache/tika/parser/xmp/XmpSaxFlattener.java:
##########
@@ -77,6 +79,26 @@ static boolean isContainer(String u, String l) {
return RDF.equals(u) && (l.equals("Bag") || l.equals("Seq") || l.equals("Alt"));
}
+ /**
+ * The prefix a namespace is normally written with, for the few whose
+ * raw keys are read elsewhere in Tika. An XMP prefix is the writer's
+ * own choice (ISO 16684-1) and rewriting a packet is enough to change
+ * it, so a key built from the document's prefix moves with it: a
+ * motion photo that has been through exiftool lists its parts under
+ * GContainer:Directory rather than Container:Directory.
+ */
+ static final Map<String, String> CANONICAL_PREFIX =
+ Map.of(Google.CONTAINER_NS, "Container", Google.ITEM_NS, "Item");
+
+ /**
+ * The name a path segment is keyed under: the document's own qName,
+ * except for the namespaces above.
+ */
+ static String canonical(String uri, String qName, String localName) {
+ String prefix = CANONICAL_PREFIX.get(uri);
+ return prefix == null ? qName : prefix + ":" + localName;
Review Comment:
This branch is only reached when the URI is one of the two Google container namespaces, and a non-empty URI means the reader is namespace aware, which is exactly when localName is set (XMLReaderUtils configures the readers that way). With namespace processing off the URI is empty, the lookup misses and the qName is used already.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tika.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] TIKA-4869: Emit the video of a motion photo as an embedded document [tika]
Posted by "dschmidt (via GitHub)" <gi...@apache.org>.
dschmidt commented on PR #3115:
URL: https://github.com/apache/tika/pull/3115#issuecomment-5510771958
Thank you, and all of it is in, pushed just now.
On 1., I agree, and I did not add the video requirement. The recursion is closed at its source instead: the branch below an emitted trailer is not searched for a trailer of its own. A PDF behind an image is still emitted, an image behind an image too, but it cannot chain. The new test builds the nesting your bot measured and asserts two documents where it got three.
Worth its own ticket, maybe: what made this reachable at all is that `ParseRecord.maxEmbeddedDepth` defaults to -1, so `/tika`, tika-app and the facade have no depth guard, only the `RecursiveParserWrapper` path does. Our chain is closed now, but any parser that emits something derived from its own input has the same shape.
On 2., the reason for preferring the declaration is that detection cannot tell these apart: an MP4 with the `isom` brand types as `video/quicktime` (that is TIKA-3646), so the extension from detection would be `.mov` on a file whose container says `video/mp4`, and MicroVideo declares no type at all, where detection would guess one. So the declaration still wins, but only within the same top level type. Where the two disagree about the kind of file it is, as with your text trailer, the bytes win and `RESOURCE_NAME_EXTENSION_INFERRED` is set, since that is the case the property describes.
On 3., `Files.size`, the detection and the region stream are inside the guard now: locating the trailer is one step that yields nothing rather than throwing, and so is opening it.
On the hygiene list: the declared `Item:Mime` stays a string and is only looked up for an extension through `EmbeddedDocumentUtil`, so neither `MediaType.parse` nor `MimeTypes.forName` sees it. `MediaType.parse` on main still caches into `SIMPLE_TYPES`, capped at 10000, and `MimeTypes.forName` still registers what it does not know, which is TIKA-4826. CHANGES.txt now says "not recognized", the embedded parse writes its own div like EMFParser and RawTiffParser, and the tests read the declared length from the fixture.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tika.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] TIKA-4869: Emit the video of a motion photo as an embedded document [tika]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #3115:
URL: https://github.com/apache/tika/pull/3115#discussion_r3915894889
##########
tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/main/java/org/apache/tika/parser/image/MotionPhoto.java:
##########
@@ -0,0 +1,299 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.parser.image;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.nio.file.Files;
+import java.nio.file.Path;
+
+import org.xml.sax.SAXException;
+
+import org.apache.tika.extractor.EmbeddedDocumentExtractor;
+import org.apache.tika.extractor.EmbeddedDocumentUtil;
+import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.Google;
+import org.apache.tika.metadata.HttpHeaders;
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.metadata.TikaCoreProperties;
+import org.apache.tika.mime.MediaType;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.sax.EmbeddedContentHandler;
+import org.apache.tika.sax.XHTMLContentHandler;
+
+/**
+ * The video of a motion photo, appended after the image and described by the
+ * XMP the image parsers already extract (TIKA-4869):
+ * <ul>
+ * <li>a Motion Photo lists its parts in {@code Container:Directory}: the
+ * primary image first, the video last with nothing after it, each with
+ * an {@code Item:Length};</li>
+ * <li>the older MicroVideo gives {@code Camera:MicroVideoOffset}, the
+ * number of bytes from the end of the file to the start of the video.</li>
+ * </ul>
+ * Either way the video ends at the end of the file, so its start follows from
+ * its length. What is found there is typed by content: the declared
+ * {@code Item:Mime} is not used as a detection hint, because a hint would
+ * make a wrong length pass as a video, and nothing is emitted when detection
+ * recognizes nothing.
+ * <p>
+ * The same holds for HEIC motion photos, whose video sits in a trailing
+ * {@code mpvd} box; its 8 byte header is the primary item's padding, so the
+ * video still ends at the end of the file.
+ */
+final class MotionPhoto {
+
+ /**
+ * The name the video is emitted under, with the extension of whatever it
+ * turns out to be.
+ */
+ private static final String NAME = "motion-photo";
+
+ private static final String ITEM = "]/Container:Item/";
+ private static final String DIRECTORY = "xmp-raw:Container:Directory[";
+
+ /**
+ * The {@code Item:Semantic} of the video.
+ */
+ private static final String MOTION_PHOTO = "MotionPhoto";
+
+ /**
+ * A directory holds a handful of items; this only bounds the walk.
+ */
+ private static final int MAX_ITEMS = 64;
+
+ /**
+ * Enough of the video for the detectors to recognize it.
+ */
+ private static final int DETECTION_PREFIX = 8 * 1024;
+
+ /**
+ * The branch below an emitted trailer, which is not searched for a trailer
+ * of its own: what is appended to an image may be an image again, and a
+ * crafted file can nest that as deep as it likes.
+ */
+ private static final class Nested {
+ }
+
+ private static final Nested NESTED = new Nested();
+
+ private MotionPhoto() {
+ }
+
+ /**
+ * Emits the trailer as an embedded document, or nothing when the image
+ * declares none, when the declared length does not fit the file, or when
+ * the bytes there are not recognized. The last two are what sharing a
+ * motion photo out of a gallery leaves behind, a common enough thing that
+ * it is not worth an exception on a file that is otherwise fine; the XMP
+ * that promised the video is in the metadata for a client to see.
+ */
+ static void extract(TikaInputStream tis, Metadata metadata, XHTMLContentHandler xhtml,
+ ParseContext context) throws IOException, SAXException {
+ if (context.get(Nested.class) != null) {
+ return;
+ }
+ Declaration declared = declaration(metadata);
+ if (declared == null) {
+ return;
+ }
+ //a length the file cannot hold is settled from what the stream already
+ //knows, before an image gets spilled to disk on the strength of it
+ if (tis.hasLength() && declared.length >= tis.getLength()) {
+ return;
+ }
+ Trailer trailer = locate(tis, declared, context);
+ if (trailer == null) {
+ return;
+ }
+ Metadata trailerMetadata = Metadata.newInstance(context);
+ //the name has to be set before the parse, and the declaration names the
+ //format the file was written with, which detection cannot always tell
+ //apart: an MP4 with the isom brand types as quicktime (TIKA-3646), and
+ //the MicroVideo format declares nothing at all. Where the two disagree
+ //about the kind of file it is, the bytes win.
+ boolean fromDetection = declared.mime != null
+ && !declared.mime.startsWith(trailer.type.getType() + "/");
+ String extension = declared.mime == null ? "" : EmbeddedDocumentUtil
+ .getExtensionForMediaType(
+ fromDetection ? trailer.type.toString() : declared.mime);
+ trailerMetadata.set(TikaCoreProperties.RESOURCE_NAME_KEY, NAME + extension);
+ if (fromDetection && !extension.isEmpty()) {
+ trailerMetadata.set(TikaCoreProperties.RESOURCE_NAME_EXTENSION_INFERRED, true);
+ }
+ trailerMetadata.set(HttpHeaders.CONTENT_TYPE, trailer.type.toString());
+ trailerMetadata.set(TikaCoreProperties.EMBEDDED_RESOURCE_TYPE,
+ TikaCoreProperties.EmbeddedResourceType.ATTACHMENT.name());
+ EmbeddedDocumentExtractor extractor =
+ EmbeddedDocumentUtil.getEmbeddedDocumentExtractor(context);
+ if (!extractor.shouldParseEmbedded(trailerMetadata, context)) {
+ return;
+ }
+ InputStream bytes = open(trailer);
+ if (bytes == null) {
+ return;
+ }
+ context.set(Nested.class, NESTED);
+ try (TikaInputStream embedded = TikaInputStream.get(bytes)) {
+ extractor.parseEmbedded(embedded, new EmbeddedContentHandler(xhtml), trailerMetadata,
+ context, true);
+ } finally {
+ context.set(Nested.class, null);
+ }
+ }
+
+ /**
+ * What the declaration points at: where the bytes start and what they turn
+ * out to be, or null when they are not there, are not recognized, or
+ * cannot be read. An image that parsed is not failed over a trailer that
+ * is out of reach.
+ */
+ private static Trailer locate(TikaInputStream tis, Declaration declared,
+ ParseContext context) {
+ try {
+ Path file = tis.getPath();
+ long start = Files.size(file) - declared.length;
+ if (start <= 0) {
+ return null;
+ }
+ MediaType type = detect(file, start, context);
+ if (type == null || MediaType.OCTET_STREAM.equals(type)) {
+ return null;
+ }
Review Comment:
`tis.getPath()` can be `null` for non-file-backed streams (or depending on how the `TikaInputStream` was constructed). If it’s null here, `Files.size(file)` will throw a `NullPointerException` and fail the whole image parse. Handle this explicitly (e.g., return `null` when `file == null`, or materialize/spool to a temp file before calling `Files.size(...)`).
##########
tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-xmp-commons/src/main/java/org/apache/tika/parser/xmp/XmpSaxFlattener.java:
##########
@@ -77,6 +79,26 @@ static boolean isContainer(String u, String l) {
return RDF.equals(u) && (l.equals("Bag") || l.equals("Seq") || l.equals("Alt"));
}
+ /**
+ * The prefix a namespace is normally written with, for the few whose
+ * raw keys are read elsewhere in Tika. An XMP prefix is the writer's
+ * own choice (ISO 16684-1) and rewriting a packet is enough to change
+ * it, so a key built from the document's prefix moves with it: a
+ * motion photo that has been through exiftool lists its parts under
+ * GContainer:Directory rather than Container:Directory.
+ */
+ static final Map<String, String> CANONICAL_PREFIX =
+ Map.of(Google.CONTAINER_NS, "Container", Google.ITEM_NS, "Item");
+
+ /**
+ * The name a path segment is keyed under: the document's own qName,
+ * except for the namespaces above.
+ */
+ static String canonical(String uri, String qName, String localName) {
+ String prefix = CANONICAL_PREFIX.get(uri);
+ return prefix == null ? qName : prefix + ":" + localName;
Review Comment:
If `localName` is empty (which can happen depending on SAX parser configuration/behavior), this will produce malformed path segments like `Container:` / `Item:` and break lookups that rely on these keys. Safer behavior is to fall back to `qName` when `localName` is null/empty, even when the namespace URI is one of the canonicalized ones.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@tika.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org