You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tika.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2026/10/19 21:50:00 UTC
[jira] [Commented] (TIKA-4980) Extract previews from Olympus ORF, Panasonic RW2 and Epson ERF
[ https://issues.apache.org/jira/browse/TIKA-4980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18128912#comment-18128912 ]
ASF GitHub Bot commented on TIKA-4980:
--------------------------------------
dschmidt opened a new pull request, #3370:
URL: https://github.com/apache/tika/pull/3370
Fixes https://issues.apache.org/jira/browse/TIKA-4980
Three TIFF-based raw formats produced no preview, each for its own reason.
**Epson ERF** is now detected as a raw by its compression code (32769) and its Make ("SEIKO EPSON"), and `image/x-raw-epson` gains `sub-class-of image/tiff`. The registry line is not cosmetic: without it a detector returning that type would leave the file with no parser and nothing to fall back on, where today TiffParser at least reads its metadata. It also makes the type reachable by a declared Content-Type, which MimeTypes only accepts as a refinement of what it detected.
**Olympus ORF and Panasonic RW2** are classic TIFF layouts behind their own magic number (0x4F52/0x5352 and 0x0055), which `locateJpegPreviews()` rejected outright. Their previews are not in the standard tags either: RW2 keeps its JPEG in the private JpgFromRaw tag 0x002E while its strip tags are placeholders, ORF keeps the camera thumbnail and the preview in the MakerNote, addressed relative to the MakerNote's first byte (0x0100, and 0x2020 CameraSettings -> 0x0101/0x0102). Only the `OLYMPUS\0II` header is read; the older `OLYMP\0` one addresses differently and is left alone.
Verified against CC0 samples from raw.pixls.us:
```
P3180165.ORF (E-M10) image/x-raw-olympus THUMBNAIL 3200x2400 INLINE 160x120
_7061961_copy.ORF (E-5) image/x-raw-olympus THUMBNAIL 3200x2400 INLINE 160x120
RAW_PANASONIC_DMC-GX-7_11 image/x-raw-panasonic THUMBNAIL 1920x1920
Lumix_DMC-LX100_1.RW2 image/x-raw-panasonic THUMBNAIL 1920x1440
```
The largest preview by JPEG length becomes the THUMBNAIL and the camera thumbnail an INLINE image, as TIKA-4851 set it up. Make, model and dimensions are extracted for all of them and none records a parse exception. An Epson R-D1 sample carries no embedded JPEG at all, so it stays without a thumbnail; its detection and typing are fixed regardless.
Tests use two hand-built files, `testORF.orf` (890 bytes) and `testRW2.rw2` (421 bytes), in the style of the existing raw test documents, plus the detection cases. `apache-rat:check` is clean on both touched modules.
Not covered: the older Olympus `OLYMP\0` MakerNote header, Fuji RAF (not a TIFF container), and the vendor-level mime entries that carry no `sub-class-of` because they span several containers (`image/x-raw-canon` covers CRW, CR2 and CR3).
> Extract previews from Olympus ORF, Panasonic RW2 and Epson ERF
> --------------------------------------------------------------
>
> Key: TIKA-4980
> URL: https://issues.apache.org/jira/browse/TIKA-4980
> Project: Tika
> Issue Type: Improvement
> Components: parser
> Reporter: Dominik Schmidt
> Priority: Major
>
> RawTiffParser (TIKA-4851, TIKA-4861) extracts the camera-generated JPEG previews of
> TIFF-based raw formats: Nikon NEF/NRW, Sony ARW/SRF/SR2, Pentax PEF/PTX, Samsung SRW,
> Adobe DNG and Canon CR2. Three further TIFF-based formats yield nothing, each for a
> different reason.
> h3. 1. Olympus ORF
> Detected correctly as image/x-raw-olympus (magic entry in tika-mimetypes.xml), but no
> parser declares the type, and RawTiffParser would refuse the file anyway: ORF uses its
> own magic number (0x4F52 for "IIRO"/"MMOR", 0x5352 for "IIRS") where a TIFF has 42, so
> locateJpegPreviews() throws "bad TIFF magic number".
> On top of that the previews are not in the IFD chain. IFD0 holds the sensor image; the
> previews hang in the MakerNote, whose offsets are relative to its own first byte:
> {code}
> EXIF IFD (0x8769) -> MakerNote (0x927C), header "OLYMPUS\0II", its IFD at +12
> 0x0100 camera thumbnail (UNDEFINED, count = length)
> 0x2020 CameraSettings sub-IFD
> 0x0101/0x0102 PreviewImageStart / PreviewImageLength
> {code}
> Sample (raw.pixls.us, CC0, Olympus E-M10, P3180165.ORF): a 3200x2400 preview of 918873
> bytes and a 160x120 thumbnail.
> h3. 2. Panasonic RW2
> Detected correctly as image/x-raw-panasonic, same two blockers: no parser declares the
> type, and the magic number is 0x0055 ("IIU\0"). The preview sits in the private tag
> 0x002E (JpgFromRaw) of IFD0, while the standard strip tags are placeholders
> (StripOffsets = 0xFFFFFFFF, StripByteCounts = 0).
> Sample (raw.pixls.us, CC0, Panasonic DMC-GX7): a 1920x1920 preview of 926720 bytes.
> h3. 3. Epson ERF
> Not recognized as a raw at all. RawTiffDetector has no Epson entry, so the file is
> detected as image/tiff and handled by TiffParser. Declaring the type does not help
> either, because image/x-raw-epson has no sub-class-of in tika-mimetypes.xml: MimeTypes
> only accepts a declared type that is a descendant of what it detected, so the hint is
> dropped. Measured against an Epson R-D1 file:
> {code}
> hint image/x-raw-epson -> detected image/tiff (hint discarded)
> hint image/x-raw-adobe -> detected image/x-raw-adobe (hint accepted: it is a descendant)
> {code}
> The raw IFD of that file has Compression 32769 (Epson's own code, as ExifTool lists it)
> and PhotometricInterpretation 32803 (CFA), and IFD0 carries Make "SEIKO EPSON CORP.",
> which is what RawTiffDetector already keys on for the other vendors. Note this
> particular sample carries no embedded JPEG at all, so it will stay without a thumbnail;
> the detection and the typing are wrong regardless, and ERFs that do carry a preview
> cannot be reached today.
> h3. Proposed fix
> * RawTiffDetector: Epson by compression code 32769 and by Make "SEIKO EPSON"/"EPSON".
> * tika-mimetypes.xml: sub-class-of image/tiff for image/x-raw-epson, so the type is a
> TIFF for the registry, a declared type can refine to it, and the parser lookup has
> something to fall back to.
> * RawTiffParser: accept the ORF and RW2 magic numbers as classic TIFF layouts; read the
> Panasonic JpgFromRaw tag; read the Olympus MakerNote (only the "OLYMPUS\0II" header,
> the older "OLYMP\0" one addresses differently) for the thumbnail and the preview; add
> image/x-raw-olympus, image/x-raw-panasonic and image/x-raw-epson to the supported
> types.
> * The existing rule picks the resource type: the largest preview by JPEG length becomes
> the THUMBNAIL, the smaller ones INLINE images.
> Verified against real files (raw.pixls.us samples, CC0):
> {code}
> P3180165.ORF (E-M10) image/x-raw-olympus THUMBNAIL 3200x2400 INLINE 160x120
> _7061961_copy.ORF (E-5) image/x-raw-olympus THUMBNAIL 3200x2400 INLINE 160x120
> RAW_PANASONIC_DMC-GX-7_11 image/x-raw-panasonic THUMBNAIL 1920x1920
> Lumix_DMC-LX100_1.RW2 image/x-raw-panasonic THUMBNAIL 1920x1440
> pixls-cc0-epson-rd1.erf image/x-raw-epson (no preview in the file)
> {code}
> Make, model and dimensions are extracted for all of them, and none records a parse
> exception. The tests use two hand-built files of 890 and 421 bytes, in the style of the
> existing raw test documents.
> Not covered: the older Olympus "OLYMP\0" MakerNote header, Fuji RAF (not a TIFF
> container), and the vendor-level mime entries that exist without a sub-class-of because
> they span several containers (image/x-raw-canon covers CRW, CR2 and CR3).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)