You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tika.apache.org by ta...@apache.org on 2026/09/02 20:04:30 UTC
(tika) branch TIKA-4872 created (now 6a68b07cc1)
This is an automated email from the ASF dual-hosted git repository.
tballison pushed a change to branch TIKA-4872
in repository https://gitbox.apache.org/repos/asf/tika.git
at 6a68b07cc1 improve documentation
This branch includes the following new commits:
new 6cc2ca70ba TIKA-4871: ocr/inference bug fixes
new 8a19875df8 TIKA-4872: content-enrichers
new 8c20152d6a TIKA-4872: review fixes
new 6a68b07cc1 improve documentation
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(tika) 01/04: TIKA-4871: ocr/inference bug fixes
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch TIKA-4872
in repository https://gitbox.apache.org/repos/asf/tika.git
commit 6cc2ca70ba236ac180e3c51da74e25972d1665b9
Author: tallison <ta...@apache.org>
AuthorDate: Wed Sep 2 10:39:08 2026 -0400
TIKA-4871: ocr/inference bug fixes
---
CHANGES.txt | 11 ++++
.../ROOT/pages/configuration/metadata-filters.adoc | 5 ++
.../pages/configuration/parsers/vlm-parsers.adoc | 12 ++--
.../tika/inference/AbstractEmbeddingFilter.java | 68 +++++++++++++++++-----
.../apache/tika/inference/JinaEmbeddingFilter.java | 5 ++
.../tika/inference/OpenAIEmbeddingFilter.java | 12 ++++
.../tika/inference/OpenAIEmbeddingFilterTest.java | 57 ++++++++++++++++++
.../apache/tika/parser/vlm/OpenAIVLMParser.java | 3 +-
.../tika/parser/vlm/OpenAIVLMParserTest.java | 17 ++++++
.../apache/tika/parser/pdf/AbstractPDF2XHTML.java | 31 +++++++++-
.../org/apache/tika/parser/pdf/PDFParserTest.java | 52 +++++++++++++++++
11 files changed, 248 insertions(+), 25 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 80e898aa72..b632080e80 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,16 @@
Release 4.1.0 - unreleased
+ * Inference/OCR hardening (TIKA-4871): OpenAIVLMParser no longer
+ auto-registers via SPI, matching its Claude/Gemini siblings; select
+ it by name ("openai-vlm-parser") in config. Per-request parse-context
+ config for the embedding filters now works and is validated:
+ {"openai-embedding-filter": {"skipEmbedding": true}} (likewise
+ "jina-embedding-filter") merges over the server config, and
+ baseUrl/apiKey/model may not be changed at runtime. The embedding
+ filters release their HTTP client resources on close(). Inline PDF
+ page OCR now accumulates tk:chunks from every OCR'd page onto the
+ parent document instead of keeping only the first page's.
+
* Raster previews for the vector thumbnails of Office documents: the new
poi-metafile-renderer draws EMF and WMF images through POI (a PNG of
a configurable width; Word's bitmap-in-WMF thumbnails from the bitmap
diff --git a/docs/modules/ROOT/pages/configuration/metadata-filters.adoc b/docs/modules/ROOT/pages/configuration/metadata-filters.adoc
index c7b1eeae30..262fee13a9 100644
--- a/docs/modules/ROOT/pages/configuration/metadata-filters.adoc
+++ b/docs/modules/ROOT/pages/configuration/metadata-filters.adoc
@@ -233,6 +233,11 @@ around.
`parse-context` can carry a per-request `metadata-filters` list, which *replaces* the configured
one for that request rather than adding to it.
+Since 4.1.0, the embedding filters also accept per-request config under their own name --
+e.g. `{"parse-context": {"openai-embedding-filter": {"skipEmbedding": true}}}` (likewise
+`jina-embedding-filter`) -- which is merged over the server-side config for that request.
+Endpoint fields (`baseUrl`, `apiKey`, `model`) cannot be changed per request.
+
NOTE: In `CONTENT_ONLY` xref:pipes/parse-modes.adoc[parse mode], Tika applies an
`include-field-metadata-filter` for `tk:content` and `tk:exception:container-exception` when you have
configured no filter of your own. Only a filter that reaches the `parse-context` replaces it —
diff --git a/docs/modules/ROOT/pages/configuration/parsers/vlm-parsers.adoc b/docs/modules/ROOT/pages/configuration/parsers/vlm-parsers.adoc
index 895c162529..8c68b7e7d4 100644
--- a/docs/modules/ROOT/pages/configuration/parsers/vlm-parsers.adoc
+++ b/docs/modules/ROOT/pages/configuration/parsers/vlm-parsers.adoc
@@ -22,27 +22,25 @@ to remote Vision-Language Model (VLM) endpoints. These parsers send images
(or PDFs) to an external API and convert the model's markdown response into
structured XHTML.
-Three implementations are provided out of the box. Only `openai-vlm-parser` is auto-loaded via SPI;
-the other two must be named explicitly in your configuration.
+Three implementations are provided out of the box. None is auto-loaded: each must be
+named explicitly in your configuration. (Changed in 4.1.0: `openai-vlm-parser` previously
+auto-registered via SPI.)
-[cols="1,2,1,1"]
+[cols="1,2,1"]
|===
-|Parser |Endpoint |Config key |Auto-loaded
+|Parser |Endpoint |Config key
|`OpenAIVLMParser`
|Any OpenAI-compatible chat completions endpoint (vLLM, Ollama, local FastAPI, OpenAI)
|`openai-vlm-parser`
-|Yes
|`ClaudeVLMParser`
|Anthropic Messages API
|`claude-vlm-parser`
-|No
|`GeminiVLMParser`
|Google Gemini `generateContent` API
|`gemini-vlm-parser`
-|No
|===
All three handle the standard OCR image types (`image/ocr-png`, `image/ocr-jpeg`, ...).
diff --git a/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/AbstractEmbeddingFilter.java b/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/AbstractEmbeddingFilter.java
index 52cf9187a1..b134dd0da8 100644
--- a/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/AbstractEmbeddingFilter.java
+++ b/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/AbstractEmbeddingFilter.java
@@ -23,6 +23,8 @@ import java.util.Locale;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
+import org.apache.tika.config.ParseContextConfig;
+import org.apache.tika.config.SelfConfiguring;
import org.apache.tika.exception.TikaConfigException;
import org.apache.tika.exception.TikaException;
import org.apache.tika.metadata.Metadata;
@@ -54,7 +56,7 @@ import org.apache.tika.parser.ParseContext;
* fully constructed. Setters must not be called concurrently with
* {@link #filter}.
*/
-public abstract class AbstractEmbeddingFilter extends MetadataFilter {
+public abstract class AbstractEmbeddingFilter extends MetadataFilter implements SelfConfiguring {
private static final long serialVersionUID = 1L;
@@ -88,24 +90,60 @@ public abstract class AbstractEmbeddingFilter extends MetadataFilter {
protected abstract void embed(List<Chunk> chunks, InferenceConfig config, ParseContext parseContext)
throws IOException, TikaException;
+ /**
+ * The {@code @TikaComponent} name this filter's per-request JSON config is keyed by
+ * in parse-context (e.g. {@code {"openai-embedding-filter": {...}}}).
+ */
+ protected abstract String getComponentName();
+
@Override
protected void doFilter(List<Metadata> metadataList, ParseContext parseContext) throws TikaException {
- InferenceConfig requestConfig = parseContext.get(InferenceConfig.class);
- if (requestConfig != null && requestConfig.isSkipEmbedding()) {
+ InferenceConfig config = resolveConfig(parseContext);
+ if (config.isSkipEmbedding()) {
return;
}
for (Metadata metadata : metadataList) {
- processOne(metadata, parseContext);
+ processOne(metadata, config, parseContext);
+ }
+ }
+
+ /**
+ * Per-request JSON config, validated through {@link InferenceConfig.RuntimeConfig}
+ * (which rejects baseUrl/apiKey/model changes) and merged over the init-time defaults.
+ * With no JSON, a class-keyed programmatic {@link InferenceConfig} is honored for
+ * skipEmbedding only, preserving the pre-4.1 contract.
+ */
+ private InferenceConfig resolveConfig(ParseContext parseContext) throws TikaException {
+ try {
+ if (ParseContextConfig.hasConfig(parseContext, getComponentName())) {
+ InferenceConfig.RuntimeConfig runtimeConfig = ParseContextConfig.getConfig(
+ parseContext, getComponentName(),
+ InferenceConfig.RuntimeConfig.class, new InferenceConfig.RuntimeConfig());
+ if (runtimeConfig.isSkipEmbedding()) {
+ return runtimeConfig;
+ }
+ return ParseContextConfig.getConfig(parseContext, getComponentName(),
+ InferenceConfig.class, defaultConfig);
+ }
+ } catch (TikaConfigException | IOException e) {
+ throw new TikaException("Failed to resolve per-request config for '"
+ + getComponentName() + "'", e);
+ }
+ InferenceConfig programmatic = parseContext.get(InferenceConfig.class);
+ if (programmatic != null && programmatic.isSkipEmbedding()) {
+ return programmatic;
}
+ return defaultConfig;
}
- private void processOne(Metadata metadata, ParseContext parseContext) throws TikaException {
- String content = metadata.get(defaultConfig.getContentField());
+ private void processOne(Metadata metadata, InferenceConfig config, ParseContext parseContext)
+ throws TikaException {
+ String content = metadata.get(config.getContentField());
if (content == null) {
LOG.debug("No content found at field '{}'; skipping embedding. "
+ "If using this filter standalone, populate metadata using "
+ "TikaCoreProperties.TIKA_CONTENT as the key.",
- defaultConfig.getContentField());
+ config.getContentField());
return;
}
if (content.isBlank()) {
@@ -124,15 +162,15 @@ public abstract class AbstractEmbeddingFilter extends MetadataFilter {
}
MarkdownChunker chunker = new MarkdownChunker(
- defaultConfig.getMaxChunkChars(),
- defaultConfig.getOverlapChars());
+ config.getMaxChunkChars(),
+ config.getOverlapChars());
List<Chunk> chunks = chunker.chunk(content);
if (chunks.isEmpty()) {
return;
}
- int maxChunks = defaultConfig.getMaxChunks();
+ int maxChunks = config.getMaxChunks();
if (maxChunks > 0 && chunks.size() > maxChunks) {
LOG.warn("Document produced {} chunks, truncating to maxChunks={}",
chunks.size(), maxChunks);
@@ -140,20 +178,20 @@ public abstract class AbstractEmbeddingFilter extends MetadataFilter {
}
try {
- int batchSize = defaultConfig.getMaxBatchSize();
+ int batchSize = config.getMaxBatchSize();
for (int i = 0; i < chunks.size(); i += batchSize) {
List<Chunk> batch = chunks.subList(
i, Math.min(i + batchSize, chunks.size()));
- embed(batch, defaultConfig, parseContext);
+ embed(batch, config, parseContext);
}
- ChunkSerializer.mergeInto(metadata, chunks, defaultConfig.getOutputField());
+ ChunkSerializer.mergeInto(metadata, chunks, config.getOutputField());
} catch (IOException e) {
throw new TikaException(
"Embedding inference failed: " + e.getMessage(), e);
}
- if (defaultConfig.isClearContentAfterChunking()) {
- metadata.remove(defaultConfig.getContentField());
+ if (config.isClearContentAfterChunking()) {
+ metadata.remove(config.getContentField());
}
}
diff --git a/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/JinaEmbeddingFilter.java b/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/JinaEmbeddingFilter.java
index 2cf3671d4f..2985c77498 100644
--- a/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/JinaEmbeddingFilter.java
+++ b/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/JinaEmbeddingFilter.java
@@ -61,6 +61,11 @@ public class JinaEmbeddingFilter extends OpenAIEmbeddingFilter {
super(config);
}
+ @Override
+ protected String getComponentName() {
+ return "jina-embedding-filter";
+ }
+
@Override
String buildRequest(List<Chunk> chunks, InferenceConfig config) {
ObjectNode root = MAPPER.createObjectNode();
diff --git a/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/OpenAIEmbeddingFilter.java b/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/OpenAIEmbeddingFilter.java
index d0b7e00902..54afc1712b 100644
--- a/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/OpenAIEmbeddingFilter.java
+++ b/tika-parsers/tika-parsers-ml/tika-inference/src/main/java/org/apache/tika/inference/OpenAIEmbeddingFilter.java
@@ -78,6 +78,18 @@ public class OpenAIEmbeddingFilter extends AbstractEmbeddingFilter {
this.httpClient = TikaHttpClient.build(30);
}
+ @Override
+ protected String getComponentName() {
+ return "openai-embedding-filter";
+ }
+
+ @Override
+ public void close() throws IOException {
+ if (httpClient != null) {
+ httpClient.close();
+ }
+ }
+
@Override
protected void embed(List<Chunk> chunks, InferenceConfig config, ParseContext parseContext)
throws IOException, TikaException {
diff --git a/tika-parsers/tika-parsers-ml/tika-inference/src/test/java/org/apache/tika/inference/OpenAIEmbeddingFilterTest.java b/tika-parsers/tika-parsers-ml/tika-inference/src/test/java/org/apache/tika/inference/OpenAIEmbeddingFilterTest.java
index a9087e6b64..fcf2c2d0d1 100644
--- a/tika-parsers/tika-parsers-ml/tika-inference/src/test/java/org/apache/tika/inference/OpenAIEmbeddingFilterTest.java
+++ b/tika-parsers/tika-parsers-ml/tika-inference/src/test/java/org/apache/tika/inference/OpenAIEmbeddingFilterTest.java
@@ -35,6 +35,7 @@ import org.apache.tika.exception.TikaException;
import org.apache.tika.http.TikaTestHttpServer;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.metadata.TikaCoreProperties;
+import org.apache.tika.parser.ParseContext;
public class OpenAIEmbeddingFilterTest {
@@ -230,6 +231,62 @@ public class OpenAIEmbeddingFilterTest {
assertNotNull(merged.get(1).getVector());
}
+ @Test
+ void testPerRequestSkipEmbedding() throws Exception {
+ Metadata metadata = new Metadata();
+ metadata.set(TikaCoreProperties.TIKA_CONTENT, "# Section A\n\nSome text.");
+ List<Metadata> list = new ArrayList<>();
+ list.add(metadata);
+
+ ParseContext context = new ParseContext();
+ context.setJsonConfig("openai-embedding-filter", "{\"skipEmbedding\": true}");
+ filter.filter(list, context);
+
+ assertNull(metadata.get(TikaCoreProperties.TIKA_CHUNKS));
+ assertEquals(0, server.getRequestCount());
+ }
+
+ @Test
+ void testPerRequestBaseUrlRejected() {
+ Metadata metadata = new Metadata();
+ metadata.set(TikaCoreProperties.TIKA_CONTENT, "# Section A\n\nSome text.");
+ List<Metadata> list = new ArrayList<>();
+ list.add(metadata);
+
+ ParseContext context = new ParseContext();
+ context.setJsonConfig("openai-embedding-filter",
+ "{\"baseUrl\": \"http://attacker.example.com\"}");
+ assertThrows(TikaException.class, () -> filter.filter(list, context));
+ assertEquals(0, server.getRequestCount());
+ }
+
+ @Test
+ void testPerRequestConfigMergesOverDefaults() throws Exception {
+ server.enqueue(new TikaTestHttpServer.MockResponse(200,
+ buildEmbeddingResponse(1, 3)));
+
+ Metadata metadata = new Metadata();
+ metadata.set(TikaCoreProperties.TIKA_CONTENT, "# Section A\n\nSome text.");
+ List<Metadata> list = new ArrayList<>();
+ list.add(metadata);
+
+ // an allowed runtime override; baseUrl/model must survive from the init-time config
+ ParseContext context = new ParseContext();
+ context.setJsonConfig("openai-embedding-filter", "{\"maxChunkChars\": 5000}");
+ filter.filter(list, context);
+
+ assertNotNull(metadata.get(TikaCoreProperties.TIKA_CHUNKS));
+ assertEquals(1, server.getRequestCount());
+ TikaTestHttpServer.RecordedRequest request = server.takeRequest();
+ assertEquals("/v1/embeddings", request.path());
+ }
+
+ @Test
+ void testCloseReleasesClient() throws Exception {
+ filter.close();
+ filter.close();
+ }
+
/**
* Build a mock OpenAI embeddings response.
*/
diff --git a/tika-parsers/tika-parsers-ml/tika-vlm/src/main/java/org/apache/tika/parser/vlm/OpenAIVLMParser.java b/tika-parsers/tika-parsers-ml/tika-vlm/src/main/java/org/apache/tika/parser/vlm/OpenAIVLMParser.java
index bf5d824e7b..517fe5f0ab 100644
--- a/tika-parsers/tika-parsers-ml/tika-vlm/src/main/java/org/apache/tika/parser/vlm/OpenAIVLMParser.java
+++ b/tika-parsers/tika-parsers-ml/tika-vlm/src/main/java/org/apache/tika/parser/vlm/OpenAIVLMParser.java
@@ -57,7 +57,8 @@ import org.apache.tika.utils.StringUtils;
*
* @since Apache Tika 4.0
*/
-@TikaComponent(name = "openai-vlm-parser")
+// spi = false: VLM parsers are selected by name in config, never auto-registered (TIKA-4871)
+@TikaComponent(name = "openai-vlm-parser", spi = false)
public class OpenAIVLMParser extends AbstractVLMParser {
private static final long serialVersionUID = 1L;
diff --git a/tika-parsers/tika-parsers-ml/tika-vlm/src/test/java/org/apache/tika/parser/vlm/OpenAIVLMParserTest.java b/tika-parsers/tika-parsers-ml/tika-vlm/src/test/java/org/apache/tika/parser/vlm/OpenAIVLMParserTest.java
index 71f6d84deb..3fc7039e64 100644
--- a/tika-parsers/tika-parsers-ml/tika-vlm/src/test/java/org/apache/tika/parser/vlm/OpenAIVLMParserTest.java
+++ b/tika-parsers/tika-parsers-ml/tika-vlm/src/test/java/org/apache/tika/parser/vlm/OpenAIVLMParserTest.java
@@ -285,6 +285,23 @@ public class OpenAIVLMParserTest {
assertEquals(0, parser.getSupportedTypes(new ParseContext()).size());
}
+ /**
+ * No VLM parser may auto-register via SPI; VLM parsers must be selected by name
+ * in config (TIKA-4871).
+ */
+ @Test
+ void testNoVlmParserAutoRegisters() throws Exception {
+ java.util.Enumeration<java.net.URL> resources = getClass().getClassLoader()
+ .getResources("META-INF/services/org.apache.tika.parser.Parser");
+ while (resources.hasMoreElements()) {
+ java.net.URL url = resources.nextElement();
+ String content = new String(url.openStream().readAllBytes(),
+ java.nio.charset.StandardCharsets.UTF_8);
+ assertTrue(!content.contains("org.apache.tika.parser.vlm."),
+ "VLM parser auto-registered via SPI in " + url + ":\n" + content);
+ }
+ }
+
private String buildChatResponse(String content, int prompt, int completion) {
return String.format(java.util.Locale.ROOT,
"{\"choices\":[{\"message\":{\"content\":\"%s\"}}],"
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java
index aa57face74..fb8a725e31 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java
@@ -598,8 +598,10 @@ class AbstractPDF2XHTML extends PDFTextStripper {
// from image embedding parsers) back to the parent document so it isn't
// silently discarded when the renderMetadata goes out of scope.
String renderChunks = renderMetadata.get(TikaCoreProperties.TIKA_CHUNKS);
- if (renderChunks != null && metadata.get(TikaCoreProperties.TIKA_CHUNKS) == null) {
- metadata.set(TikaCoreProperties.TIKA_CHUNKS, renderChunks);
+ if (renderChunks != null) {
+ metadata.setTrusted(TikaCoreProperties.TIKA_CHUNKS.getName(),
+ mergeChunkArrays(metadata.get(TikaCoreProperties.TIKA_CHUNKS),
+ renderChunks));
}
}
} catch (IOException e) {
@@ -611,6 +613,31 @@ class AbstractPDF2XHTML extends PDFTextStripper {
}
}
+ /**
+ * Appends two serialized tk:chunks JSON arrays without a JSON dependency, so each
+ * OCR'd page's chunks accumulate on the parent instead of first-page-wins.
+ * Falls back to the new value if either side is not an array.
+ */
+ static String mergeChunkArrays(String existing, String added) {
+ if (existing == null || existing.isBlank()) {
+ return added;
+ }
+ String e = existing.trim();
+ String a = added.trim();
+ if (!e.startsWith("[") || !e.endsWith("]") || !a.startsWith("[") || !a.endsWith("]")) {
+ return a;
+ }
+ String eBody = e.substring(1, e.length() - 1).trim();
+ String aBody = a.substring(1, a.length() - 1).trim();
+ if (eBody.isEmpty()) {
+ return a;
+ }
+ if (aBody.isEmpty()) {
+ return e;
+ }
+ return "[" + eBody + "," + aBody + "]";
+ }
+
private RenderResult renderCurrentPage(PDPage pdPage, TemporaryResources tmpResources)
throws IOException, TikaException {
PDFRenderingState renderingState = context.get(PDFRenderingState.class);
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java
index da6d0a7282..b44ed01df7 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java
@@ -1619,6 +1619,58 @@ public class PDFParserTest extends TikaTest {
assertTrue(sawTimeoutWarning,
"the timeout must be recorded, not silently dropped, once caught and continued past");
}
+
+ /**
+ * tk:chunks written by the OCR-slot parser (e.g. an image-embedding parser) on each
+ * rendered page must all reach the parent metadata, not just the first page's.
+ */
+ @Test
+ public void testChunksFromAllOcrPagesReachParent() throws Exception {
+ PDFParserConfig config = new PDFParserConfig();
+ config.getOcr().setStrategy(OcrConfig.Strategy.OCR_ONLY);
+
+ ParseContext context = new ParseContext();
+ context.set(PDFParserConfig.class, config);
+ context.set(Parser.class, new Parser() {
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return Collections.singleton(
+ MediaType.image("ocr-" + config.getOcr().getImageFormat().getFormatName()));
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
+ ParseContext context) throws IOException, SAXException, TikaException {
+ int currentPage = context.get(OCRPageCounter.class).getCount();
+ metadata.setTrusted(TikaCoreProperties.TIKA_CHUNKS.getName(),
+ "[{\"text\":\"chunk-page-" + currentPage + "\"}]");
+ XHTMLContentHandler xhtml = new XHTMLContentHandler(handler, metadata);
+ xhtml.startDocument();
+ xhtml.endDocument();
+ }
+ });
+
+ Metadata metadata = new Metadata();
+ try (TikaInputStream tis = getResourceAsStream("/test-documents/testPDF_bookmarks.pdf")) {
+ new PDFParser().parse(tis, new ToXMLContentHandler(), metadata, context);
+ }
+
+ String chunks = metadata.get(TikaCoreProperties.TIKA_CHUNKS);
+ assertNotNull(chunks);
+ assertContains("chunk-page-1", chunks);
+ assertContains("chunk-page-2", chunks);
+ }
+
+ @Test
+ public void testMergeChunkArrays() {
+ assertEquals("[b]", AbstractPDF2XHTML.mergeChunkArrays(null, "[b]"));
+ assertEquals("[b]", AbstractPDF2XHTML.mergeChunkArrays(" ", "[b]"));
+ assertEquals("[a,b]", AbstractPDF2XHTML.mergeChunkArrays("[a]", "[b]"));
+ assertEquals("[a]", AbstractPDF2XHTML.mergeChunkArrays("[a]", "[]"));
+ assertEquals("[b]", AbstractPDF2XHTML.mergeChunkArrays("[]", "[b]"));
+ assertEquals("[b]", AbstractPDF2XHTML.mergeChunkArrays("not-an-array", "[b]"));
+ }
+
/**
* TODO -- need to test signature extraction
*/
(tika) 04/04: improve documentation
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch TIKA-4872
in repository https://gitbox.apache.org/repos/asf/tika.git
commit 6a68b07cc10d08e5f0bcf52cdb054ca8514f27d6
Author: tallison <ta...@apache.org>
AuthorDate: Wed Sep 2 16:04:17 2026 -0400
improve documentation
---
docs/modules/ROOT/pages/configuration/index.adoc | 10 ++--
.../parser/enricher/CompositeContentEnricher.java | 32 +++++--------
.../tika/parser/enricher/ContentEnrichers.java | 56 +++++++---------------
.../tika/parser/enricher/EnrichingParser.java | 4 +-
.../parser/enricher/LegacyDispatchEnricher.java | 22 ++++-----
.../tika/parser/enricher/ContentEnrichersTest.java | 8 ++--
.../apache/tika/parser/image/ImageParserTest.java | 10 ++--
.../apache/tika/parser/ocr/TesseractOCRParser.java | 2 +-
.../org/apache/tika/parser/pdf/PDFParserTest.java | 5 +-
.../org/apache/tika/pipes/core/MockEnricher.java | 5 +-
.../apache/tika/pipes/core/PipesClientTest.java | 5 +-
.../tika/config/loader/ContentEnricherLoader.java | 12 ++---
.../apache/tika/config/loader/ParserLoader.java | 12 ++---
.../config/loader/ContentEnricherLoaderTest.java | 3 +-
.../tika/config/loader/EnrichingTestParser.java | 5 +-
.../apache/tika/config/loader/TestPngEnricher.java | 5 +-
.../config/loader/TestUnavailableEnricher.java | 5 +-
17 files changed, 72 insertions(+), 129 deletions(-)
diff --git a/docs/modules/ROOT/pages/configuration/index.adoc b/docs/modules/ROOT/pages/configuration/index.adoc
index 8a1c25f468..dc90e9fee9 100644
--- a/docs/modules/ROOT/pages/configuration/index.adoc
+++ b/docs/modules/ROOT/pages/configuration/index.adoc
@@ -149,11 +149,11 @@ order listed — so an OCR engine followed by a VLM that tags images is two entr
per image. The list is authoritative: a media type no configured enricher matches gets no
enrichment — never a classpath engine you did not name — and a named engine that reports no
media types at startup (missing native binary, unreachable inference server) fails config load
-rather than going silently inert. Failures are best-effort: one enricher failing does not stop the others, and every
-failure is still reported through the parser's normal exception handling (timeouts abort the
-chain immediately). With no `content-enrichers` configured, behavior is unchanged — whichever
-OCR engine is on the classpath is used, exactly as before, and a WARN is logged at startup when
-several engines collide. Engine names: `tesseract-ocr-parser`, `tess4j-parser`, `openai-vlm-parser`,
+rather than going silently inert. Failures are best-effort: one enricher failing does not stop
+the others, and every failure is still reported through the parser's normal exception handling
+(timeouts abort the chain immediately). With no `content-enrichers` configured, behavior is
+unchanged — whichever OCR engine is on the classpath is used, exactly as before, and a WARN is
+logged at startup when several engines collide. Engine names: `tesseract-ocr-parser`, `tess4j-parser`, `openai-vlm-parser`,
`claude-vlm-parser`, `gemini-vlm-parser`.
== Windows file paths
diff --git a/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java b/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java
index 767ab9d3cf..398b3fb225 100644
--- a/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java
+++ b/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java
@@ -29,24 +29,15 @@ import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
/**
- * Media-type-keyed registry of content enrichers: ordinary {@link Parser}s that a
- * container parser <em>invokes</em> on bytes it has already parsed to obtain derived
- * content (OCR text for an image, for a rendered PDF page, ...), rather than being
- * dispatched to by the composite parser.
+ * Media-type-keyed registry of content enrichers: ordinary {@link Parser}s that a container
+ * parser <em>invokes</em> on bytes it has already parsed (OCR text for an image or a
+ * rendered PDF page), rather than being dispatched to by the composite parser. Configured
+ * as the top-level {@code "content-enrichers"} list, mirroring {@code "renderers"}.
* <p>
- * Configured as the top-level {@code "content-enrichers"} list, mirroring
- * {@code "renderers"}; members advertise their <em>real</em> media types
- * ({@code image/png}). Legacy OCR engines that still advertise the {@code image/ocr-*}
- * pseudo-types are keyed under the corresponding real type, so they are nameable here
- * without modification. An enricher registered here does not
- * compete with the parser registered for the same type: the parser still runs and calls
- * the enricher.
- * <p>
- * <b>Every</b> enricher matching a media type runs, in config order — e.g. an OCR engine
- * followed by a VLM tagger for the same image. Output lands at the caller's chosen
- * position in that order. Failures are best-effort: one enricher's failure does not stop
- * the others; the first failure is rethrown after the chain completes with later ones
- * suppressed. Timeouts, SecurityException and SAXException abort the chain immediately.
+ * Members advertise their <em>real</em> media types ({@code image/png}); legacy engines
+ * still advertising the {@code image/ocr-*} pseudo-types are keyed under the real type, so
+ * they are nameable here unmodified. An enricher does not compete with the parser
+ * registered for the same type: that parser still runs and calls the enricher.
*
* @since Apache Tika 4.1
*/
@@ -61,8 +52,7 @@ public class CompositeContentEnricher implements Serializable {
ParseContext empty = new ParseContext();
for (Parser enricher : enrichers) {
for (MediaType mediaType : enricher.getSupportedTypes(empty)) {
- // legacy engines (Tesseract, VLM, ...) still advertise the image/ocr-*
- // pseudo-types; key them under the real type so they are nameable here
+ // legacy engines advertise image/ocr-*; key under the real type
MediaType keyType = stripLegacyOcrPrefix(mediaType.getBaseType());
List<Parser> forType = tmp.computeIfAbsent(keyType, k -> new ArrayList<>());
if (!forType.contains(enricher)) {
@@ -84,8 +74,8 @@ public class CompositeContentEnricher implements Serializable {
}
/**
- * @return the enrichers configured for this media type (parameters ignored; alias
- * normalization is the caller's job), in config order; empty when none
+ * @return the enrichers for this media type in config order, empty when none;
+ * parameters are ignored, alias normalization is the caller's job
*/
public List<Parser> getEnrichers(MediaType mediaType) {
List<Parser> enrichers = enricherMap.get(mediaType.getBaseType());
diff --git a/tika-core/src/main/java/org/apache/tika/parser/enricher/ContentEnrichers.java b/tika-core/src/main/java/org/apache/tika/parser/enricher/ContentEnrichers.java
index 81abc69863..62db23d9a6 100644
--- a/tika-core/src/main/java/org/apache/tika/parser/enricher/ContentEnrichers.java
+++ b/tika-core/src/main/java/org/apache/tika/parser/enricher/ContentEnrichers.java
@@ -37,22 +37,11 @@ import org.apache.tika.parser.Parser;
/**
* Resolves the content enricher for a media type.
* <p>
- * Contract for call sites:
- * <ul>
- * <li>The caller owns placement: wrap the handler (e.g. an
- * {@code EmbeddedContentHandler} over a {@code BodyContentHandler}) so the enricher
- * cannot emit its own document structure or metadata dump into the caller's XHTML.</li>
- * <li>The caller owns invocation granularity: once per image, per rendered page, per
- * segment; the slot does not dictate.</li>
- * <li>Resolve against the <em>detected</em> media type, captured at parse entry before
- * the parser can refine Content-Type (e.g. a PDF re-typed to Illustrator mid-parse
- * must still fire the enricher selected for the type it was dispatched on).</li>
- * <li>The enricher writes into the caller's {@link Metadata}; the caller must not assume
- * the metadata is untouched beyond the derived content.</li>
- * <li>An enricher that re-enters parsing must propagate the caller's
- * {@link ParseContext}: the recursion guard (like Tika's other in-parse limits)
- * rides the context, so a fresh context defeats it.</li>
- * </ul>
+ * Call sites: wrap the handler (an {@code EmbeddedContentHandler} over a
+ * {@code BodyContentHandler}) so the enricher cannot dump structure or metadata into the
+ * caller's XHTML; resolve on the <em>detected</em> type, captured before a parser can
+ * refine Content-Type mid-parse; and pass the caller's own {@link ParseContext} through --
+ * the recursion guard rides it, so a fresh context defeats it.
*
* @since Apache Tika 4.1
*/
@@ -63,18 +52,14 @@ public final class ContentEnrichers {
/**
* Returns the enricher to invoke for one media type, or null when none applies.
- * A configured {@code "content-enrichers"} list is authoritative: every configured
- * enricher matching the type runs, in config order, behind the single Parser
- * returned here, and a type no configured enricher matches gets no enrichment --
- * never a classpath engine the user did not name. Only when no list is configured
- * at all does the legacy {@code image/ocr-*} dispatch through the composite parser
- * apply. Returns null while an enrichment is already in progress in this context,
- * so an enricher that is (or invokes) a container parser cannot recurse into
- * enrichment.
+ * A configured list is authoritative: every matching enricher runs, in config order,
+ * behind the Parser returned here, and an uncovered type gets no enrichment -- never a
+ * classpath engine nobody named. Legacy {@code image/ocr-*} dispatch applies only when
+ * no list is configured. Null while an enrichment is already in progress in this
+ * context, so an enricher that is (or invokes) a container parser cannot recurse.
*
- * @param enrichers the injected composite; may be null when none is configured
+ * @param enrichers the injected composite; may be null when none is configured
* @param mediaType the real, normalized media type of the bytes; may be null
- * @param context the parse context
*/
public static Parser get(CompositeContentEnricher enrichers, MediaType mediaType,
ParseContext context) {
@@ -102,13 +87,10 @@ public final class ContentEnrichers {
}
/**
- * Runs each enricher in config order, best-effort: one enricher's failure does not
- * stop the others. The first failure is rethrown after the chain completes, with
- * later failures attached as suppressed, so call sites report every failure through
- * their existing exception handling. Timeouts, SecurityException, SAXException
- * (incl. write-limit aborts) and other runtime exceptions propagate immediately --
- * a spent budget or a suspect handler must not fund further enrichments -- with any
- * earlier recorded failure attached as suppressed.
+ * Runs each enricher in config order, best-effort: the first failure is rethrown once
+ * the chain completes, later ones suppressed onto it. Timeouts, SecurityException,
+ * SAXException (incl. write-limit aborts) and runtime exceptions abort immediately,
+ * carrying any earlier failure -- a spent budget must not fund more enrichments.
*/
private static final class SequentialEnricher implements Parser {
@@ -167,9 +149,8 @@ public final class ContentEnrichers {
}
/**
- * Marks enrichment in progress around the delegate so {@link #get} refuses re-entry,
- * and restores Content-Type afterwards: an enricher derives content, it does not get
- * to re-type the caller's document.
+ * Marks enrichment in progress so {@link #get} refuses re-entry, and restores
+ * Content-Type: an enricher derives content, it does not re-type the document.
*/
private static final class GuardedEnricher implements Parser {
@@ -195,8 +176,7 @@ public final class ContentEnrichers {
context.set(ActiveEnrichment.class, active);
}
String contentType = metadata.get(HttpHeaders.CONTENT_TYPE);
- // restore rather than clear: a nested invocation must not strip the
- // outer enrichment's re-entry protection when it completes
+ // restore, don't clear: a nested call must not lift the outer guard
boolean wasActive = active.active;
active.active = true;
try {
diff --git a/tika-core/src/main/java/org/apache/tika/parser/enricher/EnrichingParser.java b/tika-core/src/main/java/org/apache/tika/parser/enricher/EnrichingParser.java
index 087f34e184..f1b542e47a 100644
--- a/tika-core/src/main/java/org/apache/tika/parser/enricher/EnrichingParser.java
+++ b/tika-core/src/main/java/org/apache/tika/parser/enricher/EnrichingParser.java
@@ -17,8 +17,8 @@
package org.apache.tika.parser.enricher;
/**
- * A parser that invokes content enrichers (e.g. OCR on its images or rendered pages).
- * The configured {@link CompositeContentEnricher} is injected at load time, the way
+ * A parser that invokes content enrichers (OCR on its images or rendered pages). The
+ * configured {@link CompositeContentEnricher} is injected at load time, as
* {@link org.apache.tika.parser.RenderingParser} receives its renderer.
*
* @since Apache Tika 4.1
diff --git a/tika-core/src/main/java/org/apache/tika/parser/enricher/LegacyDispatchEnricher.java b/tika-core/src/main/java/org/apache/tika/parser/enricher/LegacyDispatchEnricher.java
index f880250e2a..1f32bff1e0 100644
--- a/tika-core/src/main/java/org/apache/tika/parser/enricher/LegacyDispatchEnricher.java
+++ b/tika-core/src/main/java/org/apache/tika/parser/enricher/LegacyDispatchEnricher.java
@@ -34,15 +34,12 @@ import org.apache.tika.parser.Parser;
/**
* Reproduces the pre-4.1 {@code image/ocr-*} dispatch when no {@code "content-enrichers"}
- * list is configured: mints the synthetic {@code ocr-} media type, sets
- * {@link TikaCoreProperties#CONTENT_TYPE_PARSER_OVERRIDE} and re-enters the composite
- * parser, restoring the metadata afterwards. Whichever engine won the {@code ocr-*}
- * registration in the composite still wins here, so precedence-by-presence (adding
- * e.g. tika-parser-tess4j-module to the classpath) is preserved exactly.
- * <p>
- * This confines the pseudo-mime dance formerly hand-rolled in both
- * {@code AbstractImageParser} and {@code AbstractPDF2XHTML} to one class, to be retired
- * once OCR engines are selected by name.
+ * list is configured: mints the synthetic {@code ocr-} type, sets
+ * {@link TikaCoreProperties#CONTENT_TYPE_PARSER_OVERRIDE}, re-enters the composite parser
+ * and restores the metadata. Whichever engine won the {@code ocr-*} registration still wins,
+ * so precedence-by-presence is preserved exactly. Confines the pseudo-mime dance formerly
+ * hand-rolled in {@code AbstractImageParser} and {@code AbstractPDF2XHTML} to one class;
+ * retire it once every engine is selected by name.
*
* @since Apache Tika 4.1
*/
@@ -57,10 +54,9 @@ public class LegacyDispatchEnricher implements Parser {
private final Parser composite;
/**
- * @param mediaType the real (already normalized) media type of the bytes to derive from
- * @param composite the composite parser to re-enter; the caller has already verified
- * it claims the synthetic {@code ocr-} type (re-verifying here would
- * rebuild the composite's full supported-types map per invocation)
+ * @param mediaType the real (already normalized) media type of the bytes
+ * @param composite the composite to re-enter; the caller has already verified it claims
+ * the {@code ocr-} type -- re-checking rebuilds its full type map
*/
public LegacyDispatchEnricher(MediaType mediaType, Parser composite) {
this.mediaType = mediaType;
diff --git a/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java b/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java
index 884a9972b5..416e08993e 100644
--- a/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java
+++ b/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java
@@ -124,8 +124,7 @@ public class ContentEnrichersTest {
@Test
public void testConfiguredListIsAuthoritative() throws Exception {
- // the composite claims ocr-tiff, but a configured list that doesn't cover tiff
- // must yield no enricher -- never a classpath engine the user did not name
+ // the composite claims ocr-tiff, but a list that doesn't cover tiff wins anyway
RecordingParser explicit = new RecordingParser(Collections.singleton(PNG));
RecordingParser composite =
new RecordingParser(Collections.singleton(MediaType.image("ocr-tiff")));
@@ -299,9 +298,8 @@ public class ContentEnrichersTest {
}
/**
- * Legacy engines (Tesseract, the VLM parsers) advertise image/ocr-* pseudo-types;
- * naming one as an enricher must still match the real type — and an engine
- * advertising both the real and the pseudo form of a type must run once, not twice.
+ * A legacy engine's image/ocr-* advertisement must match the real type, and an engine
+ * advertising both forms must run once, not twice.
*/
@Test
public void testLegacyOcrTypeAdvertisementsMatchRealTypes() throws Exception {
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/test/java/org/apache/tika/parser/image/ImageParserTest.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/test/java/org/apache/tika/parser/image/ImageParserTest.java
index c8549f31d3..dd0596bdf9 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/test/java/org/apache/tika/parser/image/ImageParserTest.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/test/java/org/apache/tika/parser/image/ImageParserTest.java
@@ -227,9 +227,8 @@ public class ImageParserTest extends TikaTest {
}
/**
- * A content enricher selected by name advertises real types and is invoked by the
- * image parser, which keeps extracting its own metadata -- the enricher does not
- * displace it (TIKA-4872).
+ * A named enricher is invoked by the image parser, which keeps extracting its own
+ * metadata -- the enricher does not displace it (TIKA-4872).
*/
@Test
public void testExplicitContentEnricher() throws Exception {
@@ -261,9 +260,8 @@ public class ImageParserTest extends TikaTest {
}
/**
- * The enricher is selected on the DETECTED media type, captured before the parser
- * can refine Content-Type mid-parse (TIKA-4872): a parser that re-types the document
- * during metadata extraction must still fire the enricher chosen at dispatch.
+ * A parser that re-types the document mid-parse must still fire the enricher chosen
+ * for the DETECTED type it was dispatched on (TIKA-4872).
*/
@Test
public void testEnricherSelectedOnDetectedTypeNotRefinedType() throws Exception {
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-ocr-module/src/main/java/org/apache/tika/parser/ocr/TesseractOCRParser.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-ocr-module/src/main/java/org/apache/tika/parser/ocr/TesseractOCRParser.java
index 266abd7df8..6312df608c 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-ocr-module/src/main/java/org/apache/tika/parser/ocr/TesseractOCRParser.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-ocr-module/src/main/java/org/apache/tika/parser/ocr/TesseractOCRParser.java
@@ -96,7 +96,7 @@ import org.apache.tika.utils.XMLReaderUtils;
* parseContext.set(TesseractOCRConfig.class, config);<br>
* </p>
*/
-// name pinned: it is the documented "content-enrichers" selector for this engine
+// name pinned: the documented "content-enrichers" selector for this engine
@TikaComponent(name = "tesseract-ocr-parser")
public class TesseractOCRParser extends AbstractExternalProcessParser implements Initializable {
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java
index 02ebd1ffe0..8fd7eabd64 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java
@@ -1662,9 +1662,8 @@ public class PDFParserTest extends TikaTest {
}
/**
- * A content enricher selected by name -- advertising real image types, never the
- * ocr- pseudo types, with no composite-parser registration at all -- receives every
- * rendered page when OCR runs (TIKA-4872).
+ * A named enricher advertising real image types, with no composite registration at all,
+ * receives every rendered page when OCR runs (TIKA-4872).
*/
@Test
public void testExplicitContentEnricherReceivesRenderedPages() throws Exception {
diff --git a/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/MockEnricher.java b/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/MockEnricher.java
index c51cf77238..73b499b270 100644
--- a/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/MockEnricher.java
+++ b/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/MockEnricher.java
@@ -30,10 +30,7 @@ import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
import org.apache.tika.sax.XHTMLContentHandler;
-/**
- * A content enricher selectable by name in "content-enrichers": proves the invocation
- * and config path (is it run, does its output arrive) with no OCR binary installed.
- */
+/** Fixture: proves the invocation and config path with no OCR binary installed. */
@TikaComponent(name = "mock-enricher", spi = false)
public class MockEnricher implements Parser {
diff --git a/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/PipesClientTest.java b/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/PipesClientTest.java
index ec1b7dc8ad..8233f939eb 100644
--- a/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/PipesClientTest.java
+++ b/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/PipesClientTest.java
@@ -84,9 +84,8 @@ public class PipesClientTest {
}
/**
- * Wire test for the content-enrichers slot (TIKA-4872): a config-named enricher must
- * be injected into the fork's parsers and its output must survive the fork boundary.
- * Uses MockEnricher, so no OCR binary is needed.
+ * Wire test for the content-enrichers slot (TIKA-4872): a config-named enricher is
+ * injected into the fork's parsers and its output survives the fork boundary.
*/
@Test
public void testContentEnricherInFork(@TempDir Path tmp) throws Exception {
diff --git a/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java b/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java
index 98aeb535cf..781b3feb93 100644
--- a/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java
+++ b/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java
@@ -29,10 +29,9 @@ import org.apache.tika.parser.Parser;
import org.apache.tika.parser.enricher.CompositeContentEnricher;
/**
- * Loads the top-level {@code "content-enrichers"} list: ordinary parsers, selected by
- * component name, that container parsers invoke for derived content (OCR, ...). Members
- * come from the same registry as {@code "parsers"} entries but do not join the composite
- * parser's media-type dispatch.
+ * Loads the top-level {@code "content-enrichers"} list: parsers selected by component name
+ * that container parsers invoke for derived content (OCR, ...). Members come from the same
+ * registry as {@code "parsers"} entries but never join the composite's media-type dispatch.
*/
class ContentEnricherLoader implements ComponentLoader<CompositeContentEnricher> {
@@ -55,9 +54,8 @@ class ContentEnricherLoader implements ComponentLoader<CompositeContentEnricher>
throw new TikaConfigException(
"Failed to load content enricher: " + entry.getKey(), e);
}
- // engines report no types when unusable (missing binary, unreachable server);
- // the media-type snapshot taken here lasts the life of the process, so an
- // explicitly named engine that can never run must fail load, not go silent
+ // this type snapshot lasts the life of the process, so an engine reporting
+ // nothing (missing binary, dead server) must fail load, not go silently inert
if (enricher.getSupportedTypes(empty).isEmpty()) {
throw new TikaConfigException("Content enricher \"" + entry.getKey()
+ "\" advertises no media types. Is the engine unavailable "
diff --git a/tika-serialization/src/main/java/org/apache/tika/config/loader/ParserLoader.java b/tika-serialization/src/main/java/org/apache/tika/config/loader/ParserLoader.java
index 9fb1cdbf64..383e908552 100644
--- a/tika-serialization/src/main/java/org/apache/tika/config/loader/ParserLoader.java
+++ b/tika-serialization/src/main/java/org/apache/tika/config/loader/ParserLoader.java
@@ -133,7 +133,6 @@ public class ParserLoader extends AbstractSpiComponentLoader<Parser> {
@Override
protected Parser postProcess(Parser parser, LoaderContext context)
throws TikaConfigException {
- // Inject EncodingDetector, Renderer and content enrichers into parsers that need them
EncodingDetector encodingDetector = context.getEncodingDetector();
Renderer renderer = context.getRenderer();
CompositeContentEnricher contentEnrichers = context.getContentEnrichers();
@@ -170,12 +169,11 @@ public class ParserLoader extends AbstractSpiComponentLoader<Parser> {
}
/**
- * The image/ocr-* pseudo-types are claimed by several OCR engines whose availability
- * is environmental, and the composite resolves a collision by last registration with
- * no warning. Name the collision and the winner once at load so engine selection is
- * debuggable; select an engine explicitly with "content-enrichers". Skipped when
- * content-enrichers is configured: the list is authoritative and legacy ocr-*
- * dispatch never runs, so the collision is moot and the advice already taken.
+ * Several OCR engines can claim the same image/ocr-* pseudo-type -- availability is
+ * environmental -- and the composite resolves the collision silently by last
+ * registration; name the collision and the winner once at load. The caller skips this
+ * when content-enrichers is configured: that list is authoritative, so legacy dispatch
+ * never runs and the advice is already taken.
*/
private void warnOnAmbiguousOcrRegistrations(Parser parser) {
if (!(parser instanceof CompositeParser cp)) {
diff --git a/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java b/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java
index 372987f348..49acce1070 100644
--- a/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java
+++ b/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java
@@ -69,8 +69,7 @@ public class ContentEnricherLoaderTest {
@Test
public void testZeroTypeEnricherFailsLoad() throws Exception {
- // an explicitly named engine that cannot run (missing binary, unreachable
- // server) must fail config load, not become a silent no-op
+ // a named engine that cannot run must fail load, not become a silent no-op
TikaLoader loader = load("""
{
"content-enrichers": [ {"test-unavailable-enricher": {}} ]
diff --git a/tika-serialization/src/test/java/org/apache/tika/config/loader/EnrichingTestParser.java b/tika-serialization/src/test/java/org/apache/tika/config/loader/EnrichingTestParser.java
index f3cbc2fe8b..aca501d486 100644
--- a/tika-serialization/src/test/java/org/apache/tika/config/loader/EnrichingTestParser.java
+++ b/tika-serialization/src/test/java/org/apache/tika/config/loader/EnrichingTestParser.java
@@ -30,10 +30,7 @@ import org.apache.tika.parser.Parser;
import org.apache.tika.parser.enricher.CompositeContentEnricher;
import org.apache.tika.parser.enricher.EnrichingParser;
-/**
- * Fixture for asserting that ParserLoader injects the configured content enrichers
- * into parsers implementing {@link EnrichingParser}.
- */
+/** Fixture: asserts ParserLoader injects the enrichers into an {@link EnrichingParser}. */
@TikaComponent(name = "enriching-test-parser", spi = false)
public class EnrichingTestParser implements Parser, EnrichingParser {
diff --git a/tika-serialization/src/test/java/org/apache/tika/config/loader/TestPngEnricher.java b/tika-serialization/src/test/java/org/apache/tika/config/loader/TestPngEnricher.java
index 6dee97ef65..8e76753420 100644
--- a/tika-serialization/src/test/java/org/apache/tika/config/loader/TestPngEnricher.java
+++ b/tika-serialization/src/test/java/org/apache/tika/config/loader/TestPngEnricher.java
@@ -28,10 +28,7 @@ import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
-/**
- * Content-enricher fixture: an ordinary parser advertising a real media type,
- * selectable by name in the "content-enrichers" list.
- */
+/** Fixture: an ordinary parser advertising a real media type, nameable in "content-enrichers". */
@TikaComponent(name = "test-png-enricher", spi = false)
public class TestPngEnricher implements Parser {
diff --git a/tika-serialization/src/test/java/org/apache/tika/config/loader/TestUnavailableEnricher.java b/tika-serialization/src/test/java/org/apache/tika/config/loader/TestUnavailableEnricher.java
index 4f3f28d02e..313413e2e4 100644
--- a/tika-serialization/src/test/java/org/apache/tika/config/loader/TestUnavailableEnricher.java
+++ b/tika-serialization/src/test/java/org/apache/tika/config/loader/TestUnavailableEnricher.java
@@ -28,10 +28,7 @@ import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
-/**
- * Content-enricher fixture mimicking an engine that is unusable at load time
- * (missing binary, unreachable server): it advertises no media types.
- */
+/** Fixture: an engine unusable at load time (missing binary, dead server) advertises nothing. */
@TikaComponent(name = "test-unavailable-enricher", spi = false)
public class TestUnavailableEnricher implements Parser {
(tika) 02/04: TIKA-4872: content-enrichers
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch TIKA-4872
in repository https://gitbox.apache.org/repos/asf/tika.git
commit 8a19875df82fe8c341476d05df9e8d905fbe5072
Author: tallison <ta...@apache.org>
AuthorDate: Wed Sep 2 14:37:50 2026 -0400
TIKA-4872: content-enrichers
---
CHANGES.txt | 17 ++
docs/modules/ROOT/pages/configuration/index.adoc | 28 ++
.../pages/configuration/parsers/tess4j-parser.adoc | 7 +
.../parsers/tesseract-ocr-parser.adoc | 18 +-
.../pages/configuration/parsers/vlm-parsers.adoc | 4 +-
.../parser/enricher/CompositeContentEnricher.java | 98 +++++++
.../tika/parser/enricher/ContentEnrichers.java | 188 ++++++++++++++
.../tika/parser/enricher/EnrichingParser.java | 29 +++
.../parser/enricher/LegacyDispatchEnricher.java | 107 ++++++++
.../tika/parser/enricher/ContentEnrichersTest.java | 281 +++++++++++++++++++++
.../tika/parser/image/AbstractImageParser.java | 61 ++---
.../apache/tika/parser/image/ImageParserTest.java | 82 +++++-
.../apache/tika/parser/ocr/TesseractOCRParser.java | 3 +-
.../apache/tika/parser/pdf/AbstractPDF2XHTML.java | 36 ++-
.../java/org/apache/tika/parser/pdf/OCR2XHTML.java | 11 +-
.../java/org/apache/tika/parser/pdf/PDF2XHTML.java | 19 +-
.../tika/parser/pdf/PDFMarkedContent2XHTML.java | 11 +-
.../java/org/apache/tika/parser/pdf/PDFParser.java | 20 +-
.../org/apache/tika/parser/pdf/PDFParserTest.java | 43 ++++
.../org/apache/tika/pipes/core/MockEnricher.java | 59 +++++
.../apache/tika/pipes/core/PipesClientTest.java | 31 +++
.../configs/tika-config-content-enrichers.json | 56 ++++
.../tika/config/loader/ContentEnricherLoader.java | 58 +++++
.../apache/tika/config/loader/LoaderContext.java | 11 +
.../apache/tika/config/loader/ParserLoader.java | 62 ++++-
.../apache/tika/config/loader/TikaJsonConfig.java | 1 +
.../org/apache/tika/config/loader/TikaLoader.java | 9 +
.../config/loader/ContentEnricherLoaderTest.java | 97 +++++++
.../tika/config/loader/EnrichingTestParser.java | 63 +++++
.../apache/tika/config/loader/TestPngEnricher.java | 50 ++++
30 files changed, 1470 insertions(+), 90 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index b632080e80..782bf4700d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,22 @@
Release 4.1.0 - unreleased
+ * New "content-enrichers" config list (TIKA-4872): select the OCR engine
+ ("tesseract-ocr-parser", "tess4j-parser", "openai-vlm-parser", ...) by
+ name instead of by classpath registration of the image/ocr-* pseudo
+ media types. Enrichers advertise real media types (legacy engines that
+ still advertise image/ocr-* are mapped to the real type, so all are
+ nameable) and are invoked by the image and PDF parsers rather than
+ dispatched to by the composite, so an enricher no longer displaces the
+ parser registered for the same type. Enricher selection uses the
+ detected media type, captured before a parser can refine Content-Type.
+ Every enricher matching a media type runs, in config order (e.g. an
+ OCR engine then a VLM tagger for the same image), best-effort: one
+ enricher's failure does not stop the others and is still reported;
+ timeouts abort the chain. With no "content-enrichers" configured, the
+ legacy ocr-* dispatch applies unchanged; a WARN at config load now
+ names colliding OCR engines and the winner. TesseractOCRParser's
+ component name is pinned as "tesseract-ocr-parser".
+
* Inference/OCR hardening (TIKA-4871): OpenAIVLMParser no longer
auto-registers via SPI, matching its Claude/Gemini siblings; select
it by name ("openai-vlm-parser") in config. Per-request parse-context
diff --git a/docs/modules/ROOT/pages/configuration/index.adoc b/docs/modules/ROOT/pages/configuration/index.adoc
index c525061962..c2804aa51d 100644
--- a/docs/modules/ROOT/pages/configuration/index.adoc
+++ b/docs/modules/ROOT/pages/configuration/index.adoc
@@ -38,6 +38,7 @@ optional; anything you omit uses its defaults.
"encoding-detectors": [ /* encoding detector declarations */ ],
"metadata-filters": [ /* metadata filter declarations */ ],
"renderers": [ /* page renderer declarations */ ],
+ "content-enrichers": [ /* OCR engines etc., selected by name; see below */ ],
"translator": { /* translator declaration */ },
"content-handler-factory": { /* handler type for emitted content */ },
"auto-detect-parser": { /* AutoDetectParser options */ },
@@ -125,6 +126,33 @@ Configuring a parser automatically excludes its default copy, so there is no dup
`default-encoding-detector`, but it must not be mixed with explicit detector entries — see
xref:configuration/encoding-detectors.adoc[Encoding Detectors].
+== The `content-enrichers` list (4.1.0+)
+
+Content enrichers are ordinary parsers that a container parser *invokes* on bytes it has already
+parsed — an OCR engine run on an embedded image or a rendered PDF page. Naming one here selects
+the engine explicitly instead of relying on which OCR module happens to be on the classpath:
+
+[source,json]
+----
+{
+ "content-enrichers": [
+ { "tesseract-ocr-parser": { "language": "eng" } }
+ ]
+}
+----
+
+An enricher advertises its *real* media types (`image/png`, ...) and does not compete with the
+parser registered for those types: `image-parser` still parses the image and calls the enricher.
+(The bundled OCR engines still advertise legacy `image/ocr-*` types; those are mapped to the
+real type, so naming them here just works.) *Every* enricher matching a media type runs, in the
+order listed — so an OCR engine followed by a VLM that tags images is two entries, both invoked
+per image. Failures are best-effort: one enricher failing does not stop the others, and every
+failure is still reported through the parser's normal exception handling (timeouts abort the
+chain immediately). With no `content-enrichers` configured, behavior is unchanged — whichever
+OCR engine is on the classpath is used, exactly as before, and a WARN is logged at startup when
+several engines collide. Engine names: `tesseract-ocr-parser`, `tess4j-parser`, `openai-vlm-parser`,
+`claude-vlm-parser`, `gemini-vlm-parser`.
+
== Windows file paths
JSON treats the backslash as an escape character, so path options (`tesseractPath`,
diff --git a/docs/modules/ROOT/pages/configuration/parsers/tess4j-parser.adoc b/docs/modules/ROOT/pages/configuration/parsers/tess4j-parser.adoc
index 882abbbca4..251f4a0081 100644
--- a/docs/modules/ROOT/pages/configuration/parsers/tess4j-parser.adoc
+++ b/docs/modules/ROOT/pages/configuration/parsers/tess4j-parser.adoc
@@ -32,6 +32,13 @@ with a measured need for in-process OCR throughput *and* the expertise to run na
safely.
====
+Component name: `tess4j-parser`. Adding `tika-parser-tess4j-module` to the classpath is a
+deliberate opt-in and is intended to make Tess4J the OCR engine — but when both engines are
+live, the winner is decided by registration order, which is not guaranteed. Since 4.1.0 a WARN
+at startup names any engine collision and the winner; to pin the engine deterministically, name
+it in the top-level `content-enrichers` list (`tess4j-parser` or `tesseract-ocr-parser`; see
+xref:configuration/index.adoc[Configuration]).
+
`Tess4JParser` calls the Tesseract native library in-process via
https://github.com/nguyenq/tess4j[Tess4J] and JNA instead of spawning a `tesseract` child process
per image. That removes the per-file process-spawn overhead and can be significantly faster on
diff --git a/docs/modules/ROOT/pages/configuration/parsers/tesseract-ocr-parser.adoc b/docs/modules/ROOT/pages/configuration/parsers/tesseract-ocr-parser.adoc
index 73f93b0fcc..0942a774e0 100644
--- a/docs/modules/ROOT/pages/configuration/parsers/tesseract-ocr-parser.adoc
+++ b/docs/modules/ROOT/pages/configuration/parsers/tesseract-ocr-parser.adoc
@@ -18,7 +18,23 @@
= TesseractOCRParser Configuration
Configuration options for `TesseractOCRParser`, which runs the `tesseract` command-line program in
-a separate process.
+a separate process. Component name: `tesseract-ocr-parser`.
+
+== Selecting the OCR engine (4.1.0+)
+
+With no configuration, Tesseract is the OCR engine whenever the `tesseract` binary is found.
+When more than one OCR engine is on the classpath (e.g. Tess4J or a VLM parser), a WARN at
+startup names the collision and the winner. To pin the engine explicitly, name it in the
+top-level `content-enrichers` list:
+
+[source,json]
+----
+{
+ "content-enrichers": [ { "tesseract-ocr-parser": { "language": "eng" } } ]
+}
+----
+
+See xref:configuration/index.adoc[Configuration] for how `content-enrichers` works.
== Basic Configuration
diff --git a/docs/modules/ROOT/pages/configuration/parsers/vlm-parsers.adoc b/docs/modules/ROOT/pages/configuration/parsers/vlm-parsers.adoc
index 8c68b7e7d4..65b55f7daf 100644
--- a/docs/modules/ROOT/pages/configuration/parsers/vlm-parsers.adoc
+++ b/docs/modules/ROOT/pages/configuration/parsers/vlm-parsers.adoc
@@ -24,7 +24,9 @@ structured XHTML.
Three implementations are provided out of the box. None is auto-loaded: each must be
named explicitly in your configuration. (Changed in 4.1.0: `openai-vlm-parser` previously
-auto-registered via SPI.)
+auto-registered via SPI.) To use a VLM as the OCR engine for embedded images and rendered
+PDF pages, name it in the `content-enrichers` list — see
+xref:configuration/index.adoc[Configuration].
[cols="1,2,1"]
|===
diff --git a/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java b/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java
new file mode 100644
index 0000000000..db614d3118
--- /dev/null
+++ b/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.parser.enricher;
+
+import java.io.Serializable;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import org.apache.tika.mime.MediaType;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.Parser;
+
+/**
+ * Media-type-keyed registry of content enrichers: ordinary {@link Parser}s that a
+ * container parser <em>invokes</em> on bytes it has already parsed to obtain derived
+ * content (OCR text for an image, for a rendered PDF page, ...), rather than being
+ * dispatched to by the composite parser.
+ * <p>
+ * Configured as the top-level {@code "content-enrichers"} list, mirroring
+ * {@code "renderers"}; members advertise their <em>real</em> media types
+ * ({@code image/png}). Legacy OCR engines that still advertise the {@code image/ocr-*}
+ * pseudo-types are keyed under the corresponding real type, so they are nameable here
+ * without modification. An enricher registered here does not
+ * compete with the parser registered for the same type: the parser still runs and calls
+ * the enricher.
+ * <p>
+ * <b>Every</b> enricher matching a media type runs, in config order — e.g. an OCR engine
+ * followed by a VLM tagger for the same image. Output lands at the caller's chosen
+ * position in that order. Failures are best-effort: one enricher's failure does not stop
+ * the others; the first failure is rethrown after the chain completes with later ones
+ * suppressed. Timeouts, SecurityException and SAXException abort the chain immediately.
+ *
+ * @since Apache Tika 4.1
+ */
+public class CompositeContentEnricher implements Serializable {
+
+ private static final long serialVersionUID = 1L;
+
+ private final Map<MediaType, List<Parser>> enricherMap;
+
+ public CompositeContentEnricher(List<Parser> enrichers) {
+ Map<MediaType, List<Parser>> tmp = new HashMap<>();
+ ParseContext empty = new ParseContext();
+ for (Parser enricher : enrichers) {
+ for (MediaType mediaType : enricher.getSupportedTypes(empty)) {
+ // legacy engines (Tesseract, VLM, ...) still advertise the image/ocr-*
+ // pseudo-types; key them under the real type so they are nameable here
+ MediaType keyType = stripLegacyOcrPrefix(mediaType);
+ List<Parser> forType = tmp.computeIfAbsent(keyType, k -> new ArrayList<>());
+ if (!forType.contains(enricher)) {
+ forType.add(enricher);
+ }
+ }
+ }
+ tmp.replaceAll((k, v) -> Collections.unmodifiableList(v));
+ this.enricherMap = Collections.unmodifiableMap(tmp);
+ }
+
+ private static MediaType stripLegacyOcrPrefix(MediaType mediaType) {
+ String subtype = mediaType.getSubtype();
+ if (subtype.startsWith(LegacyDispatchEnricher.OCR_MEDIATYPE_PREFIX)) {
+ return new MediaType(mediaType.getType(),
+ subtype.substring(LegacyDispatchEnricher.OCR_MEDIATYPE_PREFIX.length()));
+ }
+ return mediaType;
+ }
+
+ /**
+ * @return the enrichers configured for this exact media type, in config order;
+ * empty when none
+ */
+ public List<Parser> getEnrichers(MediaType mediaType) {
+ List<Parser> enrichers = enricherMap.get(mediaType);
+ return enrichers == null ? Collections.emptyList() : enrichers;
+ }
+
+ public Set<MediaType> getSupportedTypes() {
+ return enricherMap.keySet();
+ }
+}
diff --git a/tika-core/src/main/java/org/apache/tika/parser/enricher/ContentEnrichers.java b/tika-core/src/main/java/org/apache/tika/parser/enricher/ContentEnrichers.java
new file mode 100644
index 0000000000..f2177c75e0
--- /dev/null
+++ b/tika-core/src/main/java/org/apache/tika/parser/enricher/ContentEnrichers.java
@@ -0,0 +1,188 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.parser.enricher;
+
+import java.io.IOException;
+import java.nio.file.Path;
+import java.util.List;
+import java.util.Set;
+
+import org.xml.sax.ContentHandler;
+import org.xml.sax.SAXException;
+
+import org.apache.tika.exception.TikaException;
+import org.apache.tika.exception.TikaTimeoutException;
+import org.apache.tika.extractor.EmbeddedDocumentUtil;
+import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.mime.MediaType;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.Parser;
+
+/**
+ * Resolves the content enricher for a media type.
+ * <p>
+ * Contract for call sites:
+ * <ul>
+ * <li>The caller owns placement: wrap the handler (e.g. an
+ * {@code EmbeddedContentHandler} over a {@code BodyContentHandler}) so the enricher
+ * cannot emit its own document structure or metadata dump into the caller's XHTML.</li>
+ * <li>The caller owns invocation granularity: once per image, per rendered page, per
+ * segment; the slot does not dictate.</li>
+ * <li>Resolve against the <em>detected</em> media type, captured at parse entry before
+ * the parser can refine Content-Type (e.g. a PDF re-typed to Illustrator mid-parse
+ * must still fire the enricher selected for the type it was dispatched on).</li>
+ * <li>The enricher writes into the caller's {@link Metadata}; the caller must not assume
+ * the metadata is untouched beyond the derived content.</li>
+ * </ul>
+ *
+ * @since Apache Tika 4.1
+ */
+public final class ContentEnrichers {
+
+ private ContentEnrichers() {
+ }
+
+ /**
+ * Returns the enricher to invoke for one media type, or null when none applies.
+ * Explicitly configured enrichers win: every one matching the type runs, in config
+ * order, behind the single Parser returned here. Otherwise the legacy
+ * {@code image/ocr-*} dispatch through the composite parser applies when an engine
+ * claims the synthetic type. Returns null while a enrichment is already in progress
+ * in this context, so an enricher that is (or invokes) a container parser cannot
+ * recurse into enrichment.
+ *
+ * @param enrichers the injected composite; may be null when none is configured
+ * @param mediaType the real, normalized media type of the bytes; may be null
+ * @param context the parse context
+ */
+ public static Parser get(CompositeContentEnricher enrichers, MediaType mediaType,
+ ParseContext context) {
+ if (mediaType == null) {
+ return null;
+ }
+ ActiveEnrichment active = context.get(ActiveEnrichment.class);
+ if (active != null && active.active) {
+ return null;
+ }
+ if (enrichers != null) {
+ List<Parser> matched = enrichers.getEnrichers(mediaType);
+ if (!matched.isEmpty()) {
+ return new GuardedEnricher(matched.size() == 1
+ ? matched.get(0) : new SequentialEnricher(matched));
+ }
+ }
+ Parser composite = EmbeddedDocumentUtil.getStatelessParser(context);
+ if (composite != null && composite.getSupportedTypes(context)
+ .contains(LegacyDispatchEnricher.toOcrMediaType(mediaType))) {
+ return new GuardedEnricher(new LegacyDispatchEnricher(mediaType));
+ }
+ return null;
+ }
+
+ /**
+ * Runs each enricher in config order, best-effort: one enricher's failure does not
+ * stop the others. The first failure is rethrown after the chain completes, with
+ * later failures attached as suppressed, so call sites report every failure through
+ * their existing exception handling. Timeouts, SecurityException and SAXException
+ * (incl. write-limit aborts) propagate immediately -- a spent budget or a suspect
+ * handler must not fund further enrichments.
+ */
+ private static final class SequentialEnricher implements Parser {
+
+ private static final long serialVersionUID = 1L;
+
+ private final List<Parser> delegates;
+
+ private SequentialEnricher(List<Parser> delegates) {
+ this.delegates = delegates;
+ }
+
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return delegates.get(0).getSupportedTypes(context);
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
+ ParseContext context) throws IOException, SAXException, TikaException {
+ // each delegate gets the bytes from the start; getPath() spools once at most
+ Path path = tis.getPath();
+ Exception first = null;
+ for (Parser delegate : delegates) {
+ try (TikaInputStream fresh = TikaInputStream.get(path)) {
+ delegate.parse(fresh, handler, metadata, context);
+ } catch (SecurityException | TikaTimeoutException | SAXException e) {
+ if (first != null) {
+ e.addSuppressed(first);
+ }
+ throw e;
+ } catch (IOException | TikaException e) {
+ if (first == null) {
+ first = e;
+ } else {
+ first.addSuppressed(e);
+ }
+ }
+ }
+ if (first instanceof IOException e) {
+ throw e;
+ }
+ if (first instanceof TikaException e) {
+ throw e;
+ }
+ }
+ }
+
+ /** Mutable per-parse marker; single-threaded within one parse. */
+ static final class ActiveEnrichment {
+ boolean active;
+ }
+
+ /** Marks enrichment in progress around the delegate so {@link #get} refuses re-entry. */
+ private static final class GuardedEnricher implements Parser {
+
+ private static final long serialVersionUID = 1L;
+
+ private final Parser delegate;
+
+ private GuardedEnricher(Parser delegate) {
+ this.delegate = delegate;
+ }
+
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return delegate.getSupportedTypes(context);
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
+ ParseContext context) throws IOException, SAXException, TikaException {
+ ActiveEnrichment active = context.get(ActiveEnrichment.class);
+ if (active == null) {
+ active = new ActiveEnrichment();
+ context.set(ActiveEnrichment.class, active);
+ }
+ active.active = true;
+ try {
+ delegate.parse(tis, handler, metadata, context);
+ } finally {
+ active.active = false;
+ }
+ }
+ }
+}
diff --git a/tika-core/src/main/java/org/apache/tika/parser/enricher/EnrichingParser.java b/tika-core/src/main/java/org/apache/tika/parser/enricher/EnrichingParser.java
new file mode 100644
index 0000000000..087f34e184
--- /dev/null
+++ b/tika-core/src/main/java/org/apache/tika/parser/enricher/EnrichingParser.java
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.parser.enricher;
+
+/**
+ * A parser that invokes content enrichers (e.g. OCR on its images or rendered pages).
+ * The configured {@link CompositeContentEnricher} is injected at load time, the way
+ * {@link org.apache.tika.parser.RenderingParser} receives its renderer.
+ *
+ * @since Apache Tika 4.1
+ */
+public interface EnrichingParser {
+
+ void setContentEnrichers(CompositeContentEnricher contentEnrichers);
+}
diff --git a/tika-core/src/main/java/org/apache/tika/parser/enricher/LegacyDispatchEnricher.java b/tika-core/src/main/java/org/apache/tika/parser/enricher/LegacyDispatchEnricher.java
new file mode 100644
index 0000000000..7b40938fb5
--- /dev/null
+++ b/tika-core/src/main/java/org/apache/tika/parser/enricher/LegacyDispatchEnricher.java
@@ -0,0 +1,107 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.parser.enricher;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.Set;
+
+import org.xml.sax.ContentHandler;
+import org.xml.sax.SAXException;
+
+import org.apache.tika.exception.TikaException;
+import org.apache.tika.extractor.EmbeddedDocumentUtil;
+import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.HttpHeaders;
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.metadata.TikaCoreProperties;
+import org.apache.tika.mime.MediaType;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.Parser;
+
+/**
+ * Reproduces the pre-4.1 {@code image/ocr-*} dispatch when no enricher is configured for
+ * a media type: mints the synthetic {@code ocr-} media type, sets
+ * {@link TikaCoreProperties#CONTENT_TYPE_PARSER_OVERRIDE} and re-enters the composite
+ * parser, restoring the metadata afterwards. Whichever engine won the {@code ocr-*}
+ * registration in the composite still wins here, so precedence-by-presence (adding
+ * e.g. tika-parser-tess4j-module to the classpath) is preserved exactly.
+ * <p>
+ * This confines the pseudo-mime dance formerly hand-rolled in both
+ * {@code AbstractImageParser} and {@code AbstractPDF2XHTML} to one class, to be retired
+ * once OCR engines are selected by name.
+ *
+ * @since Apache Tika 4.1
+ */
+public class LegacyDispatchEnricher implements Parser {
+
+ private static final long serialVersionUID = 1L;
+
+ public static final String OCR_MEDIATYPE_PREFIX = "ocr-";
+
+ private final MediaType mediaType;
+
+ /**
+ * @param mediaType the real (already normalized) media type of the bytes to derive from
+ */
+ public LegacyDispatchEnricher(MediaType mediaType) {
+ this.mediaType = mediaType;
+ }
+
+ /**
+ * @return the synthetic dispatch type for a real media type, or null if mediaType is null
+ */
+ public static MediaType toOcrMediaType(MediaType mediaType) {
+ if (mediaType == null) {
+ return null;
+ }
+ return new MediaType(mediaType.getType(), OCR_MEDIATYPE_PREFIX + mediaType.getSubtype());
+ }
+
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return Collections.singleton(mediaType);
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
+ ParseContext context) throws IOException, SAXException, TikaException {
+ MediaType ocrMediaType = toOcrMediaType(mediaType);
+ Parser composite = EmbeddedDocumentUtil.getStatelessParser(context);
+ if (composite == null
+ || !composite.getSupportedTypes(context).contains(ocrMediaType)) {
+ throw new TikaException("No parser is registered for " + ocrMediaType);
+ }
+ String originalOverride = metadata.get(TikaCoreProperties.CONTENT_TYPE_PARSER_OVERRIDE);
+ String originalContentType = metadata.get(HttpHeaders.CONTENT_TYPE);
+ metadata.set(TikaCoreProperties.CONTENT_TYPE_PARSER_OVERRIDE, ocrMediaType.toString());
+ try {
+ composite.parse(tis, handler, metadata, context);
+ } finally {
+ if (originalOverride == null) {
+ metadata.remove(TikaCoreProperties.CONTENT_TYPE_PARSER_OVERRIDE.getName());
+ } else {
+ metadata.set(TikaCoreProperties.CONTENT_TYPE_PARSER_OVERRIDE, originalOverride);
+ }
+ if (originalContentType == null) {
+ metadata.remove(HttpHeaders.CONTENT_TYPE);
+ } else {
+ metadata.set(HttpHeaders.CONTENT_TYPE, originalContentType);
+ }
+ }
+ }
+}
diff --git a/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java b/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java
new file mode 100644
index 0000000000..8f2dbdc22f
--- /dev/null
+++ b/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java
@@ -0,0 +1,281 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.parser.enricher;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertNull;
+
+import java.io.IOException;
+import java.util.Collections;
+import java.util.List;
+import java.util.Set;
+
+import org.junit.jupiter.api.Test;
+import org.xml.sax.ContentHandler;
+import org.xml.sax.SAXException;
+import org.xml.sax.helpers.DefaultHandler;
+
+import org.apache.tika.exception.TikaException;
+import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.HttpHeaders;
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.metadata.TikaCoreProperties;
+import org.apache.tika.mime.MediaType;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.Parser;
+
+public class ContentEnrichersTest {
+
+ private static final MediaType PNG = MediaType.image("png");
+ private static final MediaType OCR_PNG = MediaType.image("ocr-png");
+
+ private static class RecordingParser implements Parser {
+ private static final long serialVersionUID = 1L;
+ private final Set<MediaType> types;
+ int calls = 0;
+ String overrideSeenDuringParse;
+
+ RecordingParser(Set<MediaType> types) {
+ this.types = types;
+ }
+
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return types;
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
+ ParseContext context) {
+ calls++;
+ overrideSeenDuringParse =
+ metadata.get(TikaCoreProperties.CONTENT_TYPE_PARSER_OVERRIDE);
+ }
+ }
+
+ private static void invoke(Parser enricher, Metadata metadata, ParseContext context)
+ throws IOException, SAXException, TikaException {
+ try (TikaInputStream tis = TikaInputStream.get(new byte[0])) {
+ enricher.parse(tis, new DefaultHandler(), metadata, context);
+ }
+ }
+
+ @Test
+ public void testExplicitEnricherWinsOverLegacy() throws Exception {
+ RecordingParser explicit = new RecordingParser(Collections.singleton(PNG));
+ RecordingParser composite = new RecordingParser(Collections.singleton(OCR_PNG));
+ CompositeContentEnricher enrichers =
+ new CompositeContentEnricher(List.of(explicit));
+ ParseContext context = new ParseContext();
+ context.set(Parser.class, composite);
+
+ Parser enricher = ContentEnrichers.get(enrichers, PNG, context);
+ assertNotNull(enricher);
+ invoke(enricher, new Metadata(), context);
+ assertEquals(1, explicit.calls);
+ assertEquals(0, composite.calls);
+ // the explicit path never mints the pseudo-mime
+ assertNull(explicit.overrideSeenDuringParse);
+ }
+
+ @Test
+ public void testLegacyFallbackMintsAndRestores() throws Exception {
+ RecordingParser composite = new RecordingParser(Collections.singleton(OCR_PNG));
+ ParseContext context = new ParseContext();
+ context.set(Parser.class, composite);
+
+ Parser enricher = ContentEnrichers.get(null, PNG, context);
+ assertNotNull(enricher);
+
+ Metadata metadata = new Metadata();
+ metadata.set(HttpHeaders.CONTENT_TYPE, PNG.toString());
+ invoke(enricher, metadata, context);
+
+ assertEquals(1, composite.calls);
+ assertEquals(OCR_PNG.toString(), composite.overrideSeenDuringParse);
+ assertNull(metadata.get(TikaCoreProperties.CONTENT_TYPE_PARSER_OVERRIDE));
+ assertEquals(PNG.toString(), metadata.get(HttpHeaders.CONTENT_TYPE));
+ }
+
+ @Test
+ public void testNoneAvailable() {
+ ParseContext context = new ParseContext();
+ assertNull(ContentEnrichers.get(null, PNG, context));
+ // composite that claims nothing
+ context.set(Parser.class, new RecordingParser(Collections.emptySet()));
+ assertNull(ContentEnrichers.get(null, PNG, context));
+ assertNull(ContentEnrichers.get(null, null, context));
+ }
+
+ @Test
+ public void testAllMatchingEnrichersRunInOrder() throws Exception {
+ List<String> order = new java.util.ArrayList<>();
+ Parser first = namedEnricher("first", order, false);
+ Parser second = namedEnricher("second", order, false);
+ CompositeContentEnricher enrichers = new CompositeContentEnricher(List.of(first, second));
+ ParseContext context = new ParseContext();
+
+ Parser enricher = ContentEnrichers.get(enrichers, PNG, context);
+ assertNotNull(enricher);
+ invoke(enricher, new Metadata(), context);
+ assertEquals(List.of("first", "second"), order);
+ }
+
+ @Test
+ public void testChainIsBestEffortAndStillReportsFailure() throws Exception {
+ List<String> order = new java.util.ArrayList<>();
+ Parser failing = namedEnricher("failing", order, true);
+ Parser second = namedEnricher("second", order, false);
+ CompositeContentEnricher enrichers = new CompositeContentEnricher(List.of(failing, second));
+ ParseContext context = new ParseContext();
+
+ Parser enricher = ContentEnrichers.get(enrichers, PNG, context);
+ assertNotNull(enricher);
+ TikaException thrown = org.junit.jupiter.api.Assertions.assertThrows(TikaException.class,
+ () -> invoke(enricher, new Metadata(), context));
+ // the failure did not stop the second enricher, and was still rethrown at the end
+ assertEquals(List.of("failing", "second"), order);
+ assertEquals("failing failed", thrown.getMessage());
+ // the guard is released even when the chain throws
+ assertNotNull(ContentEnrichers.get(enrichers, PNG, context));
+ }
+
+ @Test
+ public void testTimeoutAbortsChainImmediately() throws Exception {
+ List<String> order = new java.util.ArrayList<>();
+ Parser timingOut = new Parser() {
+ private static final long serialVersionUID = 1L;
+
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return Collections.singleton(PNG);
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
+ ParseContext context) throws TikaException {
+ order.add("timingOut");
+ throw new org.apache.tika.exception.TikaTimeoutException("budget spent", 1, 1);
+ }
+ };
+ Parser second = namedEnricher("second", order, false);
+ CompositeContentEnricher enrichers =
+ new CompositeContentEnricher(List.of(timingOut, second));
+ ParseContext context = new ParseContext();
+
+ Parser enricher = ContentEnrichers.get(enrichers, PNG, context);
+ assertNotNull(enricher);
+ org.junit.jupiter.api.Assertions.assertThrows(
+ org.apache.tika.exception.TikaTimeoutException.class,
+ () -> invoke(enricher, new Metadata(), context));
+ assertEquals(List.of("timingOut"), order);
+ }
+
+ private static Parser namedEnricher(String name, List<String> order, boolean fail) {
+ return new Parser() {
+ private static final long serialVersionUID = 1L;
+
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return Collections.singleton(PNG);
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
+ ParseContext context) throws TikaException {
+ order.add(name);
+ if (fail) {
+ throw new TikaException(name + " failed");
+ }
+ }
+ };
+ }
+
+ /**
+ * Legacy engines (Tesseract, the VLM parsers) advertise image/ocr-* pseudo-types;
+ * naming one as an enricher must still match the real type — and an engine
+ * advertising both the real and the pseudo form of a type must run once, not twice.
+ */
+ @Test
+ public void testLegacyOcrTypeAdvertisementsMatchRealTypes() throws Exception {
+ List<String> order = new java.util.ArrayList<>();
+ Parser legacyEngine = new Parser() {
+ private static final long serialVersionUID = 1L;
+
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return Set.of(OCR_PNG, MediaType.image("jp2"), MediaType.image("ocr-jp2"));
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
+ ParseContext context) {
+ order.add("legacyEngine");
+ }
+ };
+ CompositeContentEnricher enrichers =
+ new CompositeContentEnricher(List.of(legacyEngine));
+ ParseContext context = new ParseContext();
+
+ Parser forPng = ContentEnrichers.get(enrichers, PNG, context);
+ assertNotNull(forPng, "ocr-png advertisement must be nameable for image/png");
+ invoke(forPng, new Metadata(), context);
+ assertEquals(List.of("legacyEngine"), order);
+
+ order.clear();
+ Parser forJp2 = ContentEnrichers.get(enrichers, MediaType.image("jp2"), context);
+ assertNotNull(forJp2);
+ invoke(forJp2, new Metadata(), context);
+ assertEquals(List.of("legacyEngine"), order,
+ "real + pseudo advertisement of the same type must run once");
+ }
+
+ @Test
+ public void testRecursionGuard() throws Exception {
+ ParseContext context = new ParseContext();
+ // an enricher that tries to re-enter enrichment from inside its own parse
+ Parser reentrant = new Parser() {
+ private static final long serialVersionUID = 1L;
+
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext ctx) {
+ return Collections.singleton(PNG);
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
+ ParseContext ctx) {
+ metadata.set("nested-enricher",
+ ContentEnrichers.get(
+ ctx.get(CompositeContentEnricher.class), PNG, ctx) == null
+ ? "refused" : "allowed");
+ }
+ };
+ CompositeContentEnricher enrichers = new CompositeContentEnricher(List.of(reentrant));
+ context.set(CompositeContentEnricher.class, enrichers);
+
+ Parser enricher = ContentEnrichers.get(enrichers, PNG, context);
+ assertNotNull(enricher);
+ Metadata metadata = new Metadata();
+ invoke(enricher, metadata, context);
+ assertEquals("refused", metadata.get("nested-enricher"));
+
+ // and enrichment is available again once the first one completes
+ assertNotNull(ContentEnrichers.get(enrichers, PNG, context));
+ }
+}
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/main/java/org/apache/tika/parser/image/AbstractImageParser.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/main/java/org/apache/tika/parser/image/AbstractImageParser.java
index 954649b2b0..2e53f2a2c6 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/main/java/org/apache/tika/parser/image/AbstractImageParser.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/main/java/org/apache/tika/parser/image/AbstractImageParser.java
@@ -24,34 +24,28 @@ import org.xml.sax.ContentHandler;
import org.xml.sax.SAXException;
import org.apache.tika.exception.TikaException;
-import org.apache.tika.extractor.EmbeddedDocumentUtil;
import org.apache.tika.io.TemporaryResources;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.HttpHeaders;
import org.apache.tika.metadata.Metadata;
-import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
+import org.apache.tika.parser.enricher.CompositeContentEnricher;
+import org.apache.tika.parser.enricher.ContentEnrichers;
+import org.apache.tika.parser.enricher.EnrichingParser;
+import org.apache.tika.parser.enricher.LegacyDispatchEnricher;
import org.apache.tika.sax.BodyContentHandler;
import org.apache.tika.sax.EmbeddedContentHandler;
import org.apache.tika.sax.XHTMLContentHandler;
-public abstract class AbstractImageParser implements Parser {
+public abstract class AbstractImageParser implements Parser, EnrichingParser {
- public static String OCR_MEDIATYPE_PREFIX = "ocr-";
+ /** @deprecated use {@link LegacyDispatchEnricher#OCR_MEDIATYPE_PREFIX} */
+ @Deprecated
+ public static String OCR_MEDIATYPE_PREFIX = LegacyDispatchEnricher.OCR_MEDIATYPE_PREFIX;
- /**
- *
- * @param mediaType
- * @return ocr media type if mediatype is not null; returns null if mediatype is null
- */
- static MediaType convertToOCRMediaType(MediaType mediaType) {
- if (mediaType == null) {
- return null;
- }
- return new MediaType(mediaType.getType(), OCR_MEDIATYPE_PREFIX + mediaType.getSubtype());
- }
+ private CompositeContentEnricher contentEnrichers;
abstract void extractMetadata(InputStream is, ContentHandler contentHandler, Metadata metadata,
ParseContext parseContext)
@@ -63,6 +57,11 @@ public abstract class AbstractImageParser implements Parser {
return mediaType;
}
+ @Override
+ public void setContentEnrichers(CompositeContentEnricher contentEnrichers) {
+ this.contentEnrichers = contentEnrichers;
+ }
+
@Override
public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
ParseContext context) throws IOException, SAXException, TikaException {
@@ -71,10 +70,8 @@ public abstract class AbstractImageParser implements Parser {
//note: mediaType can be null if mediaTypeString is null or
//not parseable.
MediaType mediaType = normalizeMediaType(MediaType.parse(mediaTypeString));
- MediaType ocrMediaType = convertToOCRMediaType(mediaType);
- Parser ocrParser = EmbeddedDocumentUtil.getStatelessParser(context);
- if (ocrMediaType == null ||
- ocrParser == null || !ocrParser.getSupportedTypes(context).contains(ocrMediaType)) {
+ Parser enricher = ContentEnrichers.get(contentEnrichers, mediaType, context);
+ if (enricher == null) {
extractMetadata(tis, handler, metadata, context);
XHTMLContentHandler xhtml = new XHTMLContentHandler(handler, metadata, context);
xhtml.startDocument();
@@ -100,31 +97,11 @@ public abstract class AbstractImageParser implements Parser {
}
try (TikaInputStream pathStream = TikaInputStream.get(path)) {
- //specify ocr content type
- String originalParserOverride =
- metadata.get(TikaCoreProperties.CONTENT_TYPE_PARSER_OVERRIDE);
- String originalContentType = metadata.get(HttpHeaders.CONTENT_TYPE);
- metadata.set(TikaCoreProperties.CONTENT_TYPE_PARSER_OVERRIDE,
- ocrMediaType.toString());
//need to use bodycontenthandler to filter out re-dumping of metadata
//in xhtmlhandler
- try {
- ocrParser.parse(pathStream,
- new EmbeddedContentHandler(new BodyContentHandler(xhtml)), metadata,
- context);
- } finally {
- if (originalParserOverride == null) {
- metadata.remove(TikaCoreProperties.CONTENT_TYPE_PARSER_OVERRIDE.getName());
- } else {
- metadata.set(TikaCoreProperties.CONTENT_TYPE_PARSER_OVERRIDE,
- originalParserOverride);
- }
- if (originalContentType == null) {
- metadata.remove(HttpHeaders.CONTENT_TYPE);
- } else {
- metadata.set(HttpHeaders.CONTENT_TYPE, originalContentType);
- }
- }
+ enricher.parse(pathStream,
+ new EmbeddedContentHandler(new BodyContentHandler(xhtml)), metadata,
+ context);
}
xhtml.endDocument();
} finally {
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/test/java/org/apache/tika/parser/image/ImageParserTest.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/test/java/org/apache/tika/parser/image/ImageParserTest.java
index 87899d0a47..c8549f31d3 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/test/java/org/apache/tika/parser/image/ImageParserTest.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/test/java/org/apache/tika/parser/image/ImageParserTest.java
@@ -30,6 +30,7 @@ import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
+import org.apache.tika.parser.enricher.LegacyDispatchEnricher;
public class ImageParserTest extends TikaTest {
@@ -222,7 +223,86 @@ public class ImageParserTest extends TikaTest {
@Test
public void testMimeTypeToOCRMimeTypeConversion() throws Exception {
assertEquals(new MediaType("image", "OCR-png"),
- AbstractImageParser.convertToOCRMediaType(MediaType.image("png")));
+ LegacyDispatchEnricher.toOcrMediaType(MediaType.image("png")));
+ }
+
+ /**
+ * A content enricher selected by name advertises real types and is invoked by the
+ * image parser, which keeps extracting its own metadata -- the enricher does not
+ * displace it (TIKA-4872).
+ */
+ @Test
+ public void testExplicitContentEnricher() throws Exception {
+ Parser enricher = new Parser() {
+ @Override
+ public java.util.Set<MediaType> getSupportedTypes(ParseContext context) {
+ return java.util.Collections.singleton(MediaType.image("png"));
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, org.xml.sax.ContentHandler handler,
+ Metadata metadata, ParseContext context) {
+ metadata.set("derived-by", "test-enricher");
+ }
+ };
+ ImageParser imageParser = new ImageParser();
+ imageParser.setContentEnrichers(
+ new org.apache.tika.parser.enricher.CompositeContentEnricher(
+ java.util.List.of(enricher)));
+
+ Metadata metadata = new Metadata();
+ metadata.set(HttpHeaders.CONTENT_TYPE, "image/png");
+ try (TikaInputStream tis = getResourceAsStream("/test-documents/testPNG.png")) {
+ imageParser.parse(tis, new DefaultHandler(), metadata, new ParseContext());
+ }
+ assertEquals("test-enricher", metadata.get("derived-by"));
+ // the image parser still ran and extracted its own metadata
+ assertEquals("100", metadata.get(TIFF.IMAGE_WIDTH));
+ }
+
+ /**
+ * The enricher is selected on the DETECTED media type, captured before the parser
+ * can refine Content-Type mid-parse (TIKA-4872): a parser that re-types the document
+ * during metadata extraction must still fire the enricher chosen at dispatch.
+ */
+ @Test
+ public void testEnricherSelectedOnDetectedTypeNotRefinedType() throws Exception {
+ Parser enricher = new Parser() {
+ @Override
+ public java.util.Set<MediaType> getSupportedTypes(ParseContext context) {
+ return java.util.Collections.singleton(MediaType.image("png"));
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, org.xml.sax.ContentHandler handler,
+ Metadata metadata, ParseContext context) {
+ metadata.set("enriched-for", "image/png");
+ }
+ };
+ AbstractImageParser retypingParser = new AbstractImageParser() {
+ @Override
+ public java.util.Set<MediaType> getSupportedTypes(ParseContext context) {
+ return java.util.Collections.singleton(MediaType.image("png"));
+ }
+
+ @Override
+ void extractMetadata(java.io.InputStream is, org.xml.sax.ContentHandler handler,
+ Metadata metadata, ParseContext context) {
+ // simulates a parser refining detection mid-parse
+ metadata.set(HttpHeaders.CONTENT_TYPE, "application/illustrator");
+ }
+ };
+ retypingParser.setContentEnrichers(
+ new org.apache.tika.parser.enricher.CompositeContentEnricher(
+ java.util.List.of(enricher)));
+
+ Metadata metadata = new Metadata();
+ metadata.set(HttpHeaders.CONTENT_TYPE, "image/png");
+ try (TikaInputStream tis = getResourceAsStream("/test-documents/testPNG.png")) {
+ retypingParser.parse(tis, new DefaultHandler(), metadata, new ParseContext());
+ }
+ assertEquals("image/png", metadata.get("enriched-for"));
+ assertEquals("application/illustrator", metadata.get(HttpHeaders.CONTENT_TYPE));
}
@Test
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-ocr-module/src/main/java/org/apache/tika/parser/ocr/TesseractOCRParser.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-ocr-module/src/main/java/org/apache/tika/parser/ocr/TesseractOCRParser.java
index 02d49142b2..266abd7df8 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-ocr-module/src/main/java/org/apache/tika/parser/ocr/TesseractOCRParser.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-ocr-module/src/main/java/org/apache/tika/parser/ocr/TesseractOCRParser.java
@@ -96,7 +96,8 @@ import org.apache.tika.utils.XMLReaderUtils;
* parseContext.set(TesseractOCRConfig.class, config);<br>
* </p>
*/
-@TikaComponent
+// name pinned: it is the documented "content-enrichers" selector for this engine
+@TikaComponent(name = "tesseract-ocr-parser")
public class TesseractOCRParser extends AbstractExternalProcessParser implements Initializable {
public static final String TESS_META = "tess:";
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java
index fb8a725e31..b7cde47b4a 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java
@@ -116,6 +116,8 @@ import org.apache.tika.metadata.TikaPagedText;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
+import org.apache.tika.parser.enricher.CompositeContentEnricher;
+import org.apache.tika.parser.enricher.ContentEnrichers;
import org.apache.tika.parser.pdf.updates.IncrementalUpdateRecord;
import org.apache.tika.parser.pdf.updates.IsIncrementalUpdate;
import org.apache.tika.parser.pdf.updates.StartXRefOffset;
@@ -164,8 +166,8 @@ class AbstractPDF2XHTML extends PDFTextStripper {
final Metadata metadata;
final EmbeddedDocumentExtractor embeddedDocumentExtractor;
final PDFParserConfig config;
- final Parser ocrParser;
final Renderer renderer;
+ final CompositeContentEnricher contentEnrichers;
/**
* Format used for signature dates
* TODO Make this thread-safe
@@ -202,19 +204,16 @@ class AbstractPDF2XHTML extends PDFTextStripper {
int num3DAnnotations = 0;
AbstractPDF2XHTML(PDDocument pdDocument, ContentHandler handler, ParseContext context,
- Metadata metadata, PDFParserConfig config, Renderer renderer) throws IOException {
+ Metadata metadata, PDFParserConfig config, Renderer renderer,
+ CompositeContentEnricher contentEnrichers) throws IOException {
this.pdDocument = pdDocument;
this.xhtml = new XHTMLContentHandler(handler, metadata, context);
this.context = context;
this.metadata = metadata;
this.config = config;
this.renderer = renderer;
+ this.contentEnrichers = contentEnrichers;
embeddedDocumentExtractor = EmbeddedDocumentUtil.getEmbeddedDocumentExtractor(context);
- if (config.getOcr().getStrategy() == NO_OCR) {
- ocrParser = null;
- } else {
- ocrParser = EmbeddedDocumentUtil.getStatelessParser(context);
- }
}
private static void addNonNullAttribute(String name, String value, AttributesImpl attributes) {
@@ -570,17 +569,17 @@ class AbstractPDF2XHTML extends PDFTextStripper {
if (maxPagesToOcr > 0 && c != null && c.getCount() > maxPagesToOcr) {
return;
}
- MediaType ocrImageMediaType = MediaType.image("ocr-" + config.getOcr().getImageFormat().getFormatName());
- Set<MediaType> supportedTypes = ocrParser.getSupportedTypes(context);
- if (supportedTypes == null || !supportedTypes.contains(ocrImageMediaType)) {
+ MediaType imageMediaType =
+ MediaType.image(config.getOcr().getImageFormat().getFormatName());
+ Parser enricher = ContentEnrichers.get(contentEnrichers, imageMediaType, context);
+ if (enricher == null) {
if (ocrStrategy == OCR_ONLY || ocrStrategy == OCR_AND_TEXT_EXTRACTION) {
throw new TikaException(
- "" + "I regret that I couldn't find an OCR parser to handle " +
- ocrImageMediaType + "." +
- "Please set the OCR_STRATEGY to NO_OCR or configure your" +
- "OCR parser correctly");
+ "I regret that I couldn't find an OCR engine to handle " +
+ imageMediaType + ". Configure one in \"content-enrichers\", " +
+ "add one to the classpath, or set the OCR strategy to NO_OCR.");
} else if (ocrStrategy == AUTO) {
- //silently skip if there's no parser to run ocr
+ //silently skip if there's no engine to run ocr
return;
}
}
@@ -589,9 +588,8 @@ class AbstractPDF2XHTML extends PDFTextStripper {
try (RenderResult renderResult = renderCurrentPage(pdPage, tmp)) {
Metadata renderMetadata = renderResult.getMetadata();
try (TikaInputStream tis = renderResult.getInputStream()) {
- renderMetadata.set(TikaCoreProperties.CONTENT_TYPE_PARSER_OVERRIDE,
- ocrImageMediaType.toString());
- ocrParser.parse(tis, new EmbeddedContentHandler(new BodyContentHandler(xhtml)),
+ renderMetadata.set(HttpHeaders.CONTENT_TYPE, imageMediaType.toString());
+ enricher.parse(tis, new EmbeddedContentHandler(new BodyContentHandler(xhtml)),
renderMetadata, context);
}
// Propagate enrichment metadata added by the OCR parser (e.g. tk:chunks
@@ -599,7 +597,7 @@ class AbstractPDF2XHTML extends PDFTextStripper {
// silently discarded when the renderMetadata goes out of scope.
String renderChunks = renderMetadata.get(TikaCoreProperties.TIKA_CHUNKS);
if (renderChunks != null) {
- metadata.setTrusted(TikaCoreProperties.TIKA_CHUNKS.getName(),
+ metadata.set(TikaCoreProperties.TIKA_CHUNKS,
mergeChunkArrays(metadata.get(TikaCoreProperties.TIKA_CHUNKS),
renderChunks));
}
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/OCR2XHTML.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/OCR2XHTML.java
index 8eff5c597e..3e8a2bd70d 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/OCR2XHTML.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/OCR2XHTML.java
@@ -29,6 +29,7 @@ import org.xml.sax.SAXException;
import org.apache.tika.exception.TikaException;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.enricher.CompositeContentEnricher;
import org.apache.tika.renderer.Renderer;
@@ -39,8 +40,9 @@ import org.apache.tika.renderer.Renderer;
class OCR2XHTML extends AbstractPDF2XHTML {
private OCR2XHTML(PDDocument document, ContentHandler handler, ParseContext context,
- Metadata metadata, PDFParserConfig config, Renderer renderer) throws IOException {
- super(document, handler, context, metadata, config, renderer);
+ Metadata metadata, PDFParserConfig config, Renderer renderer,
+ CompositeContentEnricher contentEnrichers) throws IOException {
+ super(document, handler, context, metadata, config, renderer, contentEnrichers);
}
/**
@@ -57,12 +59,13 @@ class OCR2XHTML extends AbstractPDF2XHTML {
*/
public static void process(PDDocument document, ContentHandler handler, ParseContext context,
Metadata metadata,
- PDFParserConfig config, Renderer renderer)
+ PDFParserConfig config, Renderer renderer,
+ CompositeContentEnricher contentEnrichers)
throws SAXException, TikaException {
OCR2XHTML ocr2XHTML = null;
try {
- ocr2XHTML = new OCR2XHTML(document, handler, context, metadata, config, renderer);
+ ocr2XHTML = new OCR2XHTML(document, handler, context, metadata, config, renderer, contentEnrichers);
ocr2XHTML.writeText(document, new Writer() {
@Override
public void write(char[] cbuf, int off, int len) {
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDF2XHTML.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDF2XHTML.java
index bbf4e0fbd7..1496f586d0 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDF2XHTML.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDF2XHTML.java
@@ -42,6 +42,7 @@ import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.metadata.TikaCoreProperties;
import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.enricher.CompositeContentEnricher;
import org.apache.tika.parser.pdf.image.ImageGraphicsEngine;
import org.apache.tika.renderer.PageRangeRequest;
import org.apache.tika.renderer.RenderRequest;
@@ -73,8 +74,9 @@ class PDF2XHTML extends AbstractPDF2XHTML {
private AtomicInteger inlineImageCounter = new AtomicInteger(0);
PDF2XHTML(PDDocument document, ContentHandler handler, ParseContext context, Metadata metadata,
- PDFParserConfig config, Renderer renderer) throws IOException {
- super(document, handler, context, metadata, config, renderer);
+ PDFParserConfig config, Renderer renderer,
+ CompositeContentEnricher contentEnrichers) throws IOException {
+ super(document, handler, context, metadata, config, renderer, contentEnrichers);
}
/**
@@ -89,7 +91,8 @@ class PDF2XHTML extends AbstractPDF2XHTML {
* @throws TikaException if there was an exception outside of per page processing
*/
public static void process(PDDocument document, ContentHandler handler, ParseContext context,
- Metadata metadata, PDFParserConfig config, Renderer renderer)
+ Metadata metadata, PDFParserConfig config, Renderer renderer,
+ CompositeContentEnricher contentEnrichers)
throws SAXException, TikaException {
PDF2XHTML pdf2XHTML = null;
try {
@@ -98,9 +101,10 @@ class PDF2XHTML extends AbstractPDF2XHTML {
// handler.
if (config.isDetectAngles()) {
pdf2XHTML =
- new AngleDetectingPDF2XHTML(document, handler, context, metadata, config, renderer);
+ new AngleDetectingPDF2XHTML(document, handler, context, metadata,
+ config, renderer, contentEnrichers);
} else {
- pdf2XHTML = new PDF2XHTML(document, handler, context, metadata, config, renderer);
+ pdf2XHTML = new PDF2XHTML(document, handler, context, metadata, config, renderer, contentEnrichers);
}
config.configure(pdf2XHTML);
@@ -270,8 +274,9 @@ class PDF2XHTML extends AbstractPDF2XHTML {
private AngleDetectingPDF2XHTML(PDDocument document, ContentHandler handler,
ParseContext context, Metadata metadata,
- PDFParserConfig config, Renderer renderer) throws IOException {
- super(document, handler, context, metadata, config, renderer);
+ PDFParserConfig config, Renderer renderer,
+ CompositeContentEnricher contentEnrichers) throws IOException {
+ super(document, handler, context, metadata, config, renderer, contentEnrichers);
}
@Override
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDFMarkedContent2XHTML.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDFMarkedContent2XHTML.java
index ca016ac4d9..570c1c282d 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDFMarkedContent2XHTML.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDFMarkedContent2XHTML.java
@@ -47,6 +47,7 @@ import org.xml.sax.SAXException;
import org.apache.tika.exception.TikaException;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.enricher.CompositeContentEnricher;
import org.apache.tika.renderer.Renderer;
/**
@@ -91,9 +92,10 @@ public class PDFMarkedContent2XHTML extends PDF2XHTML {
private PDFMarkedContent2XHTML(PDDocument document, ContentHandler handler,
ParseContext context, Metadata metadata, PDFParserConfig config,
- Renderer renderer)
+ Renderer renderer,
+ CompositeContentEnricher contentEnrichers)
throws IOException {
- super(document, handler, context, metadata, config, renderer);
+ super(document, handler, context, metadata, config, renderer, contentEnrichers);
}
/**
@@ -111,14 +113,15 @@ public class PDFMarkedContent2XHTML extends PDF2XHTML {
*/
public static void process(PDDocument pdDocument, ContentHandler handler,
ParseContext context,
- Metadata metadata, PDFParserConfig config, Renderer renderer)
+ Metadata metadata, PDFParserConfig config, Renderer renderer,
+ CompositeContentEnricher contentEnrichers)
throws SAXException, TikaException {
PDFMarkedContent2XHTML pdfMarkedContent2XHTML = null;
try {
pdfMarkedContent2XHTML =
new PDFMarkedContent2XHTML(pdDocument, handler, context, metadata, config,
- renderer);
+ renderer, contentEnrichers);
} catch (IOException e) {
throw new TikaException("couldn't initialize PDFMarkedContent2XHTML", e);
}
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDFParser.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDFParser.java
index 4ad0e95401..5c987e1e64 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDFParser.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/PDFParser.java
@@ -79,6 +79,8 @@ import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
import org.apache.tika.parser.PasswordProvider;
import org.apache.tika.parser.RenderingParser;
+import org.apache.tika.parser.enricher.CompositeContentEnricher;
+import org.apache.tika.parser.enricher.EnrichingParser;
import org.apache.tika.parser.pdf.updates.IncrementalUpdateRecord;
import org.apache.tika.parser.pdf.updates.IsIncrementalUpdate;
import org.apache.tika.parser.pdf.updates.StartXRefOffset;
@@ -122,7 +124,7 @@ import org.apache.tika.sax.XHTMLContentHandler;
* {@link PDFParserConfig#setExtractMarkedContent(boolean)}
*/
@TikaComponent
-public class PDFParser implements Parser, RenderingParser {
+public class PDFParser implements Parser, RenderingParser, EnrichingParser {
public static final MediaType MEDIA_TYPE = MediaType.application("pdf");
/**
@@ -136,6 +138,7 @@ public class PDFParser implements Parser, RenderingParser {
private static COSName ENCRYPTED_PAYLOAD = COSName.getPDFName("EncryptedPayload");
private PDFParserConfig defaultConfig = new PDFParserConfig();
private Renderer renderer;
+ private CompositeContentEnricher contentEnrichers;
public PDFParser() {
}
@@ -220,14 +223,14 @@ public class PDFParser implements Parser, RenderingParser {
} else if (localConfig.getOcr().getStrategy()
.equals(OcrConfig.Strategy.OCR_ONLY)) {
OCR2XHTML.process(pdfDocument, handler, context, metadata,
- localConfig, renderer);
+ localConfig, renderer, contentEnrichers);
} else if (hasMarkedContent && localConfig.isExtractMarkedContent()) {
PDFMarkedContent2XHTML
.process(pdfDocument, handler, context, metadata,
- localConfig, renderer);
+ localConfig, renderer, contentEnrichers);
} else {
PDF2XHTML.process(pdfDocument, handler, context, metadata,
- localConfig, renderer);
+ localConfig, renderer, contentEnrichers);
}
}
} catch (InvalidPasswordException e) {
@@ -792,6 +795,15 @@ public class PDFParser implements Parser, RenderingParser {
this.renderer = renderer;
}
+ @Override
+ public void setContentEnrichers(CompositeContentEnricher contentEnrichers) {
+ this.contentEnrichers = contentEnrichers;
+ }
+
+ public CompositeContentEnricher getContentEnrichers() {
+ return contentEnrichers;
+ }
+
public Renderer getRenderer() {
return renderer;
}
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java
index b44ed01df7..02ebd1ffe0 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/test/java/org/apache/tika/parser/pdf/PDFParserTest.java
@@ -1661,6 +1661,49 @@ public class PDFParserTest extends TikaTest {
assertContains("chunk-page-2", chunks);
}
+ /**
+ * A content enricher selected by name -- advertising real image types, never the
+ * ocr- pseudo types, with no composite-parser registration at all -- receives every
+ * rendered page when OCR runs (TIKA-4872).
+ */
+ @Test
+ public void testExplicitContentEnricherReceivesRenderedPages() throws Exception {
+ PDFParserConfig config = new PDFParserConfig();
+ config.getOcr().setStrategy(OcrConfig.Strategy.OCR_ONLY);
+ ParseContext context = new ParseContext();
+ context.set(PDFParserConfig.class, config);
+
+ Parser enricher = new Parser() {
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return Collections.singleton(MediaType.image("png"));
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
+ ParseContext context) throws IOException, SAXException {
+ assertEquals("image/png",
+ metadata.get(org.apache.tika.metadata.HttpHeaders.CONTENT_TYPE));
+ XHTMLContentHandler xhtml = new XHTMLContentHandler(handler, metadata);
+ xhtml.startDocument();
+ xhtml.characters("DERIVED-PAGE-" + context.get(OCRPageCounter.class).getCount());
+ xhtml.endDocument();
+ }
+ };
+ PDFParser parser = new PDFParser();
+ parser.setContentEnrichers(
+ new org.apache.tika.parser.enricher.CompositeContentEnricher(List.of(enricher)));
+
+ Metadata metadata = new Metadata();
+ ToXMLContentHandler xmlHandler = new ToXMLContentHandler();
+ try (TikaInputStream tis = getResourceAsStream("/test-documents/testPDF_bookmarks.pdf")) {
+ parser.parse(tis, xmlHandler, metadata, context);
+ }
+ String xml = xmlHandler.toString();
+ assertContains("DERIVED-PAGE-1", xml);
+ assertContains("DERIVED-PAGE-2", xml);
+ }
+
@Test
public void testMergeChunkArrays() {
assertEquals("[b]", AbstractPDF2XHTML.mergeChunkArrays(null, "[b]"));
diff --git a/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/MockEnricher.java b/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/MockEnricher.java
new file mode 100644
index 0000000000..c51cf77238
--- /dev/null
+++ b/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/MockEnricher.java
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.pipes.core;
+
+import java.util.Collections;
+import java.util.Set;
+
+import org.xml.sax.ContentHandler;
+import org.xml.sax.SAXException;
+
+import org.apache.tika.annotation.TikaComponent;
+import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.mime.MediaType;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.Parser;
+import org.apache.tika.sax.XHTMLContentHandler;
+
+/**
+ * A content enricher selectable by name in "content-enrichers": proves the invocation
+ * and config path (is it run, does its output arrive) with no OCR binary installed.
+ */
+@TikaComponent(name = "mock-enricher", spi = false)
+public class MockEnricher implements Parser {
+
+ private static final long serialVersionUID = 1L;
+
+ public static final String MARKER_KEY = "mock-enricher";
+ public static final String MARKER_TEXT = "MOCK-ENRICHED-TEXT";
+
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return Collections.singleton(MediaType.image("png"));
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
+ ParseContext context) throws SAXException {
+ metadata.set(MARKER_KEY, "ENRICHED");
+ XHTMLContentHandler xhtml = new XHTMLContentHandler(handler, metadata);
+ xhtml.startDocument();
+ xhtml.characters(MARKER_TEXT);
+ xhtml.endDocument();
+ }
+}
diff --git a/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/PipesClientTest.java b/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/PipesClientTest.java
index 3a1002976d..ec1b7dc8ad 100644
--- a/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/PipesClientTest.java
+++ b/tika-pipes/tika-pipes-integration-tests/src/test/java/org/apache/tika/pipes/core/PipesClientTest.java
@@ -83,6 +83,37 @@ public class PipesClientTest {
}
}
+ /**
+ * Wire test for the content-enrichers slot (TIKA-4872): a config-named enricher must
+ * be injected into the fork's parsers and its output must survive the fork boundary.
+ * Uses MockEnricher, so no OCR binary is needed.
+ */
+ @Test
+ public void testContentEnricherInFork(@TempDir Path tmp) throws Exception {
+ Path tikaConfigPath = PluginsTestHelper.getFileSystemFetcherConfig(
+ "tika-config-content-enrichers.json", tmp, tmp.resolve("input"),
+ tmp.resolve("output"), false);
+ Path inputDir = tmp.resolve("input");
+ Files.createDirectories(inputDir);
+ java.awt.image.BufferedImage image =
+ new java.awt.image.BufferedImage(10, 10, java.awt.image.BufferedImage.TYPE_INT_RGB);
+ javax.imageio.ImageIO.write(image, "png", inputDir.resolve("test.png").toFile());
+
+ TikaJsonConfig tikaJsonConfig = TikaJsonConfig.load(tikaConfigPath);
+ PipesConfig pipesConfig = PipesConfig.load(tikaJsonConfig);
+ try (PipesClient pipesClient = new PipesClient(pipesConfig, tikaConfigPath)) {
+ PipesResult pipesResult = pipesClient.process(
+ new FetchEmitTuple("test.png", new FetchKey(fetcherName, "test.png"),
+ new EmitKey(), new Metadata(), new ParseContext(),
+ FetchEmitTuple.ON_PARSE_EXCEPTION.SKIP));
+ Assertions.assertNotNull(pipesResult.emitData().getMetadataList());
+ Metadata metadata = pipesResult.emitData().getMetadataList().get(0);
+ assertEquals("ENRICHED", metadata.get(MockEnricher.MARKER_KEY));
+ assertTrue(metadata.get(TikaCoreProperties.TIKA_CONTENT)
+ .contains(MockEnricher.MARKER_TEXT));
+ }
+ }
+
@Test
public void testMetadataFilter(@TempDir Path tmp) throws Exception {
ParseContext parseContext = new ParseContext();
diff --git a/tika-pipes/tika-pipes-integration-tests/src/test/resources/configs/tika-config-content-enrichers.json b/tika-pipes/tika-pipes-integration-tests/src/test/resources/configs/tika-config-content-enrichers.json
new file mode 100644
index 0000000000..798fe4ff1d
--- /dev/null
+++ b/tika-pipes/tika-pipes-integration-tests/src/test/resources/configs/tika-config-content-enrichers.json
@@ -0,0 +1,56 @@
+{
+ "content-enrichers": [ { "mock-enricher": {} } ],
+ "content-handler-factory": {
+ "basic-content-handler-factory": {
+ "type": "TEXT",
+ "writeLimit": -1,
+ "throwOnWriteLimitReached": true
+ }
+ },
+ "fetchers": {
+ "fsf": {
+ "file-system-fetcher": {
+ "basePath": "FETCHER_BASE_PATH",
+ "extractFileSystemMetadata": false
+ }
+ }
+ },
+ "emitters": {
+ "fse": {
+ "file-system-emitter": {
+ "basePath": "EMITTER_BASE_PATH",
+ "fileExtension": "json",
+ "onExists": "EXCEPTION"
+ }
+ }
+ },
+ "pipes-iterator": {
+ "file-system-pipes-iterator": {
+ "basePath": "FETCHER_BASE_PATH",
+ "countTotal": true,
+ "fetcherId": "fsf",
+ "emitterId": "fse"
+ }
+ },
+ "pipes": {
+ "parseMode": "RMETA",
+ "onParseException": "EMIT",
+ "numClients": 4,
+ "emitIntermediateResults": "EMIT_INTERMEDIATE_RESULTS",
+ "forkedJvmArgs": ["-Xmx512m"],
+ "emitStrategy": {
+ "type": "DYNAMIC",
+ "thresholdBytes": 1000000
+ }
+ },
+ "auto-detect-parser": {
+ "throwOnZeroBytes": false
+ },
+ "parse-context": {
+ "mock-digester-factory": {},
+ "timeout-limits": {
+ "progressTimeoutMillis": 5000
+ }
+ },
+ "plugin-roots": "PLUGINS_PATHS"
+}
diff --git a/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java b/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java
new file mode 100644
index 0000000000..56330a790f
--- /dev/null
+++ b/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java
@@ -0,0 +1,58 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.config.loader;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+
+import org.apache.tika.exception.TikaConfigException;
+import org.apache.tika.parser.Parser;
+import org.apache.tika.parser.enricher.CompositeContentEnricher;
+
+/**
+ * Loads the top-level {@code "content-enrichers"} list: ordinary parsers, selected by
+ * component name, that container parsers invoke for derived content (OCR, ...). Members
+ * come from the same registry as {@code "parsers"} entries but do not join the composite
+ * parser's media-type dispatch.
+ */
+class ContentEnricherLoader implements ComponentLoader<CompositeContentEnricher> {
+
+ @Override
+ public CompositeContentEnricher load(TikaJsonConfig config, LoaderContext context)
+ throws TikaConfigException {
+ List<Map.Entry<String, JsonNode>> entries = config.getArrayComponents("content-enrichers");
+ if (entries.isEmpty()) {
+ return null;
+ }
+ List<Parser> enrichers = new ArrayList<>();
+ for (Map.Entry<String, JsonNode> entry : entries) {
+ try {
+ ObjectNode wrapper = context.getObjectMapper().createObjectNode();
+ wrapper.set(entry.getKey(), entry.getValue());
+ enrichers.add(context.getObjectMapper().treeToValue(wrapper, Parser.class));
+ } catch (Exception e) {
+ throw new TikaConfigException(
+ "Failed to load content enricher: " + entry.getKey(), e);
+ }
+ }
+ return new CompositeContentEnricher(enrichers);
+ }
+}
diff --git a/tika-serialization/src/main/java/org/apache/tika/config/loader/LoaderContext.java b/tika-serialization/src/main/java/org/apache/tika/config/loader/LoaderContext.java
index 8d240d2e28..592de5bbbd 100644
--- a/tika-serialization/src/main/java/org/apache/tika/config/loader/LoaderContext.java
+++ b/tika-serialization/src/main/java/org/apache/tika/config/loader/LoaderContext.java
@@ -22,6 +22,7 @@ import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.tika.detect.EncodingDetector;
import org.apache.tika.exception.TikaConfigException;
+import org.apache.tika.parser.enricher.CompositeContentEnricher;
import org.apache.tika.renderer.Renderer;
/**
@@ -98,6 +99,16 @@ public class LoaderContext {
return get(Renderer.class);
}
+ /**
+ * Get the configured content enrichers for injection into enriching parsers.
+ *
+ * @return the composite, or null when no "content-enrichers" are configured
+ * @throws TikaConfigException if loading fails
+ */
+ public CompositeContentEnricher getContentEnrichers() throws TikaConfigException {
+ return get(CompositeContentEnricher.class);
+ }
+
/**
* Instantiate a component by name and config.
*
diff --git a/tika-serialization/src/main/java/org/apache/tika/config/loader/ParserLoader.java b/tika-serialization/src/main/java/org/apache/tika/config/loader/ParserLoader.java
index 1c37c68fff..5e44cf5cf7 100644
--- a/tika-serialization/src/main/java/org/apache/tika/config/loader/ParserLoader.java
+++ b/tika-serialization/src/main/java/org/apache/tika/config/loader/ParserLoader.java
@@ -19,9 +19,12 @@ package org.apache.tika.config.loader;
import java.io.IOException;
import java.util.HashSet;
import java.util.List;
+import java.util.Map;
import java.util.Set;
import com.fasterxml.jackson.databind.JsonNode;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import org.apache.tika.config.ServiceLoader;
import org.apache.tika.detect.EncodingDetector;
@@ -30,9 +33,12 @@ import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.AbstractEncodingDetectorParser;
import org.apache.tika.parser.CompositeParser;
import org.apache.tika.parser.DefaultParser;
+import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
import org.apache.tika.parser.ParserDecorator;
import org.apache.tika.parser.RenderingParser;
+import org.apache.tika.parser.enricher.CompositeContentEnricher;
+import org.apache.tika.parser.enricher.EnrichingParser;
import org.apache.tika.renderer.Renderer;
/**
@@ -40,11 +46,13 @@ import org.apache.tika.renderer.Renderer;
* <ul>
* <li>SPI fallback via "default-parser" marker with exclusions</li>
* <li>Mime type filtering decorations (_mime-include, _mime-exclude)</li>
- * <li>EncodingDetector and Renderer dependency injection</li>
+ * <li>EncodingDetector, Renderer and content-enricher dependency injection</li>
* </ul>
*/
public class ParserLoader extends AbstractSpiComponentLoader<Parser> {
+ private static final Logger LOG = LoggerFactory.getLogger(ParserLoader.class);
+
public ParserLoader() {
super("parsers", "default-parser", Parser.class);
}
@@ -125,10 +133,12 @@ public class ParserLoader extends AbstractSpiComponentLoader<Parser> {
@Override
protected Parser postProcess(Parser parser, LoaderContext context)
throws TikaConfigException {
- // Inject EncodingDetector and Renderer into parsers that need them
+ // Inject EncodingDetector, Renderer and content enrichers into parsers that need them
EncodingDetector encodingDetector = context.getEncodingDetector();
Renderer renderer = context.getRenderer();
- injectDependenciesRecursively(parser, encodingDetector, renderer);
+ CompositeContentEnricher contentEnrichers = context.getContentEnrichers();
+ injectDependenciesRecursively(parser, encodingDetector, renderer, contentEnrichers);
+ warnOnAmbiguousOcrRegistrations(parser);
return parser;
}
@@ -136,19 +146,59 @@ public class ParserLoader extends AbstractSpiComponentLoader<Parser> {
* Recursively inject dependencies into a parser and its children.
*/
private void injectDependenciesRecursively(Parser parser, EncodingDetector encodingDetector,
- Renderer renderer) {
+ Renderer renderer,
+ CompositeContentEnricher contentEnrichers) {
if (encodingDetector != null && parser instanceof AbstractEncodingDetectorParser aedp) {
aedp.setEncodingDetector(encodingDetector);
}
if (renderer != null && parser instanceof RenderingParser rp) {
rp.setRenderer(renderer);
}
+ if (contentEnrichers != null && parser instanceof EnrichingParser dp) {
+ dp.setContentEnrichers(contentEnrichers);
+ }
if (parser instanceof CompositeParser cp) {
for (Parser child : cp.getAllComponentParsers()) {
- injectDependenciesRecursively(child, encodingDetector, renderer);
+ injectDependenciesRecursively(child, encodingDetector, renderer, contentEnrichers);
}
} else if (parser instanceof ParserDecorator pd) {
- injectDependenciesRecursively(pd.getWrappedParser(), encodingDetector, renderer);
+ injectDependenciesRecursively(pd.getWrappedParser(), encodingDetector, renderer,
+ contentEnrichers);
+ }
+ }
+
+ /**
+ * The image/ocr-* pseudo-types are claimed by several OCR engines whose availability
+ * is environmental, and the composite resolves a collision by last registration with
+ * no warning. Name the collision and the winner once at load so engine selection is
+ * debuggable; select an engine explicitly with "content-enrichers".
+ */
+ private void warnOnAmbiguousOcrRegistrations(Parser parser) {
+ if (!(parser instanceof CompositeParser cp)) {
+ return;
+ }
+ ParseContext empty = new ParseContext();
+ Map<MediaType, List<Parser>> duplicates = cp.findDuplicateParsers(empty);
+ if (duplicates.isEmpty()) {
+ return;
+ }
+ Map<MediaType, Parser> winners = cp.getParsers(empty);
+ for (Map.Entry<MediaType, List<Parser>> e : duplicates.entrySet()) {
+ if (!e.getKey().getSubtype().startsWith("ocr-")) {
+ continue;
+ }
+ StringBuilder claimants = new StringBuilder();
+ for (Parser p : e.getValue()) {
+ if (claimants.length() > 0) {
+ claimants.append(", ");
+ }
+ claimants.append(p.getClass().getName());
+ }
+ Parser winner = winners.get(e.getKey());
+ LOG.warn("Multiple OCR engines claim {}: [{}]; {} wins by registration order. "
+ + "Select one explicitly with \"content-enrichers\".",
+ e.getKey(), claimants,
+ winner == null ? "unknown" : winner.getClass().getName());
}
}
diff --git a/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaJsonConfig.java b/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaJsonConfig.java
index d15815b4ac..04af3fb1c9 100644
--- a/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaJsonConfig.java
+++ b/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaJsonConfig.java
@@ -113,6 +113,7 @@ public class TikaJsonConfig {
"metadata-filters",
"content-handler-factory",
"renderers",
+ "content-enrichers",
"translator",
"auto-detect-parser",
"parse-context",
diff --git a/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaLoader.java b/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaLoader.java
index a1ee066b15..e117b71553 100644
--- a/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaLoader.java
+++ b/tika-serialization/src/main/java/org/apache/tika/config/loader/TikaLoader.java
@@ -48,6 +48,7 @@ import org.apache.tika.parser.AutoDetectParserConfig;
import org.apache.tika.parser.CompositeParser;
import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
+import org.apache.tika.parser.enricher.CompositeContentEnricher;
import org.apache.tika.renderer.CompositeRenderer;
import org.apache.tika.renderer.Renderer;
import org.apache.tika.sax.BasicContentHandlerFactory;
@@ -125,6 +126,10 @@ public class TikaLoader {
.wrapWith(list -> new CompositeRenderer((List<Renderer>) list))
.register();
+ ComponentConfig.builder("content-enrichers", CompositeContentEnricher.class)
+ .customLoader(new ContentEnricherLoader())
+ .register();
+
ComponentConfig.builder("translator", Translator.class)
.loadAsList()
.wrapWith(list -> list.isEmpty() ? null : (Translator) list.get(0))
@@ -789,6 +794,10 @@ public class TikaLoader {
output.set("renderers", config.getRootNode().get("renderers"));
}
+ if (config.hasArrayComponents("content-enrichers")) {
+ output.set("content-enrichers", config.getRootNode().get("content-enrichers"));
+ }
+
// Preserve auto-detect-parser config if present
JsonNode adpNode = config.getRootNode().get("auto-detect-parser");
if (adpNode != null && !adpNode.isNull()) {
diff --git a/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java b/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java
new file mode 100644
index 0000000000..46fcfeff84
--- /dev/null
+++ b/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.config.loader;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.nio.file.Files;
+import java.nio.file.Path;
+
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+import org.apache.tika.mime.MediaType;
+import org.apache.tika.parser.CompositeParser;
+import org.apache.tika.parser.Parser;
+import org.apache.tika.parser.enricher.CompositeContentEnricher;
+
+public class ContentEnricherLoaderTest {
+
+ @TempDir
+ Path tmp;
+
+ private TikaLoader load(String json) throws Exception {
+ Path config = tmp.resolve("tika-config.json");
+ Files.writeString(config, json);
+ return TikaLoader.load(config);
+ }
+
+ @Test
+ public void testContentEnrichersLoadAndInject() throws Exception {
+ TikaLoader loader = load("""
+ {
+ "parsers": [ {"enriching-test-parser": {}} ],
+ "content-enrichers": [ {"test-png-enricher": {}} ]
+ }
+ """);
+
+ CompositeContentEnricher enrichers = loader.get(CompositeContentEnricher.class);
+ assertNotNull(enrichers);
+ java.util.List<Parser> matched = enrichers.getEnrichers(MediaType.image("png"));
+ assertEquals(1, matched.size());
+ assertTrue(matched.get(0) instanceof TestPngEnricher,
+ "expected TestPngEnricher, got " + matched.get(0));
+ assertEquals(1, enrichers.getSupportedTypes().size());
+
+ EnrichingTestParser enrichingParser = findEnrichingParser(loader.get(Parser.class));
+ assertNotNull(enrichingParser, "enriching-test-parser not found in loaded parsers");
+ assertNotNull(enrichingParser.getContentEnrichers(),
+ "content enrichers were not injected into the EnrichingParser");
+ assertEquals(enrichers, enrichingParser.getContentEnrichers());
+ }
+
+ @Test
+ public void testNoContentEnrichersConfigured() throws Exception {
+ TikaLoader loader = load("""
+ {
+ "parsers": [ {"enriching-test-parser": {}} ]
+ }
+ """);
+ assertNull(loader.get(CompositeContentEnricher.class));
+ EnrichingTestParser enrichingParser = findEnrichingParser(loader.get(Parser.class));
+ assertNotNull(enrichingParser);
+ assertNull(enrichingParser.getContentEnrichers());
+ }
+
+ private EnrichingTestParser findEnrichingParser(Parser parser) {
+ if (parser instanceof EnrichingTestParser dtp) {
+ return dtp;
+ }
+ if (parser instanceof CompositeParser cp) {
+ for (Parser child : cp.getAllComponentParsers()) {
+ EnrichingTestParser found = findEnrichingParser(child);
+ if (found != null) {
+ return found;
+ }
+ }
+ }
+ return null;
+ }
+}
diff --git a/tika-serialization/src/test/java/org/apache/tika/config/loader/EnrichingTestParser.java b/tika-serialization/src/test/java/org/apache/tika/config/loader/EnrichingTestParser.java
new file mode 100644
index 0000000000..f3cbc2fe8b
--- /dev/null
+++ b/tika-serialization/src/test/java/org/apache/tika/config/loader/EnrichingTestParser.java
@@ -0,0 +1,63 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.config.loader;
+
+import java.util.Collections;
+import java.util.Set;
+
+import org.xml.sax.ContentHandler;
+
+import org.apache.tika.annotation.TikaComponent;
+import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.mime.MediaType;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.Parser;
+import org.apache.tika.parser.enricher.CompositeContentEnricher;
+import org.apache.tika.parser.enricher.EnrichingParser;
+
+/**
+ * Fixture for asserting that ParserLoader injects the configured content enrichers
+ * into parsers implementing {@link EnrichingParser}.
+ */
+@TikaComponent(name = "enriching-test-parser", spi = false)
+public class EnrichingTestParser implements Parser, EnrichingParser {
+
+ private static final long serialVersionUID = 1L;
+
+ private transient CompositeContentEnricher contentEnrichers;
+
+ @Override
+ public void setContentEnrichers(CompositeContentEnricher contentEnrichers) {
+ this.contentEnrichers = contentEnrichers;
+ }
+
+ public CompositeContentEnricher getContentEnrichers() {
+ return contentEnrichers;
+ }
+
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return Collections.singleton(MediaType.parse("application/test+deriving"));
+ }
+
+ @Override
+ public void parse(TikaInputStream stream, ContentHandler handler, Metadata metadata,
+ ParseContext context) {
+ metadata.set("parser-type", "deriving");
+ }
+}
diff --git a/tika-serialization/src/test/java/org/apache/tika/config/loader/TestPngEnricher.java b/tika-serialization/src/test/java/org/apache/tika/config/loader/TestPngEnricher.java
new file mode 100644
index 0000000000..6dee97ef65
--- /dev/null
+++ b/tika-serialization/src/test/java/org/apache/tika/config/loader/TestPngEnricher.java
@@ -0,0 +1,50 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.config.loader;
+
+import java.util.Collections;
+import java.util.Set;
+
+import org.xml.sax.ContentHandler;
+
+import org.apache.tika.annotation.TikaComponent;
+import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.mime.MediaType;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.Parser;
+
+/**
+ * Content-enricher fixture: an ordinary parser advertising a real media type,
+ * selectable by name in the "content-enrichers" list.
+ */
+@TikaComponent(name = "test-png-enricher", spi = false)
+public class TestPngEnricher implements Parser {
+
+ private static final long serialVersionUID = 1L;
+
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return Collections.singleton(MediaType.image("png"));
+ }
+
+ @Override
+ public void parse(TikaInputStream stream, ContentHandler handler, Metadata metadata,
+ ParseContext context) {
+ metadata.set("derived-by", "test-png-enricher");
+ }
+}
(tika) 03/04: TIKA-4872: review fixes
Posted by ta...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
tballison pushed a commit to branch TIKA-4872
in repository https://gitbox.apache.org/repos/asf/tika.git
commit 8c20152d6a996d359025b8d7678f9a933554f2b7
Author: tallison <ta...@apache.org>
AuthorDate: Wed Sep 2 15:43:11 2026 -0400
TIKA-4872: review fixes
---
CHANGES.txt | 11 ++-
docs/modules/ROOT/pages/configuration/index.adoc | 5 +-
.../parser/enricher/CompositeContentEnricher.java | 8 +-
.../tika/parser/enricher/ContentEnrichers.java | 56 +++++++++----
.../parser/enricher/LegacyDispatchEnricher.java | 17 ++--
.../tika/parser/enricher/ContentEnrichersTest.java | 92 ++++++++++++++++++++++
.../tika/parser/image/AbstractImageParser.java | 5 --
.../apache/tika/parser/pdf/AbstractPDF2XHTML.java | 6 +-
.../tika/config/loader/ContentEnricherLoader.java | 15 +++-
.../apache/tika/config/loader/ParserLoader.java | 8 +-
.../config/loader/ContentEnricherLoaderTest.java | 17 ++++
.../config/loader/TestUnavailableEnricher.java | 49 ++++++++++++
12 files changed, 249 insertions(+), 40 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 782bf4700d..fb54122b1e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -12,9 +12,14 @@ Release 4.1.0 - unreleased
Every enricher matching a media type runs, in config order (e.g. an
OCR engine then a VLM tagger for the same image), best-effort: one
enricher's failure does not stop the others and is still reported;
- timeouts abort the chain. With no "content-enrichers" configured, the
- legacy ocr-* dispatch applies unchanged; a WARN at config load now
- names colliding OCR engines and the winner. TesseractOCRParser's
+ timeouts abort the chain. The list is authoritative: a media type no
+ configured enricher matches gets no enrichment -- never a classpath
+ engine that was not named -- and a named engine that reports no media
+ types at load (missing binary, unreachable inference server) fails
+ config load instead of going silently inert. With no
+ "content-enrichers" configured, the legacy ocr-* dispatch applies
+ unchanged; a WARN at config load now names colliding OCR engines and
+ the winner. TesseractOCRParser's
component name is pinned as "tesseract-ocr-parser".
* Inference/OCR hardening (TIKA-4871): OpenAIVLMParser no longer
diff --git a/docs/modules/ROOT/pages/configuration/index.adoc b/docs/modules/ROOT/pages/configuration/index.adoc
index c2804aa51d..8a1c25f468 100644
--- a/docs/modules/ROOT/pages/configuration/index.adoc
+++ b/docs/modules/ROOT/pages/configuration/index.adoc
@@ -146,7 +146,10 @@ parser registered for those types: `image-parser` still parses the image and cal
(The bundled OCR engines still advertise legacy `image/ocr-*` types; those are mapped to the
real type, so naming them here just works.) *Every* enricher matching a media type runs, in the
order listed — so an OCR engine followed by a VLM that tags images is two entries, both invoked
-per image. Failures are best-effort: one enricher failing does not stop the others, and every
+per image. The list is authoritative: a media type no configured enricher matches gets no
+enrichment — never a classpath engine you did not name — and a named engine that reports no
+media types at startup (missing native binary, unreachable inference server) fails config load
+rather than going silently inert. Failures are best-effort: one enricher failing does not stop the others, and every
failure is still reported through the parser's normal exception handling (timeouts abort the
chain immediately). With no `content-enrichers` configured, behavior is unchanged — whichever
OCR engine is on the classpath is used, exactly as before, and a WARN is logged at startup when
diff --git a/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java b/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java
index db614d3118..767ab9d3cf 100644
--- a/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java
+++ b/tika-core/src/main/java/org/apache/tika/parser/enricher/CompositeContentEnricher.java
@@ -63,7 +63,7 @@ public class CompositeContentEnricher implements Serializable {
for (MediaType mediaType : enricher.getSupportedTypes(empty)) {
// legacy engines (Tesseract, VLM, ...) still advertise the image/ocr-*
// pseudo-types; key them under the real type so they are nameable here
- MediaType keyType = stripLegacyOcrPrefix(mediaType);
+ MediaType keyType = stripLegacyOcrPrefix(mediaType.getBaseType());
List<Parser> forType = tmp.computeIfAbsent(keyType, k -> new ArrayList<>());
if (!forType.contains(enricher)) {
forType.add(enricher);
@@ -84,11 +84,11 @@ public class CompositeContentEnricher implements Serializable {
}
/**
- * @return the enrichers configured for this exact media type, in config order;
- * empty when none
+ * @return the enrichers configured for this media type (parameters ignored; alias
+ * normalization is the caller's job), in config order; empty when none
*/
public List<Parser> getEnrichers(MediaType mediaType) {
- List<Parser> enrichers = enricherMap.get(mediaType);
+ List<Parser> enrichers = enricherMap.get(mediaType.getBaseType());
return enrichers == null ? Collections.emptyList() : enrichers;
}
diff --git a/tika-core/src/main/java/org/apache/tika/parser/enricher/ContentEnrichers.java b/tika-core/src/main/java/org/apache/tika/parser/enricher/ContentEnrichers.java
index f2177c75e0..81abc69863 100644
--- a/tika-core/src/main/java/org/apache/tika/parser/enricher/ContentEnrichers.java
+++ b/tika-core/src/main/java/org/apache/tika/parser/enricher/ContentEnrichers.java
@@ -28,6 +28,7 @@ import org.apache.tika.exception.TikaException;
import org.apache.tika.exception.TikaTimeoutException;
import org.apache.tika.extractor.EmbeddedDocumentUtil;
import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.HttpHeaders;
import org.apache.tika.metadata.Metadata;
import org.apache.tika.mime.MediaType;
import org.apache.tika.parser.ParseContext;
@@ -48,6 +49,9 @@ import org.apache.tika.parser.Parser;
* must still fire the enricher selected for the type it was dispatched on).</li>
* <li>The enricher writes into the caller's {@link Metadata}; the caller must not assume
* the metadata is untouched beyond the derived content.</li>
+ * <li>An enricher that re-enters parsing must propagate the caller's
+ * {@link ParseContext}: the recursion guard (like Tika's other in-parse limits)
+ * rides the context, so a fresh context defeats it.</li>
* </ul>
*
* @since Apache Tika 4.1
@@ -59,12 +63,14 @@ public final class ContentEnrichers {
/**
* Returns the enricher to invoke for one media type, or null when none applies.
- * Explicitly configured enrichers win: every one matching the type runs, in config
- * order, behind the single Parser returned here. Otherwise the legacy
- * {@code image/ocr-*} dispatch through the composite parser applies when an engine
- * claims the synthetic type. Returns null while a enrichment is already in progress
- * in this context, so an enricher that is (or invokes) a container parser cannot
- * recurse into enrichment.
+ * A configured {@code "content-enrichers"} list is authoritative: every configured
+ * enricher matching the type runs, in config order, behind the single Parser
+ * returned here, and a type no configured enricher matches gets no enrichment --
+ * never a classpath engine the user did not name. Only when no list is configured
+ * at all does the legacy {@code image/ocr-*} dispatch through the composite parser
+ * apply. Returns null while an enrichment is already in progress in this context,
+ * so an enricher that is (or invokes) a container parser cannot recurse into
+ * enrichment.
*
* @param enrichers the injected composite; may be null when none is configured
* @param mediaType the real, normalized media type of the bytes; may be null
@@ -81,15 +87,16 @@ public final class ContentEnrichers {
}
if (enrichers != null) {
List<Parser> matched = enrichers.getEnrichers(mediaType);
- if (!matched.isEmpty()) {
- return new GuardedEnricher(matched.size() == 1
- ? matched.get(0) : new SequentialEnricher(matched));
+ if (matched.isEmpty()) {
+ return null;
}
+ return new GuardedEnricher(matched.size() == 1
+ ? matched.get(0) : new SequentialEnricher(matched));
}
Parser composite = EmbeddedDocumentUtil.getStatelessParser(context);
if (composite != null && composite.getSupportedTypes(context)
.contains(LegacyDispatchEnricher.toOcrMediaType(mediaType))) {
- return new GuardedEnricher(new LegacyDispatchEnricher(mediaType));
+ return new GuardedEnricher(new LegacyDispatchEnricher(mediaType, composite));
}
return null;
}
@@ -98,9 +105,10 @@ public final class ContentEnrichers {
* Runs each enricher in config order, best-effort: one enricher's failure does not
* stop the others. The first failure is rethrown after the chain completes, with
* later failures attached as suppressed, so call sites report every failure through
- * their existing exception handling. Timeouts, SecurityException and SAXException
- * (incl. write-limit aborts) propagate immediately -- a spent budget or a suspect
- * handler must not fund further enrichments.
+ * their existing exception handling. Timeouts, SecurityException, SAXException
+ * (incl. write-limit aborts) and other runtime exceptions propagate immediately --
+ * a spent budget or a suspect handler must not fund further enrichments -- with any
+ * earlier recorded failure attached as suppressed.
*/
private static final class SequentialEnricher implements Parser {
@@ -137,6 +145,11 @@ public final class ContentEnrichers {
} else {
first.addSuppressed(e);
}
+ } catch (RuntimeException e) {
+ if (first != null) {
+ e.addSuppressed(first);
+ }
+ throw e;
}
}
if (first instanceof IOException e) {
@@ -153,7 +166,11 @@ public final class ContentEnrichers {
boolean active;
}
- /** Marks enrichment in progress around the delegate so {@link #get} refuses re-entry. */
+ /**
+ * Marks enrichment in progress around the delegate so {@link #get} refuses re-entry,
+ * and restores Content-Type afterwards: an enricher derives content, it does not get
+ * to re-type the caller's document.
+ */
private static final class GuardedEnricher implements Parser {
private static final long serialVersionUID = 1L;
@@ -177,11 +194,20 @@ public final class ContentEnrichers {
active = new ActiveEnrichment();
context.set(ActiveEnrichment.class, active);
}
+ String contentType = metadata.get(HttpHeaders.CONTENT_TYPE);
+ // restore rather than clear: a nested invocation must not strip the
+ // outer enrichment's re-entry protection when it completes
+ boolean wasActive = active.active;
active.active = true;
try {
delegate.parse(tis, handler, metadata, context);
} finally {
- active.active = false;
+ active.active = wasActive;
+ if (contentType == null) {
+ metadata.remove(HttpHeaders.CONTENT_TYPE);
+ } else {
+ metadata.set(HttpHeaders.CONTENT_TYPE, contentType);
+ }
}
}
}
diff --git a/tika-core/src/main/java/org/apache/tika/parser/enricher/LegacyDispatchEnricher.java b/tika-core/src/main/java/org/apache/tika/parser/enricher/LegacyDispatchEnricher.java
index 7b40938fb5..f880250e2a 100644
--- a/tika-core/src/main/java/org/apache/tika/parser/enricher/LegacyDispatchEnricher.java
+++ b/tika-core/src/main/java/org/apache/tika/parser/enricher/LegacyDispatchEnricher.java
@@ -24,7 +24,6 @@ import org.xml.sax.ContentHandler;
import org.xml.sax.SAXException;
import org.apache.tika.exception.TikaException;
-import org.apache.tika.extractor.EmbeddedDocumentUtil;
import org.apache.tika.io.TikaInputStream;
import org.apache.tika.metadata.HttpHeaders;
import org.apache.tika.metadata.Metadata;
@@ -34,8 +33,8 @@ import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
/**
- * Reproduces the pre-4.1 {@code image/ocr-*} dispatch when no enricher is configured for
- * a media type: mints the synthetic {@code ocr-} media type, sets
+ * Reproduces the pre-4.1 {@code image/ocr-*} dispatch when no {@code "content-enrichers"}
+ * list is configured: mints the synthetic {@code ocr-} media type, sets
* {@link TikaCoreProperties#CONTENT_TYPE_PARSER_OVERRIDE} and re-enters the composite
* parser, restoring the metadata afterwards. Whichever engine won the {@code ocr-*}
* registration in the composite still wins here, so precedence-by-presence (adding
@@ -55,11 +54,17 @@ public class LegacyDispatchEnricher implements Parser {
private final MediaType mediaType;
+ private final Parser composite;
+
/**
* @param mediaType the real (already normalized) media type of the bytes to derive from
+ * @param composite the composite parser to re-enter; the caller has already verified
+ * it claims the synthetic {@code ocr-} type (re-verifying here would
+ * rebuild the composite's full supported-types map per invocation)
*/
- public LegacyDispatchEnricher(MediaType mediaType) {
+ public LegacyDispatchEnricher(MediaType mediaType, Parser composite) {
this.mediaType = mediaType;
+ this.composite = composite;
}
/**
@@ -81,9 +86,7 @@ public class LegacyDispatchEnricher implements Parser {
public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
ParseContext context) throws IOException, SAXException, TikaException {
MediaType ocrMediaType = toOcrMediaType(mediaType);
- Parser composite = EmbeddedDocumentUtil.getStatelessParser(context);
- if (composite == null
- || !composite.getSupportedTypes(context).contains(ocrMediaType)) {
+ if (composite == null) {
throw new TikaException("No parser is registered for " + ocrMediaType);
}
String originalOverride = metadata.get(TikaCoreProperties.CONTENT_TYPE_PARSER_OVERRIDE);
diff --git a/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java b/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java
index 8f2dbdc22f..884a9972b5 100644
--- a/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java
+++ b/tika-core/src/test/java/org/apache/tika/parser/enricher/ContentEnrichersTest.java
@@ -122,6 +122,98 @@ public class ContentEnrichersTest {
assertNull(ContentEnrichers.get(null, null, context));
}
+ @Test
+ public void testConfiguredListIsAuthoritative() throws Exception {
+ // the composite claims ocr-tiff, but a configured list that doesn't cover tiff
+ // must yield no enricher -- never a classpath engine the user did not name
+ RecordingParser explicit = new RecordingParser(Collections.singleton(PNG));
+ RecordingParser composite =
+ new RecordingParser(Collections.singleton(MediaType.image("ocr-tiff")));
+ CompositeContentEnricher enrichers = new CompositeContentEnricher(List.of(explicit));
+ ParseContext context = new ParseContext();
+ context.set(Parser.class, composite);
+
+ assertNull(ContentEnrichers.get(enrichers, MediaType.image("tiff"), context));
+ // with no list configured, the same composite is reachable via legacy dispatch
+ assertNotNull(ContentEnrichers.get(null, MediaType.image("tiff"), context));
+ }
+
+ @Test
+ public void testParametersIgnoredInMatching() throws Exception {
+ RecordingParser explicit = new RecordingParser(Collections.singleton(PNG));
+ CompositeContentEnricher enrichers = new CompositeContentEnricher(List.of(explicit));
+ ParseContext context = new ParseContext();
+
+ Parser enricher = ContentEnrichers.get(enrichers,
+ MediaType.parse("image/png; charset=binary"), context);
+ assertNotNull(enricher, "parameterized type must match the base-type registration");
+ invoke(enricher, new Metadata(), context);
+ assertEquals(1, explicit.calls);
+ }
+
+ @Test
+ public void testEnricherCannotRewriteContentType() throws Exception {
+ Parser rewriting = new Parser() {
+ private static final long serialVersionUID = 1L;
+
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return Collections.singleton(PNG);
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
+ ParseContext context) {
+ metadata.set(HttpHeaders.CONTENT_TYPE, "application/pdf");
+ }
+ };
+ CompositeContentEnricher enrichers = new CompositeContentEnricher(List.of(rewriting));
+ ParseContext context = new ParseContext();
+
+ Metadata metadata = new Metadata();
+ metadata.set(HttpHeaders.CONTENT_TYPE, PNG.toString());
+ invoke(ContentEnrichers.get(enrichers, PNG, context), metadata, context);
+ assertEquals(PNG.toString(), metadata.get(HttpHeaders.CONTENT_TYPE));
+
+ Metadata unset = new Metadata();
+ invoke(ContentEnrichers.get(enrichers, PNG, context), unset, context);
+ assertNull(unset.get(HttpHeaders.CONTENT_TYPE));
+ }
+
+ @Test
+ public void testRuntimeFailureAbortsChainWithEarlierFailureSuppressed() throws Exception {
+ List<String> order = new java.util.ArrayList<>();
+ Parser failing = namedEnricher("failing", order, true);
+ Parser blowingUp = new Parser() {
+ private static final long serialVersionUID = 1L;
+
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return Collections.singleton(PNG);
+ }
+
+ @Override
+ public void parse(TikaInputStream tis, ContentHandler handler, Metadata metadata,
+ ParseContext context) {
+ order.add("blowingUp");
+ throw new NullPointerException("boom");
+ }
+ };
+ Parser third = namedEnricher("third", order, false);
+ CompositeContentEnricher enrichers =
+ new CompositeContentEnricher(List.of(failing, blowingUp, third));
+ ParseContext context = new ParseContext();
+
+ Parser enricher = ContentEnrichers.get(enrichers, PNG, context);
+ assertNotNull(enricher);
+ NullPointerException thrown = org.junit.jupiter.api.Assertions.assertThrows(
+ NullPointerException.class, () -> invoke(enricher, new Metadata(), context));
+ assertEquals(List.of("failing", "blowingUp"), order);
+ // the recorded checked failure rides along instead of vanishing
+ assertEquals(1, thrown.getSuppressed().length);
+ assertEquals("failing failed", thrown.getSuppressed()[0].getMessage());
+ }
+
@Test
public void testAllMatchingEnrichersRunInOrder() throws Exception {
List<String> order = new java.util.ArrayList<>();
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/main/java/org/apache/tika/parser/image/AbstractImageParser.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/main/java/org/apache/tika/parser/image/AbstractImageParser.java
index 2e53f2a2c6..76f9af40e0 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/main/java/org/apache/tika/parser/image/AbstractImageParser.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-image-module/src/main/java/org/apache/tika/parser/image/AbstractImageParser.java
@@ -34,17 +34,12 @@ import org.apache.tika.parser.Parser;
import org.apache.tika.parser.enricher.CompositeContentEnricher;
import org.apache.tika.parser.enricher.ContentEnrichers;
import org.apache.tika.parser.enricher.EnrichingParser;
-import org.apache.tika.parser.enricher.LegacyDispatchEnricher;
import org.apache.tika.sax.BodyContentHandler;
import org.apache.tika.sax.EmbeddedContentHandler;
import org.apache.tika.sax.XHTMLContentHandler;
public abstract class AbstractImageParser implements Parser, EnrichingParser {
- /** @deprecated use {@link LegacyDispatchEnricher#OCR_MEDIATYPE_PREFIX} */
- @Deprecated
- public static String OCR_MEDIATYPE_PREFIX = LegacyDispatchEnricher.OCR_MEDIATYPE_PREFIX;
-
private CompositeContentEnricher contentEnrichers;
abstract void extractMetadata(InputStream is, ContentHandler contentHandler, Metadata metadata,
diff --git a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java
index b7cde47b4a..7502d29c16 100644
--- a/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java
+++ b/tika-parsers/tika-parsers-standard/tika-parsers-standard-modules/tika-parser-pdf-module/src/main/java/org/apache/tika/parser/pdf/AbstractPDF2XHTML.java
@@ -576,8 +576,10 @@ class AbstractPDF2XHTML extends PDFTextStripper {
if (ocrStrategy == OCR_ONLY || ocrStrategy == OCR_AND_TEXT_EXTRACTION) {
throw new TikaException(
"I regret that I couldn't find an OCR engine to handle " +
- imageMediaType + ". Configure one in \"content-enrichers\", " +
- "add one to the classpath, or set the OCR strategy to NO_OCR.");
+ imageMediaType + ". Name one that covers it in " +
+ "\"content-enrichers\" (a configured list is authoritative), " +
+ "add one to the classpath when no list is configured, " +
+ "or set the OCR strategy to NO_OCR.");
} else if (ocrStrategy == AUTO) {
//silently skip if there's no engine to run ocr
return;
diff --git a/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java b/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java
index 56330a790f..98aeb535cf 100644
--- a/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java
+++ b/tika-serialization/src/main/java/org/apache/tika/config/loader/ContentEnricherLoader.java
@@ -24,6 +24,7 @@ import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import org.apache.tika.exception.TikaConfigException;
+import org.apache.tika.parser.ParseContext;
import org.apache.tika.parser.Parser;
import org.apache.tika.parser.enricher.CompositeContentEnricher;
@@ -43,15 +44,27 @@ class ContentEnricherLoader implements ComponentLoader<CompositeContentEnricher>
return null;
}
List<Parser> enrichers = new ArrayList<>();
+ ParseContext empty = new ParseContext();
for (Map.Entry<String, JsonNode> entry : entries) {
+ Parser enricher;
try {
ObjectNode wrapper = context.getObjectMapper().createObjectNode();
wrapper.set(entry.getKey(), entry.getValue());
- enrichers.add(context.getObjectMapper().treeToValue(wrapper, Parser.class));
+ enricher = context.getObjectMapper().treeToValue(wrapper, Parser.class);
} catch (Exception e) {
throw new TikaConfigException(
"Failed to load content enricher: " + entry.getKey(), e);
}
+ // engines report no types when unusable (missing binary, unreachable server);
+ // the media-type snapshot taken here lasts the life of the process, so an
+ // explicitly named engine that can never run must fail load, not go silent
+ if (enricher.getSupportedTypes(empty).isEmpty()) {
+ throw new TikaConfigException("Content enricher \"" + entry.getKey()
+ + "\" advertises no media types. Is the engine unavailable "
+ + "(missing native binary, unreachable inference server) or "
+ + "configured to skip enrichment?");
+ }
+ enrichers.add(enricher);
}
return new CompositeContentEnricher(enrichers);
}
diff --git a/tika-serialization/src/main/java/org/apache/tika/config/loader/ParserLoader.java b/tika-serialization/src/main/java/org/apache/tika/config/loader/ParserLoader.java
index 5e44cf5cf7..9fb1cdbf64 100644
--- a/tika-serialization/src/main/java/org/apache/tika/config/loader/ParserLoader.java
+++ b/tika-serialization/src/main/java/org/apache/tika/config/loader/ParserLoader.java
@@ -138,7 +138,9 @@ public class ParserLoader extends AbstractSpiComponentLoader<Parser> {
Renderer renderer = context.getRenderer();
CompositeContentEnricher contentEnrichers = context.getContentEnrichers();
injectDependenciesRecursively(parser, encodingDetector, renderer, contentEnrichers);
- warnOnAmbiguousOcrRegistrations(parser);
+ if (contentEnrichers == null) {
+ warnOnAmbiguousOcrRegistrations(parser);
+ }
return parser;
}
@@ -171,7 +173,9 @@ public class ParserLoader extends AbstractSpiComponentLoader<Parser> {
* The image/ocr-* pseudo-types are claimed by several OCR engines whose availability
* is environmental, and the composite resolves a collision by last registration with
* no warning. Name the collision and the winner once at load so engine selection is
- * debuggable; select an engine explicitly with "content-enrichers".
+ * debuggable; select an engine explicitly with "content-enrichers". Skipped when
+ * content-enrichers is configured: the list is authoritative and legacy ocr-*
+ * dispatch never runs, so the collision is moot and the advice already taken.
*/
private void warnOnAmbiguousOcrRegistrations(Parser parser) {
if (!(parser instanceof CompositeParser cp)) {
diff --git a/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java b/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java
index 46fcfeff84..372987f348 100644
--- a/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java
+++ b/tika-serialization/src/test/java/org/apache/tika/config/loader/ContentEnricherLoaderTest.java
@@ -67,6 +67,23 @@ public class ContentEnricherLoaderTest {
assertEquals(enrichers, enrichingParser.getContentEnrichers());
}
+ @Test
+ public void testZeroTypeEnricherFailsLoad() throws Exception {
+ // an explicitly named engine that cannot run (missing binary, unreachable
+ // server) must fail config load, not become a silent no-op
+ TikaLoader loader = load("""
+ {
+ "content-enrichers": [ {"test-unavailable-enricher": {}} ]
+ }
+ """);
+ org.apache.tika.exception.TikaConfigException e =
+ org.junit.jupiter.api.Assertions.assertThrows(
+ org.apache.tika.exception.TikaConfigException.class,
+ () -> loader.get(CompositeContentEnricher.class));
+ assertTrue(e.getMessage().contains("advertises no media types"),
+ "unexpected message: " + e.getMessage());
+ }
+
@Test
public void testNoContentEnrichersConfigured() throws Exception {
TikaLoader loader = load("""
diff --git a/tika-serialization/src/test/java/org/apache/tika/config/loader/TestUnavailableEnricher.java b/tika-serialization/src/test/java/org/apache/tika/config/loader/TestUnavailableEnricher.java
new file mode 100644
index 0000000000..4f3f28d02e
--- /dev/null
+++ b/tika-serialization/src/test/java/org/apache/tika/config/loader/TestUnavailableEnricher.java
@@ -0,0 +1,49 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.tika.config.loader;
+
+import java.util.Collections;
+import java.util.Set;
+
+import org.xml.sax.ContentHandler;
+
+import org.apache.tika.annotation.TikaComponent;
+import org.apache.tika.io.TikaInputStream;
+import org.apache.tika.metadata.Metadata;
+import org.apache.tika.mime.MediaType;
+import org.apache.tika.parser.ParseContext;
+import org.apache.tika.parser.Parser;
+
+/**
+ * Content-enricher fixture mimicking an engine that is unusable at load time
+ * (missing binary, unreachable server): it advertises no media types.
+ */
+@TikaComponent(name = "test-unavailable-enricher", spi = false)
+public class TestUnavailableEnricher implements Parser {
+
+ private static final long serialVersionUID = 1L;
+
+ @Override
+ public Set<MediaType> getSupportedTypes(ParseContext context) {
+ return Collections.emptySet();
+ }
+
+ @Override
+ public void parse(TikaInputStream stream, ContentHandler handler, Metadata metadata,
+ ParseContext context) {
+ }
+}