[jira] [Created] (TIKA-4815) tika-grpc: all parse requests share one single-threaded PipesClient

["Davide Polato (Jira)" <ji...@apache.org>]

Davide Polato created TIKA-4815:
-----------------------------------

             Summary: tika-grpc: all parse requests share one single-threaded PipesClient
                 Key: TIKA-4815
                 URL: https://issues.apache.org/jira/browse/TIKA-4815
             Project: Tika
          Issue Type: Bug
          Components: tika-pipes
    Affects Versions: 4.0.0
            Reporter: Davide Polato
         Attachments: ConcurrencyLocalDiagnostic.java

The gRPC server creates exactly one PipesClient and every parse handler thread uses it. PipesClient's own javadoc says it is single-threaded, and nothing in TikaGrpcServerImpl synchronizes access to it. The management RPCs are not involved - this is about fetchAndParse and its streaming variants, which all funnel through the same client.

Attached is a small repro: one sequential warmup call so the forked worker is up and serving, then 4 concurrent fetchAndParse calls at an in-process server built without directExecutor (the production server uses a thread pool too). Measured on current master:

DIAG-CONC WARMUP status=PARSE_SUCCESS
DIAG-CONC OK ... status=UNSPECIFIED_CRASH
DIAG-CONC OK ... status=FAILED_TO_INITIALIZE
DIAG-CONC OK ... status=UNSPECIFIED_CRASH
DIAG-CONC OK ... status=UNSPECIFIED_CRASH
SUMMARY concurrency=4 answered=4 transportFailed=0 elapsedMs=61358
Every call got a normal gRPC OK answer - transport-level success masking application-level failure. The statuses say the worker died or never came up for the request, and four tiny HTML files took a minute. A crawler pointed at this records every page as unparseable while the server looks perfectly healthy.

Why no test caught it: TikaGrpcServerTest sends its requests sequentially and builds servers with directExecutor(), so nothing ever overlaps.

Effectively today's server has an undeclared limit of one in-flight parse. This is also what blocks cancellation (see TIKA-4795): you can't interrupt one request's parse without disturbing the worker everyone else is on.

Measured on current master (4.0.0-SNAPSHOT). The 3.x server has the same shape but I haven't run the probe there.

tika-pipes already has the right mechanism: PipesParser keeps a queue of clients sized by numClients, admission with a timeout, release in finally. tika-grpc bypasses it and builds a bare client. Moving the server onto a pool looks like most of the fix, with two open questions: what should pool exhaustion map to on the wire (I'd lean gRPC RESOURCE_EXHAUSTED - infrastructure outcomes as status, parse outcomes in the reply - but that's this project's call), and what marks a client as unreusable before pooling - PipesClient.process() rethrows InterruptedException without closing the connection, and PipesParser's finally re-offers the client unconditionally.

I can work on this - the repro converts to a regression test directly. Drop the attachment into tika-grpc/src/test/java/org/apache/tika/pipes/grpc/ and run with -Dtest=ConcurrencyLocalDiagnostic.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)