Re: [I] meta server does not work [hudi]

["rangareddy (via GitHub)" <gi...@apache.org>]

rangareddy commented on issue #16876:
URL: https://github.com/apache/hudi/issues/16876#issuecomment-5177154491

   I looked into this against current master, and **I do not think it should be closed** — the report is accurate, the bug is unfixed, and the CI that used to cover it no longer does. Recording what I found.
   
   ### The metaserver options were removed from Hudi's own bundle validation
   
   The diff in the description is not a proposal; it is what master now looks like. `packaging/bundle-validation/service/write.scala` and `read.scala` no longer set `hoodie.metaserver.enabled` or `hoodie.metaserver.uris` at all:
   
   ```bash
   $ grep -n metaserver packaging/bundle-validation/service/write.scala packaging/bundle-validation/service/read.scala
   packaging/bundle-validation/service/read.scala:25:  load(basePath).coalesce(1).write.csv("/tmp/metaserver-bundle/...")
   ```
   
   Removed by `d3d83020b77e`, **[HUDI-8992] "Deprecate all byte array usage in metadata deserialization path" (#12826)** — a 141-file change whose subject has nothing to do with the metaserver. The four option lines went with it.
   
   The consequence is that `test_metaserver_bundle` in `validate.sh` still starts `metaserver.jar` and still runs both Spark scripts, but since those scripts no longer enable the metaserver, the stage now validates a **plain filesystem-backed write and read that happens to have a metaserver process running alongside it**. It asserts 10 records and passes. Nothing in it exercises the metaserver as a metastore any more.
   
   So the feature is not covered by CI, which is consistent with it being broken and staying broken: `git log --since=2025-11 --grep=metaserver -i` turns up no fix.
   
   ### One half of the report is by design, though
   
   The `completionTime` being null is expected for metaserver instants, and Hudi already compensates. Both timeline implementations carry the same fallback:
   
   ```java
   // CompletionTimeQueryViewV2.java:312 (and V1:230)
   if (completionTime == null) {
     // the meta-server instant does not have completion time
     completionTime = beginInstantTime;
   }
   ```
   
   So a null there is not itself the defect — it is handled, and the instant time is substituted. The defect is the first half: the write not landing.
   
   ### Worth knowing for anyone reproducing
   
   `hudi-metaserver` lives under `hudi-platform-service`, which is behind a profile activated only by `deployArtifacts=true`:
   
   ```xml
   <id>hudi-platform-service</id>
   <activation><property><name>deployArtifacts</name><value>true</value></property></activation>
   ```
   
   So the module is not built by a default `mvn install`. Reproducing needs `-Phudi-platform-service` (or `-DdeployArtifacts=true`) **and** the runtime configs — two separate opt-ins, which is probably part of why this has gone unnoticed.
   
   ### Suggested next step
   
   Rather than closing, I would restore the two options in the bundle-validation scripts and let CI show the failure. That turns this from a report into a reproducible red build, which is what the fix needs — and it also answers whether the breakage is in the metaserver itself or in the timeline/deserialisation change that removed the options.
   
   I have not made that change, since re-enabling a known-broken path in CI is a maintainer's call: it would go red until the underlying bug is fixed. Happy to raise it if that is wanted, either as a PR or as a separate issue for the coverage gap.
   
   cc @danny0405 @zhangyue19921010 — you have both touched this area; is the metaserver still intended to be a supported path, or should it be deprecated rather than fixed? That would decide whether restoring the validation is worth it.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org