[PR] Pass jfr, gc and jit telemetry devices to the provisioned pipelines [solr-orbit]

["syahra712 (via GitHub)" <gi...@apache.org>]

syahra712 opened a new pull request, #143:
URL: https://github.com/apache/solr-orbit/pull/143

   ### Description
   `--telemetry jfr`, `gc` and `jit` did nothing under `run`. `SolrProvisioner` already appends each device's `instrument_java_opts()` to `SOLR_OPTS`, but `solr_from_distribution` and `solr_from_sources` never passed it any `telemetry_devices`.
   
   This PR adds `_create_jvm_telemetry_devices(cfg)` to `test_run_orchestrator.py`. It builds `FlightRecorder`, `Gc` and `JitCompiler` from `telemetry.devices` / `telemetry.params` and returns only the enabled ones. It has to filter because the provisioner adds flags from every device it gets, not just the enabled ones. Both pipelines now pass the result to `SolrProvisioner`. Output goes to `<test-run-dir>/telemetry/`, so `gc.log`, `profile.jfr` and `jit.log` sit next to `test_run.json`. That directory is also outside the install dir, which `clean()` deletes on teardown.
   
   I left the `docker` pipeline unchanged on purpose. The device flags contain host paths, and the JVM in the container exits on startup if it can't open them. With `solr:9` I get `Error opening log file '...gc.log': No such file or directory`, then `Could not create the Java Virtual Machine`. Docker needs the log-storage change discussed in #138, so it comes in the second PR, together with `heapdump`.
   
   `docs/reference/telemetry.md` now lists `from-distribution` / `from-sources` for these three devices and gives the output path. The multi-device example also uses `from-distribution` instead of `docker`.
   
   ### Issues Resolved
   Part of #138 (this covers from-distribution and from-sources; Docker and heapdump follow in a second PR)
   
   ### Testing
   - [x] New functionality includes testing
   
   New tests in `tests/test_execution_orchestrator_test.py`:
   - `test_from_distribution_passes_jvm_telemetry_devices` and `test_from_sources_passes_jvm_telemetry_devices` run each pipeline with `SolrProvisioner` mocked and check the `telemetry_devices` it receives. On `main` they fail with `KeyError: 'telemetry_devices'`.
   - `test_jvm_telemetry_devices_add_their_flags_to_solr_opts` passes the devices to a real `SolrProvisioner` and checks that `_build_env()["SOLR_OPTS"]` contains the `-Xlog:gc...:file=.../gc.log`, `StartFlightRecording...filename=.../profile.jfr` and `-XX:LogFile=.../jit.log` flags. It also checks that `gc-log-config` from `--telemetry-params` is respected.
   - Two tests for the helper itself: it returns only enabled devices (`segment-stats` in the list is ignored) and nothing when `--telemetry` is empty.
   
   The full `tests/` suite passes locally on Python 3.13 (1180 passed, 5 skipped), and so does `ruff check .`. I haven't run the integration suite (`it/`).
   
   ---
   By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org

Re: [PR] Pass jfr, gc and jit telemetry devices to the provisioned pipelines [solr-orbit]

["Copilot (via GitHub)" <gi...@apache.org>]

Copilot commented on code in PR #143:
URL: https://github.com/apache/solr-orbit/pull/143#discussion_r4382338441


##########
solrorbit/test_run_orchestrator.py:
##########
@@ -462,6 +462,25 @@ def _load_cluster_config(cfg):
     return instance
 
 
+def _create_jvm_telemetry_devices(cfg):
+    """
+    Build the JVM telemetry devices (``jfr``, ``gc``, ``jit``) enabled with ``--telemetry``.
+
+    ``SolrProvisioner`` adds the ``instrument_java_opts()`` of every device it is given to
+    ``SOLR_OPTS``, so only the enabled ones are returned. Their output goes to a
+    ``telemetry`` directory inside the test run directory.
+    """
+    enabled_devices = cfg.opts("telemetry", "devices", mandatory=False, default_value=[]) or []
+    telemetry_params = cfg.opts("telemetry", "params", mandatory=False, default_value={}) or {}
+    log_root = os.path.join(paths.test_run_root(cfg), "telemetry")
+    devices = [
+        telemetry.FlightRecorder(telemetry_params, log_root, java_major_version=None),
+        telemetry.Gc(telemetry_params, log_root, java_major_version=None),
+        telemetry.JitCompiler(log_root, telemetry_params),

Review Comment:
   Enabling `--telemetry jit` now prevents Solr from starting on supported modern JDKs: `JitCompiler.instrument_java_opts()` still emits `-XX:+TraceClassLoading`, which the JVM rejects with `Unrecognized VM option 'TraceClassLoading'`. Both provisioned pipelines now pass that flag through to the JVM. Remove the obsolete option or replace it with `-Xlog:class+load=info` in `JitCompiler` before wiring it in, and add a regression assertion that the obsolete flag is absent.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@solr.apache.org
For additional commands, e-mail: issues-help@solr.apache.org