Re: [I] [Feature] Add lineage fields to the event log for per-run trace reconstruction [flink-agents]

["zxs1633079383 (via GitHub)" <gi...@apache.org>]

zxs1633079383 commented on issue #841:
URL: https://github.com/apache/flink-agents/issues/841#issuecomment-4889908286

   I took a pass through the current event-log/runtime shape and I think this can be a narrow first PR if the lineage data stays on the log record/context rather than inside user event attributes.
   
   Current insertion-point observation:
   
   - `EventLogRecord` is already the right wire object for the log; it currently carries `EventContext` + `Event`.
   - `EventContext` currently has only `eventType` and `timestamp`.
   - `EventRouter.notifyEventProcessed(Event)` only receives the event, so it cannot know which action emitted it.
   - `RunnerContextImpl.sendEvent(Event)` does know the current `actionName`, but that metadata is lost before the event reaches the logger.
   
   My suggested first-PR shape:
   
   1. Add a small lineage/context object to `EventLogRecord` or extend `EventContext` with optional fields:
      - `logSchemaVersion`
      - `runId` / `traceId`
      - `eventId`
      - `sourceEventId` or `parentEventId`
      - `emittingAction`
      - optionally the current key/sequence number if we can expose them without leaking unstable internals
   2. Keep these fields outside `event.attributes`, so the event payload remains user-owned and cross-language JSON-compatible.
   3. Treat each input event as the root of a run: `runId = inputEvent.id`, no parent/source event, no emitting action.
   4. For events emitted from an action, set `runId` from the source event, `sourceEventId = actionTask.event.id`, and `emittingAction = action.name`.
   5. Leave reverse edges out of the log, as proposed. They can still be derived from the agent plan's trigger rules during offline reconstruction.
   
   This keeps the first implementation scoped to recording and replay/debuggability. Visualization, OTel export, and trace-tree reconstruction can be follow-ups once the log has stable causal fields.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org