Re: [I] Bump `elasticsearch-java` to version 9.x [logging-log4j2]

["ramanathan1504 (via GitHub)" <gi...@apache.org>]

ramanathan1504 commented on issue #3663:
URL: https://github.com/apache/logging-log4j2/issues/3663#issuecomment-3217865814

   # Proposal: Upgrade elasticsearch-java Dependency to 9.x in Log4j2
   
   ## Background
   
   The current Log4j2 integration with Elasticsearch relies on `elasticsearch-java` version 8.x. With the release of version 9.x, the Elasticsearch Java client introduces significant API changes and enhancements that require explicit handling in the Log4j2 Elasticsearch appender.
   
   ## Motivation
   
   *   **Compatibility:** Ensure Log4j2 remains compatible with modern Elasticsearch clusters that require or benefit from the 9.x client.
   *   **Modern Features:** Leverage improvements in performance, security, and the more modern, maintained APIs available in the new client.
   *   **Reduce Technical Debt:** Proactively adapt to new versions of core dependencies to avoid the challenges of relying on outdated and potentially unsupported client versions.
   
   ## Challenges
   
   *   **Breaking Changes:** The 9.x client is not backward-compatible with the 8.x client. This includes changes to API contracts, method signatures, and builder patterns.
   *   **Backward Compatibility:** A key challenge is to manage the transition without breaking existing setups for users who are still on older versions of the Elasticsearch client.
   *   **Testing Overhead:** Additional integration testing will be required to validate that Log4j2 appenders function correctly with both old and new client versions against corresponding Elasticsearch clusters.
   
   ## Proposed Approach
   
   ### 1. Analysis Phase
   
   *   **Identify Breaking Changes:** Conduct a thorough analysis of the differences between `elasticsearch-java` 8.x and 9.x, focusing on areas directly impacting the Log4j2 appender (e.g., request/response handling, transport layer changes, authentication).
   *   **Map API Usage:** Map the current implementation of the Elasticsearch appenders to the new APIs and patterns introduced in the 9.x client.
   
   ### 2. Implementation Options
   
   *   **Option A: Direct Upgrade**
       *   Update the Log4j2 appenders to fully support the 9.x client.
       *   This would mean dropping support for the 8.x client entirely in a future major release of Log4j2.
       *   **Pros:** Simpler implementation.
       *   **Cons:** Disruptive for users unable to upgrade their Elasticsearch client immediately.
   
   *   **Option B: Dual Support Layer (Recommended)**
       *   Introduce an abstraction layer over the Elasticsearch client interactions within Log4j2.
       *   Create two separate implementation modules or adapters: one for the 8.x client and one for the 9.x client.
       *   The correct implementation would be chosen at runtime based on the `elasticsearch-java` version available on the classpath.
       *   **Pros:** Ensures a smoother migration path for all users and maintains backward compatibility.
       *   **Cons:** More complex to implement and maintain initially.
   
   ### 3. Testing Strategy
   
   *   **Expand Integration Tests:** Add new integration test suites that run against both Elasticsearch 8.x and 9.x clusters.
   *   **Validate Core Functionality:** Verify that logging events are correctly created, sent, and indexed in both scenarios.
   *   **Test Edge Cases:** Validate error handling, bulk request processing, and various authentication mechanisms (e.g., API Key, Basic Auth) for both client versions.
   
   ### 4. Documentation Updates
   
   *   **Update Log4j2 Manual:** Clearly document which versions of the `elasticsearch-java` client are supported.
   *   **Provide Migration Guide:** Create clear migration notes for users, explaining how to configure the appender when upgrading from `elasticsearch-java` 8.x to 9.x. This should include dependency changes and any necessary configuration adjustments.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org