[jira] [Commented] (CAMEL-21413) Create a TransactionLog transaction log to provide detailed tracking and performance metrics on end-to-end transactions

["Raymond (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CAMEL-21413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17895126#comment-17895126 ] 

Raymond commented on CAMEL-21413:
---------------------------------

As a small note, probably OpenTelemetry has the best connection to such an implementation as it can capture, process, and export telemetry data (such as metrics, traces, and logs) from Camel.

> Create a TransactionLog transaction log to provide detailed tracking and performance metrics on end-to-end transactions 
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-21413
>                 URL: https://issues.apache.org/jira/browse/CAMEL-21413
>             Project: Camel
>          Issue Type: Wish
>          Components: came-core
>            Reporter: Raymond
>            Priority: Major
>         Attachments: TransactionStore_TrackAndTrace.jpg
>
>
> *Wish*
> The wish is for a transaction log store in Camel to provide detailed tracking and performance metrics on end-to-end transactions across multiple exchanges/routes, storing metadata, exchange data, and performance metrics in a configurable, persistent registry to better monitor business processes.
> *Background*
> A lot of data in tracing and observability is on the exchange level. How many exchanges are completed or failed, for example. In many cases, what I really want to know if a transaction is successful or failed.
> With a transaction I mean that the complete lifecycle of a message on Camel, this 
> may contain many routes and exchanges. For example, when Seda or ActiveMQ is used 
> a new exchange is created, but it's still the same transaction.
>  
> To have metrics and more insights on transactions, I would like to have a transaction log (through a separate registry?). This is a separate store with information about the transaction. 
> *Implementation ideas*
> Below are few ideas to implement this wish:
> _1. Initialize the store_
> Setup a transaction log/store (either in memory or external store) through a registry. This is to enable it and to configure the store.
> _2. Transaction item_
> A transaction item is a definition of a specific transaction. It may contain the following items:
>  * {*}Name{*}: The name to indicate the transaction. Like "order" or "invoice". This is so to say the overarching name of the integration/business process.
>  * {*}StartingPoint{*}: A routeId or groupId where the transaction starts (triggers a new transaction).
>  * {*}CorrelationID(s){*}: One or more correlationId (headers) that can be set just like a header/exchangePropery/variable, this contains for example the orderID that is set on the first step of the transaction.
> _3. Tracking transactions_
> A transaction is probably best tracked by something like a breadcrumbId. Preferably a transactionId. At least something that cannot be edited by users (I sometimes see people change the breadcrumbId, to separate transactions into multiple segments).
> By default, only the start and end of a transaction is tracked. To get more fine-grained control, one could also track intermediate points like a route, a step, or a node.
> The transaction is tracked during processing, and then finished on completion. The whole counts as one "UnitOfWork".
> _4. The transaction store_
> The transaction store contains all the tracked items. Maybe there could be divided into separate segments like:
>  # metadata
>     - name (name of the transaction)
>     - description (description of type of transaction)
>     - transactionId
>     - transactionHistory (similar to messageHistory)
>     - correlationId(s)
>     - timestamps
>     - exceptions
>  # exchange data
>     - headers, properties, variables
>     - bodyType
>     - body
>     etc
>  # performance metrics
>     - load (CPU), memory (RAM), threads
>     - number of transactions (completed, pending, failed)
>     - throughput
>     - response time
>     - total process time of an transaction
>     - average
>     - etc
> It should be possible to configure which of these items the user what want to store. By default, only metadata.
> _5. Transaction API's_
> There need to be an API to configure and operate the transaction store. (including JMX), similar to backlogtracer.
> *Production*
> Some of the solutions I saw are only available during development or test, but the need is especially big for production environment. A solution needs to keep in mind that this would run for billion of messages. The flexibility to turn it easily on and off.
> For example, by default, the body of a message doesn't need to be stored. Or only the first 32kb need to be stored. But for specific case users, one to enable this on the fly for a specific transaction.
> Transactions are probably needed persistent, so either need a messagestore (disk) similar to brokers or an external database. I'm not sure what the best stores are for this kind of data. A normal SQL database like Postgres, a search database like Elastic, a metrics database like Prometheus or key/value store like Redis?
> *End users*
> The idea is that end users have a good insight in what is going on in the integration layer. So that they can question like the following:
>  * How many order transactions are processed?
>  * Give me a specific order by orderId that custom says he didn't receive?
>  * How many failed messages are there? Which ones did go wrong, where did it go wrong, and what is the exact error?
>  * Can we replay the failed order?
>  * Where is the bottleneck on the system
> *Conclusion*
> This is just an idea/wish, but something I see coming back from the start of my career as integration specialist (around 2007), until today. 
> Most consultancy firms or end customers have built something themselves, I see with names like "Track & Trace", "MessageStore", "ESB Collector", "Transactions", "Control center".
> I also built a solution myself, and are happy to share this solution, or others I saw. A more generic solution to monitor (track & trace) transactions of business processes would be very welcome.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)