You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tinkerpop.apache.org by pieter <pi...@riseup.net> on 2026/01/07 20:24:30 UTC
Re: [DISCUSS] Handling of transactions - proposal
Hi,
I had a brief discussion with Andrii in the discord chat regarding the
transaction api.
My idea is that the transaction api should be unopinionated.
That means it should follow closely to
`javax.transaction.UserTransaction` which in turn means we have a
dedicated class modelling the transaction with `begin()`, `commit()`
and `rollback()`.
My reasoning is that gremlin is a low level graph language. There are
many ways different developers / architects can go about integrating
gremlin into their stack.
Some like lambdas, some like IOT containers, some have domain entity
frameworks, some use aspect orienated programming and on an on.
It is common for IOT containers to manage transaction boundaries and
lifecycles. Similarly it is common to use the web server's interceptors
to manage transaction boundaries. And it is very common to do it all
manually...
Further, I'd say that transactions should be optional. Many persistent
db/graph systems are eventually consistent and do not support
transactions, so they should not be burdened with concepts foreign to
them.
Regarding binding the transaction to the thread:
This is the only way I have ever worked so its entirely natural to me.
However I can image that people want to defince their own transaction
context, so perhaps we should not hardcode that part. Long ago Gavin
King from Hibernate fame had the notion of `conversation` context.
Perhaps such things are still around?
Regards
Pieter
On Thu, 2025-12-11 at 09:02 +0100, Andrii Lomakin via dev wrote:
> Unfortunately, all images have been removed, so I am sending a link
> to the diagrams instead
> https://miro.com/app/board/uXjVGdd8m44=/?share_link_id=686668490384
> .
>
>
>
> On Thu, Dec 11, 2025 at 8:53 AM Andrii Lomakin
> <an...@jetbrains.com> wrote:
> > Good day.
> >
> > If I followed Ken's line of thought correctly, I would like to
> > expand it to be part of the new proposal for the transaction API.
> > First of all, about the model, I think we came to a consensus that
> > it should be:
> >
> >
> > image.png
> > The point is that we do not have a notion of connection in
> > TinkerPop, but we have a notion of GraphTraversal.
> > So, from my perspective, this diagram can be changed to
> > image.png
> >
> > There is a corner case for Non-TX databases that we need to
> > discuss.
> > I believe Non-TX mode is not appropriate for handling graph data
> > structures. GQL, for example, requires serializable isolation to
> > satisfy the standard requirements.
> > Instead, we should require to have presence of a transaction, which
> > is started implicitly when GraphTraversal is created and committed
> > once it is closed.
> >
> > I think we DO need methods for explicit transaction management, but
> > 90% of the cases will be covered by implicit transaction management
> > throughout the GraphTraversal life cycle.
> > 1. TX is started when GraphTraversal is created.
> > 2. TX is committed when GraphTraversal is closed (which includes
> > terminal operations like toList or iterate).
> > 3. TX is rolled back in case of exceptional execution of
> > GraphTraversal. Users can always roll back a transaction by using
> > the fail() step.
> >
> > We also provide in the YTDB the following methods that I hope will
> > become part of the framework: gSource.computeInTx(g ->{}),
> > gSource.executeInTx(g - {}), gSource.autoExecuteInTx(g - {}: g)
> > (this one executes g.iterate() on the returned GraphTraversal,
> > which we find really handy).
> >
> > Now, regarding explicit TX management.
> > I think that explicit TX management is still needed if you want to
> > execute several queries and mix them with complex logic, but I
> > think that methods that explicitly cause TX begin/start/commit are
> > confusing and provide bad DevX.
> > What I propose is to use lambda-controlled TXs (slightly different
> > from the above). I think that an explicit rollback/commit of TX by
> > the user is similar to the goto operator in old programming
> > languages; it gives finer control over TXs, but overall only adds
> > confusion and can be completely avoided in modern languages.
> >
> > Explicit control on TXs is proposed to be implemented using methods
> > similar to above, with a different signature, namely:
> > gSource.executeInTx(graphTraversalSupplier -> {
> > var g = graphTraversalSupplier.get();
> > var v = g.V().next();
> > g.close()
> >
> > v.property("p", "val");
> > int i = v.value("int");
> >
> > if(i > 0) {
> > g = graphTraversalSupplier.get();
> > g.V().has("i2", i).property("i2", i + 1).iterate();
> > }
> > })
> >
> > In this case, the transaction is detached from GraphTraversal, and
> > its lifecycle is not controlled by GraphTraversal.
> > Instead, TX is committed if an exception is not thrown from the
> > lambda method.
> >
> > To inform the user if GraphTraversal controls TX or not, a simple
> > isTxAttached method is proposed to be introduced
> > gSource.<V>computeInTX(graphTraversalSupplier -> {}) works in
> > similar way but returns value.
> >
> >
> > So the final diagram could look like the following:
> >
> >
> > image.png
> >
> > Please note that in this model, there is no need for an explicit
> > transaction object.
> >
> > On Fri, Dec 5, 2025 at 10:18 AM Andrii Lomakin
> > <an...@jetbrains.com> wrote:
> > > Good day, Ken.
> > >
> > > >This could map to multiple threads on the provider side. It's
> > > probably OK if something like "g.tx()..." returns the same
> > > transaction. We could require users to open multiple
> > > DriverRemoteConnections/Clients to achieve this and that would be
> > > similar to other data connectivity APIs.
> > >
> > > Unfortunately I do not follow, could you elaborate more or give
> > > me example how do you see it ?
> > >
> > > On Thu, Dec 4, 2025 at 7:06 PM Ken Hu <ke...@gmail.com> wrote:
> > > > I might be thinking about this too much for just the remote
> > > > case and not considering the embedded case enough. In my mind,
> > > > there just needs to be an easy way for users of the language
> > > > variants (in particular JavaScript because of its single
> > > > threaded nature) to start multiple transactions from a single
> > > > thread. This could map to multiple threads on the provider
> > > > side. It's probably OK if something like "g.tx()..." returns
> > > > the same transaction. We could require users to open multiple
> > > > DriverRemoteConnections/Clients to achieve this and that would
> > > > be similar to other data connectivity APIs.
> > > >
> > > > On Thu, Dec 4, 2025 at 2:37 AM Andrii Lomakin via dev
> > > > <de...@tinkerpop.apache.org> wrote:
> > > > > The point about having multiple concurrent transactions
> > > > > within the same
> > > > > thread warrants a more detailed discussion.
> > > > >
> > > > > To compare this behavior, consider how major SQL databases
> > > > > handle attempts
> > > > > to start a second transaction when one is already active:
> > > > >
> > > > > 1. Error out: The database prevents the operation, signaling
> > > > > that a
> > > > > transaction is already active (e.g., "Transaction already
> > > > > active").
> > > > > 2. Create a nested transaction (savepoint): The database
> > > > > creates a
> > > > > sub-scope of the first transaction, not an independent
> > > > > transaction. If the
> > > > > parent transaction fails, the child scope also fails.
> > > > > 3. Implicitly commit the first: The database automatically
> > > > > commits the
> > > > > first transaction to open the second one (a common behavior
> > > > > in MySQL DDL).
> > > > >
> > > > > As far as I know, only Firebird has historically allowed
> > > > > several truly
> > > > > independent transactions concurrently on the same thread. I
> > > > > suspect support
> > > > > for simultaneous transactions may be even more limited among
> > > > > graph database
> > > > > vendors.
> > > > >
> > > > > On Thu, Dec 4, 2025 at 10:47 AM Andrii Lomakin
> > > > > <an...@jetbrains.com>
> > > > > wrote:
> > > > >
> > > > > > Good day.
> > > > > > As for questions from Ken
> > > > > > >What OGMs are you referring to here? Regular transactions
> > > > > are complicated
> > > > > > enough by itself, I would only want to introduce nested
> > > > > transactions if
> > > > > > there were well maintained OGMs that would have a
> > > > > demonstrable benefit from
> > > > > > this.
> > > > > >
> > > > > > We have such an OGM using TinkerPop that is well
> > > > > maintained, if you mean
> > > > > > by this years of support and thousands of instances running
> > > > > on OS version.
> > > > > > However, I have given it thought and believe that it is a
> > > > > relatively rare
> > > > > > case, and OGM is so complex that it can manage nested
> > > > > transactions itself
> > > > > > if needed.
> > > > > > I think we can support nested transactions on pause for
> > > > > now, till we
> > > > > > receive more feedback from users.
> > > > > >
> > > > > > There are many inconsistencies to discuss in the TX
> > > > > lifecycle, even
> > > > > > without nested transactions.
> > > > > >
> > > > > > >Sessions won't exist moving forward (4.x+) as they are
> > > > > tied to remote
> > > > > > Groovy execution. So let's continue this conversation as
> > > > > simply
> > > > > > transactions and not refer to sessions at all to prevent
> > > > > confusion, unless
> > > > > > you intend to have these changes in the 3.8.x line?
> > > > > > Yeah, we already removed Groovy support in our fork, so it
> > > > > is even now in
> > > > > > our distribution TX, and the session is the same.
> > > > > >
> > > > > >
> > > > > > >Could you expand a bit more on "Transaction scope is
> > > > > limited to the
> > > > > > current thread only".
> > > > > > I mean that TX visibility and manipulation are limited to a
> > > > > single thread
> > > > > > where it is created.
> > > > > > If some of the providers support TX visibility between
> > > > > several threads,
> > > > > > that will only add a bonus for them if such a need arises,
> > > > > of course,
> > > > > > without breaking the API.
> > > > > >
> > > > > > > I find it limiting that an attempt to open multiple
> > > > > transactions from
> > > > > > the same thread ends up returning the same Transaction
> > > > > instance
> > > > > > Do you propose to support several transactions in the same
> > > > > thread?
> > > > > > I am afraid not all current vendors can support this model.
> > > > > >
> > > > > > On Wed, Dec 3, 2025 at 1:23 PM Stephen Mallette
> > > > > <sp...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > >> Andrii, I'd like to just quickly clarify Ken's point
> > > > > stating " Sessions
> > > > > >> won't exist moving forward (4.x+) ". He's saying what you
> > > > > said: "
> > > > > >> 'session'
> > > > > >> and 'tx' are essentially the same concepts". We
> > > > > acknowledged that and
> > > > > >> decided to stop using "session" terminology and instead
> > > > > use "transaction"
> > > > > >> terminology only to avoid confusion.
> > > > > >>
> > > > > >> Much of what you wrote is in the spirit of what has been
> > > > > considered for
> > > > > >> 4.x, so that is nice alignment. I do think the goal though
> > > > > is to have a
> > > > > >> consistent transaction API/system that works for both
> > > > > embedded and remote
> > > > > >> cases. It will be interesting to see how that is achieved.
> > > > > >>
> > > > > >> On Tue, Dec 2, 2025 at 3:53 PM Ken Hu <ke...@gmail.com>
> > > > > wrote:
> > > > > >>
> > > > > >> > Hi Andrii,
> > > > > >> >
> > > > > >> > Good day to you as well. I have a couple questions about
> > > > > this that I
> > > > > >> would
> > > > > >> > like some more details on.
> > > > > >> >
> > > > > >> > 1. What OGMs are you referring to here? Regular
> > > > > transactions are
> > > > > >> > complicated enough by itself, I would only want to
> > > > > introduce nested
> > > > > >> > transactions if there were well maintained OGMs that
> > > > > would have a
> > > > > >> > demonstrable benefit from this.
> > > > > >> > 2. I like some of your ideas about removing Transaction
> > > > > as its own
> > > > > >> class,
> > > > > >> > but I'm not sure GraphTraversalSource is where it should
> > > > > be instead. Do
> > > > > >> you
> > > > > >> > have some more details about what you think the
> > > > > Transaction API should
> > > > > >> look
> > > > > >> > like?
> > > > > >> > 3. Could you expand a bit more on "Transaction scope is
> > > > > limited to the
> > > > > >> > current thread only". I always get confused by
> > > > > TinkerPop's threaded vs
> > > > > >> > multithreaded transaction. I find it limiting that an
> > > > > attempt to open
> > > > > >> > multiple transactions from the same thread ends up
> > > > > returning the same
> > > > > >> > Transaction instance. Maybe I didn't quite understand
> > > > > what you meant by
> > > > > >> > this.
> > > > > >> > 4. Sessions won't exist moving forward (4.x+) as they
> > > > > are tied to remote
> > > > > >> > Groovy execution. So let's continue this conversation as
> > > > > simply
> > > > > >> > transactions and not refer to sessions at all to prevent
> > > > > confusion,
> > > > > >> unless
> > > > > >> > you intend to have these changes in the 3.8.x line?
> > > > > >> >
> > > > > >> > Regards,
> > > > > >> > Ken
> > > > > >> >
> > > > > >> >
> > > > > >> > On Tue, Dec 2, 2025 at 3:33 AM Andrii Lomakin via dev <
> > > > > >> > dev@tinkerpop.apache.org> wrote:
> > > > > >> >
> > > > > >> > > Small clarification.
> > > > > >> > > Implementation notes are related to changes in the
> > > > > remote protocol.
> > > > > >> > >
> > > > > >> > > On Tue, Dec 2, 2025 at 8:55 AM Andrii Lomakin <
> > > > > >> > > andrey.lomakin@jetbrains.com>
> > > > > >> > > wrote:
> > > > > >> > >
> > > > > >> > > > To clarify: in this model, 'session' and 'tx' are
> > > > > essentially the
> > > > > >> same
> > > > > >> > > > concepts.
> > > > > >> > > >
> > > > > >> > > > I also think it is essential to have a common
> > > > > denominator that
> > > > > >> makes TP
> > > > > >> > > > applications portable across implementations.
> > > > > >> > > > That is why I also propose to restrict transaction
> > > > > scope to the
> > > > > >> thread
> > > > > >> > > > scope. It will be a common denominator that will not
> > > > > harm user
> > > > > >> > experience
> > > > > >> > > > but will ensure uniform and expected API behavior.
> > > > > >> > > > There should not be both `begin` and `open` methods
> > > > > because the
> > > > > >> > contract
> > > > > >> > > > of the beginning method is vague and unpredictable.
> > > > > >> > > >
> > > > > >> > > > On Tue, Dec 2, 2025 at 8:18 AM Andrii Lomakin <
> > > > > >> > > > andrey.lomakin@jetbrains.com> wrote:
> > > > > >> > > >
> > > > > >> > > >> Good day,
> > > > > >> > > >>
> > > > > >> > > >> I have a proposal to enhance the transaction
> > > > > handling logic in TP,
> > > > > >> > > >> specifically regarding nested transaction calls,
> > > > > which are common
> > > > > >> in
> > > > > >> > EE
> > > > > >> > > >> development, particularly when utilizing OGM.
> > > > > >> > > >>
> > > > > >> > > >> Currently, calling open multiple times causes an
> > > > > exception. This
> > > > > >> > > behavior
> > > > > >> > > >> is restrictive, and the term "open" can be
> > > > > confusing.
> > > > > >> > > >>
> > > > > >> > > >> I propose a design that improves the developer
> > > > > experience by using
> > > > > >> an
> > > > > >> > > >> internal counter for transaction calls:
> > > > > >> > > >>
> > > > > >> > > >> Proposed transaction counter logic
> > > > > >> > > >> 1. begin/commit counter:
> > > > > >> > > >> - begin would increment an internal counter.
> > > > > >> > > >> - commit would decrement the internal counter,
> > > > > causing the actual
> > > > > >> > > >> transaction commit only when the counter reaches
> > > > > zero.
> > > > > >> > > >> 2. rollback behavior:
> > > > > >> > > >> - rollback would immediately force a transaction
> > > > > rollback and
> > > > > >> > > decrements
> > > > > >> > > >> the internal counter
> > > > > >> > > >> - the subsequent calls to rollback on the same
> > > > > transaction object
> > > > > >> > > should
> > > > > >> > > >> be allowed until the stack trace reaches the
> > > > > initial begin method
> > > > > >> > call,
> > > > > >> > > >> after which further rollback calls would throw an
> > > > > exception.
> > > > > >> > > >> 3. Error handling:
> > > > > >> > > >> - A commit without a matching begin (e.g., counter
> > > > > is already zero)
> > > > > >> > > >> should throw an exception.
> > > > > >> > > >> - A rollback without a matching begin should throw
> > > > > an exception
> > > > > >> > > >>
> > > > > >> > > >> I would also consider hiding the Transaction object
> > > > > altogether and
> > > > > >> > > >> delegating the logic of transaction management to
> > > > > >> GraphTraversalSource
> > > > > >> > > it
> > > > > >> > > >> will simplify the API a lot, both for users and
> > > > > vendors.
> > > > > >> > > >>
> > > > > >> > > >> I also propose to restrict transaction scope to the
> > > > > thread scope,
> > > > > >> it
> > > > > >> > > will
> > > > > >> > > >> be a common denominator that will not harm user
> > > > > experience but will
> > > > > >> > > ensure
> > > > > >> > > >> uniform and expected API behavior across all
> > > > > implementations.
> > > > > >> > > >>
> > > > > >> > > >> Proposed API changes
> > > > > >> > > >> - I propose deprecating open and using begin with
> > > > > the semantics
> > > > > >> > > >> described above to avoid confusion.
> > > > > >> > > >> - Transaction object is deprecated, and TX control
> > > > > functionality is
> > > > > >> > > >> delegated to the GraphTraversal/Graph instances.
> > > > > >> > > >> - tx() method is deprecated.
> > > > > >> > > >> - Transaction scope is limited to the current
> > > > > thread only.
> > > > > >> > > >>
> > > > > >> > > >> Behavior for non-direct calls
> > > > > >> > > >> When begin/commit/rollback methods are not called
> > > > > directly, the
> > > > > >> > > >> transaction should be automatically committed by a
> > > > > terminal
> > > > > >> operation
> > > > > >> > > >> (e.g., when hasNext returns false) in both remote
> > > > > and embedded
> > > > > >> modes.
> > > > > >> > > >>
> > > > > >> > > >> Implementation notes:
> > > > > >> > > >> 1. The begin command should be added. Relying on
> > > > > the implicit
> > > > > >> start of
> > > > > >> > > TX
> > > > > >> > > >> by traversal can lead to non-controlled side
> > > > > effects.
> > > > > >> > > >> 2. For the sake of optimization of the remote
> > > > > protocol, while
> > > > > >> multiple
> > > > > >> > > >> calls of begin/commit/rollback are allowed in the
> > > > > remote protocol,
> > > > > >> > > >> practically, it will mean that we will track the
> > > > > counter locally on
> > > > > >> > the
> > > > > >> > > >> client and will send begin/commit/rollback only
> > > > > once for the TX
> > > > > >> > > lifecycle.
> > > > > >> > > >>
> > > > > >> > > >> This model should be uniform across all deployment
> > > > > types.
> > > > > >> > > >> What are your thoughts on this approach?
> > > > > >> > > >>
> > > > > >> > > >> -----
> > > > > >> > > >> Andrii Lomakin
> > > > > >> > > >> YouTrackDB development lead
> > > > > >> > > >>
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > > --
> > > > > >> > > > Andrii Lomakin
> > > > > >> > > > YouTrackDB development lead
> > > > > >> > > >
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > --
> > > > > >> > > Andrii Lomakin
> > > > > >> > > YouTrackDB development lead
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Andrii Lomakin
> > > > > > YouTrackDB development lead
> > > > > >
> > > > >
> > > > >
Re: [DISCUSS] Handling of transactions - proposal
Posted by Ken Hu <ke...@gmail.com>.
Something to note about replacing open() with begin(). The legacy embedded
open() threw transactionAlreadyOpen() if a user attempted to open a
transaction more than once. begin()'s documentation states that its up to
the provider what that method does. Since begin() will replace open(), I'm
proposing that begin()'s semantics be tightened to instead be idempotent.
When a transaction is already open, it does not start a new transaction and
does not throw, it simply returns a usable TraversalSource bound to the
existing transaction. This is a behavioral breaking change: code that today
relies on catching an exception from a second begin() will no longer see
one. This makes sense because strict throw-on-already-open is fundamentally
incompatible with the embedded AUTO read/write default. Under AUTO, a read
or write implicitly opens the transaction, so a subsequent explicit begin()
would throw even though the caller did nothing wrong.
On Thu, Jun 11, 2026 at 5:33 PM Ken Hu <ke...@gmail.com> wrote:
> We haven't talked about this for a while but I think there were some good
> changes that were proposed that we should go ahead with so I've broken some
> of the smaller parts out into their own Jiras.
>
> https://issues.apache.org/jira/browse/TINKERPOP-3252 which is removing
> open() since it has the same functionality as begin().
> https://issues.apache.org/jira/browse/TINKERPOP-3253 which is adding
> convenience lambdas. Manual control of transactions still exist and this is
> only an addition.
>
> From what I remember, I believe these were mostly agreed to in some form.
> I plan to have pull requests for these soon so please provide feedback if
> you have any.
>
> On Wed, Feb 18, 2026 at 6:44 PM Ken Hu <ke...@gmail.com> wrote:
>
>> Sorry for not keeping this thread more up to date. I've been playing
>> around with some of the ideas from this thread and ones that Pieter
>> suggested in the TinkerPop Gatherings but I haven't yet found something I
>> find completely satisfactory. Just wanted to let everyone know that I'm
>> still looking into this and will post an update when I've found something
>> more promising.
>>
>> On Sun, Jan 25, 2026 at 10:05 PM Ken Hu <ke...@gmail.com> wrote:
>>
>>> Given that this is a major version upgrade, I think we can break the
>>> core interfaces as much as needed as long as there is a good reason for
>>> doing so. I understand the hesitancy to make breaking changes, but it's
>>> become apparent that the APIs don't work well for the remote case. I also
>>> expect that the 3.x line will be maintained for a while even after the
>>> release of TinkerPop 4 so that will give users extra time to upgrade. But
>>> yes, we should keep the upgrade path in mind.
>>>
>>> On Fri, Jan 23, 2026 at 11:49 PM pieter <pi...@riseup.net>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Before discussing these changes further, I need some clarity on how
>>>> much we are willing to break core interfaces in TP 4?
>>>>
>>>> The core interfaces have been around for more than a decade. i.e.
>>>> millions and millions of lines of code in production.
>>>>
>>>> Personally I am in the 'don't break user space'. camp.
>>>>
>>>> If we do make drastic changes, we should right from the start think
>>>> about the upgrade path and think about the impact on existing code
>>>> bases.
>>>>
>>>> Regards
>>>> Pieter
>>>>
>>>>
>>>>
>>>>
>>>> On Wed, 2026-01-21 at 20:40 -0800, Ken Hu wrote:
>>>> > I agree that record consistency is important, but I feel that where
>>>> > this
>>>> > gets solved depends on how users integrate graph databases into their
>>>> > applications. There are probably frameworks and OGM/ORMs that can do
>>>> > this
>>>> > detaching/unloading that you are mentioning as a solution. If a user
>>>> > directly integrates the database as an external service using a
>>>> > driver,
>>>> > then having it as part of the API could be useful. I need to think
>>>> > about
>>>> > this more.
>>>> >
>>>> > To your second point about platform-independence, I think this has
>>>> > always
>>>> > been a goal of TinkerPop that has never quite been realized. Even
>>>> > today,
>>>> > there are some examples of Gremlin that return results that are
>>>> > provider-dependent and so there are specific cases where it isn't
>>>> > truly
>>>> > portable. I agree that a difference in read consistency amongst
>>>> > vendors/providers means those using transactions can't move between
>>>> > different databases unless they find another database with the exact
>>>> > same
>>>> > transaction behavior. But I'm not entirely sure we can get all
>>>> > providers to
>>>> > match the same behavior in this case.
>>>> >
>>>> > On Tue, Jan 20, 2026 at 11:26 PM Andrii Lomakin via dev <
>>>> > dev@tinkerpop.apache.org> wrote:
>>>> >
>>>> > > Hi Ken.
>>>> > > Thank you for sharing this with us.
>>>> > >
>>>> > > After some thought, I agree that we need explicit TX management
>>>> > > methods.
>>>> > > From my perspective, what I wrote is only the tip of the iceberg,
>>>> > > as such
>>>> > > questions as consistency of the queries and validity of the data
>>>> > > contained
>>>> > > in vertices directly bound to the concept of TX lifecycle.
>>>> > >
>>>> > > Example (it can be applied to any database, not just a graph
>>>> > > database, so I
>>>> > > will use the general record concept).
>>>> > >
>>>> > > I have created record A { id: 1, value: "val1"} , committed it, and
>>>> > > passed
>>>> > > it to another thread.
>>>> > > After that, another method updated the same record B {id: 1, value
>>>> > > :
>>>> > > "val2"} and committed it.
>>>> > >
>>>> > > A and B in this context object identity, not DB record identity.
>>>> > >
>>>> > > But I have the record A in another thread with potentially stale
>>>> > > data.
>>>> > > What then should another thread read when it accesses `A.value`?
>>>> > >
>>>> > > I doubt it should read stale data. In our database, records are
>>>> > > automatically switched to the unloaded state after commit and can
>>>> > > be read
>>>> > > in a new thread only when a new transaction is started.
>>>> > > The question of record consistency, from my point of view, is an
>>>> > > important
>>>> > > one and cannot be overlooked, as it influences how users interact
>>>> > > with the
>>>> > > TP API and to what degree it is really platform-independent.
>>>> > >
>>>> > >
>>>> > > On Wed, Jan 21, 2026 at 7:58 AM Ken Hu <ke...@gmail.com> wrote:
>>>> > >
>>>> > > > Here are some of my current thoughts on the proposed changes and
>>>> > > > discussion.
>>>> > > >
>>>> > > > -deprecate open() and replace with begin()
>>>> > > > -I agree with this as having both is confusing and unncessary
>>>> > > >
>>>> > > > -Transaction class is deprecated, and TX control functionality is
>>>> > > delegated
>>>> > > > to the GraphTraversal/Graph instances
>>>> > > > -I mostly agree with this as well, but I think it's probably
>>>> > > > better
>>>> > > if
>>>> > > > it was delegated to GraphTraversalSource.
>>>> > > >
>>>> > > > -tx() method is deprecated.
>>>> > > > -I think this makes sense as I always found how tx() works
>>>> > > > with
>>>> > > remote
>>>> > > > to be unintuitive.
>>>> > > >
>>>> > > > -Transaction scope is limited to the current thread only
>>>> > > > -I had some questions about this earlier, but having thought
>>>> > > > about it
>>>> > > > after seeing Andrii's architectural diagram, it's probably fine
>>>> > > > since
>>>> > > most
>>>> > > > developers (in all languages TinkerPop supports) will be used to
>>>> > > > this
>>>> > > being
>>>> > > > how it works as long as we have something that represents what a
>>>> > > connection
>>>> > > > is in other databases.
>>>> > > >
>>>> > > > -I believe Non-TX mode is not appropriate for handling graph data
>>>> > > > structures.
>>>> > > > -Further, I'd say that transactions should be optional. Many
>>>> > > > persistent
>>>> > > > db/graph systems are eventually consistent and do not support
>>>> > > transactions,
>>>> > > > so they should not be burdened with concepts foreign to them.
>>>> > > > -I tend to agree with Pieter here. My reasoning is different
>>>> > > > though,
>>>> > > > one of the use cases for graphs is analytics and transactions
>>>> > > > aren't
>>>> > > really
>>>> > > > used in that case.
>>>> > > >
>>>> > > > -I think we DO need methods for explicit transaction management,
>>>> > > > but 90%
>>>> > > of
>>>> > > > the cases will be covered by implicit transaction management
>>>> > > > throughout
>>>> > > the
>>>> > > > GraphTraversal life cycle.
>>>> > > > -What I propose is to use lambda-controlled TXs
>>>> > > > -My idea is that the transaction api should be unopinionated.
>>>> > > > That means
>>>> > > it
>>>> > > > should follow closely to `UserTransaction`
>>>> > > > -I think having implicit transactions is fine as long as it's
>>>> > > > just an
>>>> > > > option. I'm not sure which should be the default. I'm not sure we
>>>> > > > can
>>>> > > make
>>>> > > > lambda-controlled transactions work though since lambdas wouldn't
>>>> > > > be
>>>> > > > compatible with Gremlin scripts. At this point, we probably still
>>>> > > > want
>>>> > > > parity between scripts and GraphTraversals.
>>>> > > >
>>>> > > > I think we need some more proposals for what the explicit
>>>> > > > transaction
>>>> > > > management API could look like because we probably can't move
>>>> > > > ahead with
>>>> > > > lambda-controlled TX for now and the current tx() solution is
>>>> > > > clunky.
>>>> > > I'll
>>>> > > > try to propose one in the next couple of days.
>>>> > > >
>>>> > > > On Wed, Jan 7, 2026 at 12:24 PM pieter <pi...@riseup.net>
>>>> > > > wrote:
>>>> > > >
>>>> > > > > Hi,
>>>> > > > >
>>>> > > > > I had a brief discussion with Andrii in the discord chat
>>>> > > > > regarding the
>>>> > > > > transaction api.
>>>> > > > >
>>>> > > > > My idea is that the transaction api should be unopinionated.
>>>> > > > > That means it should follow closely to
>>>> > > > > `javax.transaction.UserTransaction` which in turn means we have
>>>> > > > > a
>>>> > > > > dedicated class modelling the transaction with `begin()`,
>>>> > > > > `commit()`
>>>> > > > > and `rollback()`.
>>>> > > > >
>>>> > > > > My reasoning is that gremlin is a low level graph language.
>>>> > > > > There are
>>>> > > > > many ways different developers / architects can go about
>>>> > > > > integrating
>>>> > > > > gremlin into their stack.
>>>> > > > >
>>>> > > > > Some like lambdas, some like IOT containers, some have domain
>>>> > > > > entity
>>>> > > > > frameworks, some use aspect orienated programming and on an on.
>>>> > > > >
>>>> > > > > It is common for IOT containers to manage transaction
>>>> > > > > boundaries and
>>>> > > > > lifecycles. Similarly it is common to use the web server's
>>>> > > > > interceptors
>>>> > > > > to manage transaction boundaries. And it is very common to do
>>>> > > > > it all
>>>> > > > > manually...
>>>> > > > >
>>>> > > > > Further, I'd say that transactions should be optional. Many
>>>> > > > > persistent
>>>> > > > > db/graph systems are eventually consistent and do not support
>>>> > > > > transactions, so they should not be burdened with concepts
>>>> > > > > foreign to
>>>> > > > > them.
>>>> > > > >
>>>> > > > > Regarding binding the transaction to the thread:
>>>> > > > > This is the only way I have ever worked so its entirely natural
>>>> > > > > to me.
>>>> > > > > However I can image that people want to defince their own
>>>> > > > > transaction
>>>> > > > > context, so perhaps we should not hardcode that part. Long ago
>>>> > > > > Gavin
>>>> > > > > King from Hibernate fame had the notion of `conversation`
>>>> > > > > context.
>>>> > > > > Perhaps such things are still around?
>>>> > > > >
>>>> > > > > Regards
>>>> > > > > Pieter
>>>> > > > >
>>>> > > > > On Thu, 2025-12-11 at 09:02 +0100, Andrii Lomakin via dev
>>>> > > > > wrote:
>>>> > > > > > Unfortunately, all images have been removed, so I am sending
>>>> > > > > > a link
>>>> > > > > > to the diagrams instead
>>>> > > > > >
>>>> https://miro.com/app/board/uXjVGdd8m44=/?share_link_id=686668490384
>>>> > > > > > .
>>>> > > > > >
>>>> > > > > >
>>>> > > > > >
>>>> > > > > > On Thu, Dec 11, 2025 at 8:53 AM Andrii Lomakin
>>>> > > > > > <an...@jetbrains.com> wrote:
>>>> > > > > > > Good day.
>>>> > > > > > >
>>>> > > > > > > If I followed Ken's line of thought correctly, I would like
>>>> > > > > > > to
>>>> > > > > > > expand it to be part of the new proposal for the
>>>> > > > > > > transaction API.
>>>> > > > > > > First of all, about the model, I think we came to a
>>>> > > > > > > consensus that
>>>> > > > > > > it should be:
>>>> > > > > > >
>>>> > > > > > >
>>>> > > > > > > image.png
>>>> > > > > > > The point is that we do not have a notion of connection in
>>>> > > > > > > TinkerPop, but we have a notion of GraphTraversal.
>>>> > > > > > > So, from my perspective, this diagram can be changed to
>>>> > > > > > > image.png
>>>> > > > > > >
>>>> > > > > > > There is a corner case for Non-TX databases that we need to
>>>> > > > > > > discuss.
>>>> > > > > > > I believe Non-TX mode is not appropriate for handling graph
>>>> > > > > > > data
>>>> > > > > > > structures. GQL, for example, requires serializable
>>>> > > > > > > isolation to
>>>> > > > > > > satisfy the standard requirements.
>>>> > > > > > > Instead, we should require to have presence of a
>>>> > > > > > > transaction, which
>>>> > > > > > > is started implicitly when GraphTraversal is created and
>>>> > > > > > > committed
>>>> > > > > > > once it is closed.
>>>> > > > > > >
>>>> > > > > > > I think we DO need methods for explicit transaction
>>>> > > > > > > management, but
>>>> > > > > > > 90% of the cases will be covered by implicit transaction
>>>> > > > > > > management
>>>> > > > > > > throughout the GraphTraversal life cycle.
>>>> > > > > > > 1. TX is started when GraphTraversal is created.
>>>> > > > > > > 2. TX is committed when GraphTraversal is closed (which
>>>> > > > > > > includes
>>>> > > > > > > terminal operations like toList or iterate).
>>>> > > > > > > 3. TX is rolled back in case of exceptional execution of
>>>> > > > > > > GraphTraversal. Users can always roll back a transaction by
>>>> > > > > > > using
>>>> > > > > > > the fail() step.
>>>> > > > > > >
>>>> > > > > > > We also provide in the YTDB the following methods that I
>>>> > > > > > > hope will
>>>> > > > > > > become part of the framework: gSource.computeInTx(g ->{}),
>>>> > > > > > > gSource.executeInTx(g - {}), gSource.autoExecuteInTx(g -
>>>> > > > > > > {}: g)
>>>> > > > > > > (this one executes g.iterate() on the returned
>>>> > > > > > > GraphTraversal,
>>>> > > > > > > which we find really handy).
>>>> > > > > > >
>>>> > > > > > > Now, regarding explicit TX management.
>>>> > > > > > > I think that explicit TX management is still needed if you
>>>> > > > > > > want to
>>>> > > > > > > execute several queries and mix them with complex logic,
>>>> > > > > > > but I
>>>> > > > > > > think that methods that explicitly cause TX
>>>> > > > > > > begin/start/commit are
>>>> > > > > > > confusing and provide bad DevX.
>>>> > > > > > > What I propose is to use lambda-controlled TXs (slightly
>>>> > > > > > > different
>>>> > > > > > > from the above). I think that an explicit rollback/commit
>>>> > > > > > > of TX by
>>>> > > > > > > the user is similar to the goto operator in old programming
>>>> > > > > > > languages; it gives finer control over TXs, but overall
>>>> > > > > > > only adds
>>>> > > > > > > confusion and can be completely avoided in modern
>>>> > > > > > > languages.
>>>> > > > > > >
>>>> > > > > > > Explicit control on TXs is proposed to be implemented using
>>>> > > > > > > methods
>>>> > > > > > > similar to above, with a different signature, namely:
>>>> > > > > > > gSource.executeInTx(graphTraversalSupplier -> {
>>>> > > > > > > var g = graphTraversalSupplier.get();
>>>> > > > > > > var v = g.V().next();
>>>> > > > > > > g.close()
>>>> > > > > > >
>>>> > > > > > > v.property("p", "val");
>>>> > > > > > > int i = v.value("int");
>>>> > > > > > >
>>>> > > > > > > if(i > 0) {
>>>> > > > > > > g = graphTraversalSupplier.get();
>>>> > > > > > > g.V().has("i2", i).property("i2", i +
>>>> > > > > > > 1).iterate();
>>>> > > > > > > }
>>>> > > > > > > })
>>>> > > > > > >
>>>> > > > > > > In this case, the transaction is detached from
>>>> > > > > > > GraphTraversal, and
>>>> > > > > > > its lifecycle is not controlled by GraphTraversal.
>>>> > > > > > > Instead, TX is committed if an exception is not thrown from
>>>> > > > > > > the
>>>> > > > > > > lambda method.
>>>> > > > > > >
>>>> > > > > > > To inform the user if GraphTraversal controls TX or not, a
>>>> > > > > > > simple
>>>> > > > > > > isTxAttached method is proposed to be introduced
>>>> > > > > > > gSource.<V>computeInTX(graphTraversalSupplier -> {}) works
>>>> > > > > > > in
>>>> > > > > > > similar way but returns value.
>>>> > > > > > >
>>>> > > > > > >
>>>> > > > > > > So the final diagram could look like the following:
>>>> > > > > > >
>>>> > > > > > >
>>>> > > > > > > image.png
>>>> > > > > > >
>>>> > > > > > > Please note that in this model, there is no need for an
>>>> > > > > > > explicit
>>>> > > > > > > transaction object.
>>>> > > > > > >
>>>> > > > > > > On Fri, Dec 5, 2025 at 10:18 AM Andrii Lomakin
>>>> > > > > > > <an...@jetbrains.com> wrote:
>>>> > > > > > > > Good day, Ken.
>>>> > > > > > > >
>>>> > > > > > > > > This could map to multiple threads on the provider
>>>> > > > > > > > > side. It's
>>>> > > > > > > > probably OK if something like "g.tx()..." returns the
>>>> > > > > > > > same
>>>> > > > > > > > transaction. We could require users to open multiple
>>>> > > > > > > > DriverRemoteConnections/Clients to achieve this and that
>>>> > > > > > > > would be
>>>> > > > > > > > similar to other data connectivity APIs.
>>>> > > > > > > >
>>>> > > > > > > > Unfortunately I do not follow, could you elaborate more
>>>> > > > > > > > or give
>>>> > > > > > > > me example how do you see it ?
>>>> > > > > > > >
>>>> > > > > > > > On Thu, Dec 4, 2025 at 7:06 PM Ken Hu
>>>> > > > > > > > <ke...@gmail.com>
>>>> > > wrote:
>>>> > > > > > > > > I might be thinking about this too much for just the
>>>> > > > > > > > > remote
>>>> > > > > > > > > case and not considering the embedded case enough. In
>>>> > > > > > > > > my mind,
>>>> > > > > > > > > there just needs to be an easy way for users of the
>>>> > > > > > > > > language
>>>> > > > > > > > > variants (in particular JavaScript because of its
>>>> > > > > > > > > single
>>>> > > > > > > > > threaded nature) to start multiple transactions from a
>>>> > > > > > > > > single
>>>> > > > > > > > > thread. This could map to multiple threads on the
>>>> > > > > > > > > provider
>>>> > > > > > > > > side. It's probably OK if something like "g.tx()..."
>>>> > > > > > > > > returns
>>>> > > > > > > > > the same transaction. We could require users to open
>>>> > > > > > > > > multiple
>>>> > > > > > > > > DriverRemoteConnections/Clients to achieve this and
>>>> > > > > > > > > that would
>>>> > > > > > > > > be similar to other data connectivity APIs.
>>>> > > > > > > > >
>>>> > > > > > > > > On Thu, Dec 4, 2025 at 2:37 AM Andrii Lomakin via dev
>>>> > > > > > > > > <de...@tinkerpop.apache.org> wrote:
>>>> > > > > > > > > > The point about having multiple concurrent
>>>> > > > > > > > > > transactions
>>>> > > > > > > > > > within the same
>>>> > > > > > > > > > thread warrants a more detailed discussion.
>>>> > > > > > > > > >
>>>> > > > > > > > > > To compare this behavior, consider how major SQL
>>>> > > > > > > > > > databases
>>>> > > > > > > > > > handle attempts
>>>> > > > > > > > > > to start a second transaction when one is already
>>>> > > > > > > > > > active:
>>>> > > > > > > > > >
>>>> > > > > > > > > > 1. Error out: The database prevents the operation,
>>>> > > > > > > > > > signaling
>>>> > > > > > > > > > that a
>>>> > > > > > > > > > transaction is already active (e.g., "Transaction
>>>> > > > > > > > > > already
>>>> > > > > > > > > > active").
>>>> > > > > > > > > > 2. Create a nested transaction (savepoint): The
>>>> > > > > > > > > > database
>>>> > > > > > > > > > creates a
>>>> > > > > > > > > > sub-scope of the first transaction, not an
>>>> > > > > > > > > > independent
>>>> > > > > > > > > > transaction. If the
>>>> > > > > > > > > > parent transaction fails, the child scope also fails.
>>>> > > > > > > > > > 3. Implicitly commit the first: The database
>>>> > > > > > > > > > automatically
>>>> > > > > > > > > > commits the
>>>> > > > > > > > > > first transaction to open the second one (a common
>>>> > > > > > > > > > behavior
>>>> > > > > > > > > > in MySQL DDL).
>>>> > > > > > > > > >
>>>> > > > > > > > > > As far as I know, only Firebird has historically
>>>> > > > > > > > > > allowed
>>>> > > > > > > > > > several truly
>>>> > > > > > > > > > independent transactions concurrently on the same
>>>> > > > > > > > > > thread. I
>>>> > > > > > > > > > suspect support
>>>> > > > > > > > > > for simultaneous transactions may be even more
>>>> > > > > > > > > > limited among
>>>> > > > > > > > > > graph database
>>>> > > > > > > > > > vendors.
>>>> > > > > > > > > >
>>>> > > > > > > > > > On Thu, Dec 4, 2025 at 10:47 AM Andrii Lomakin
>>>> > > > > > > > > > <an...@jetbrains.com>
>>>> > > > > > > > > > wrote:
>>>> > > > > > > > > >
>>>> > > > > > > > > > > Good day.
>>>> > > > > > > > > > > As for questions from Ken
>>>> > > > > > > > > > > > What OGMs are you referring to here? Regular
>>>> > > > > > > > > > > > transactions
>>>> > > > > > > > > > are complicated
>>>> > > > > > > > > > > enough by itself, I would only want to introduce
>>>> > > > > > > > > > > nested
>>>> > > > > > > > > > transactions if
>>>> > > > > > > > > > > there were well maintained OGMs that would have a
>>>> > > > > > > > > > demonstrable benefit from
>>>> > > > > > > > > > > this.
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > We have such an OGM using TinkerPop that is well
>>>> > > > > > > > > > maintained, if you mean
>>>> > > > > > > > > > > by this years of support and thousands of instances
>>>> > > > > > > > > > > running
>>>> > > > > > > > > > on OS version.
>>>> > > > > > > > > > > However, I have given it thought and believe that
>>>> > > > > > > > > > > it is a
>>>> > > > > > > > > > relatively rare
>>>> > > > > > > > > > > case, and OGM is so complex that it can manage
>>>> > > > > > > > > > > nested
>>>> > > > > > > > > > transactions itself
>>>> > > > > > > > > > > if needed.
>>>> > > > > > > > > > > I think we can support nested transactions on pause
>>>> > > > > > > > > > > for
>>>> > > > > > > > > > now, till we
>>>> > > > > > > > > > > receive more feedback from users.
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > There are many inconsistencies to discuss in the TX
>>>> > > > > > > > > > lifecycle, even
>>>> > > > > > > > > > > without nested transactions.
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > > Sessions won't exist moving forward (4.x+) as
>>>> > > > > > > > > > > > they are
>>>> > > > > > > > > > tied to remote
>>>> > > > > > > > > > > Groovy execution. So let's continue this
>>>> > > > > > > > > > > conversation as
>>>> > > > > > > > > > simply
>>>> > > > > > > > > > > transactions and not refer to sessions at all to
>>>> > > > > > > > > > > prevent
>>>> > > > > > > > > > confusion, unless
>>>> > > > > > > > > > > you intend to have these changes in the 3.8.x line?
>>>> > > > > > > > > > > Yeah, we already removed Groovy support in our
>>>> > > > > > > > > > > fork, so it
>>>> > > > > > > > > > is even now in
>>>> > > > > > > > > > > our distribution TX, and the session is the same.
>>>> > > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > > Could you expand a bit more on "Transaction scope
>>>> > > > > > > > > > > > is
>>>> > > > > > > > > > limited to the
>>>> > > > > > > > > > > current thread only".
>>>> > > > > > > > > > > I mean that TX visibility and manipulation are
>>>> > > > > > > > > > > limited to a
>>>> > > > > > > > > > single thread
>>>> > > > > > > > > > > where it is created.
>>>> > > > > > > > > > > If some of the providers support TX visibility
>>>> > > > > > > > > > > between
>>>> > > > > > > > > > several threads,
>>>> > > > > > > > > > > that will only add a bonus for them if such a need
>>>> > > > > > > > > > > arises,
>>>> > > > > > > > > > of course,
>>>> > > > > > > > > > > without breaking the API.
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > > I find it limiting that an attempt to open
>>>> > > > > > > > > > > > multiple
>>>> > > > > > > > > > transactions from
>>>> > > > > > > > > > > the same thread ends up returning the same
>>>> > > > > > > > > > > Transaction
>>>> > > > > > > > > > instance
>>>> > > > > > > > > > > Do you propose to support several transactions in
>>>> > > > > > > > > > > the same
>>>> > > > > > > > > > thread?
>>>> > > > > > > > > > > I am afraid not all current vendors can support
>>>> > > > > > > > > > > this model.
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > On Wed, Dec 3, 2025 at 1:23 PM Stephen Mallette
>>>> > > > > > > > > > <sp...@gmail.com>
>>>> > > > > > > > > > > wrote:
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > > Andrii, I'd like to just quickly clarify Ken's
>>>> > > > > > > > > > > > point
>>>> > > > > > > > > > stating " Sessions
>>>> > > > > > > > > > > > won't exist moving forward (4.x+) ". He's saying
>>>> > > > > > > > > > > > what you
>>>> > > > > > > > > > said: "
>>>> > > > > > > > > > > > 'session'
>>>> > > > > > > > > > > > and 'tx' are essentially the same concepts". We
>>>> > > > > > > > > > acknowledged that and
>>>> > > > > > > > > > > > decided to stop using "session" terminology and
>>>> > > > > > > > > > > > instead
>>>> > > > > > > > > > use "transaction"
>>>> > > > > > > > > > > > terminology only to avoid confusion.
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > > Much of what you wrote is in the spirit of what
>>>> > > > > > > > > > > > has been
>>>> > > > > > > > > > considered for
>>>> > > > > > > > > > > > 4.x, so that is nice alignment. I do think the
>>>> > > > > > > > > > > > goal though
>>>> > > > > > > > > > is to have a
>>>> > > > > > > > > > > > consistent transaction API/system that works for
>>>> > > > > > > > > > > > both
>>>> > > > > > > > > > embedded and remote
>>>> > > > > > > > > > > > cases. It will be interesting to see how that is
>>>> > > > > > > > > > > > achieved.
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > > On Tue, Dec 2, 2025 at 3:53 PM Ken Hu
>>>> > > > > > > > > > > > <kenhuoss@gmail.com
>>>> > > >
>>>> > > > > > > > > > wrote:
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > > > > Hi Andrii,
>>>> > > > > > > > > > > > >
>>>> > > > > > > > > > > > > Good day to you as well. I have a couple
>>>> > > > > > > > > > > > > questions about
>>>> > > > > > > > > > this that I
>>>> > > > > > > > > > > > would
>>>> > > > > > > > > > > > > like some more details on.
>>>> > > > > > > > > > > > >
>>>> > > > > > > > > > > > > 1. What OGMs are you referring to here? Regular
>>>> > > > > > > > > > transactions are
>>>> > > > > > > > > > > > > complicated enough by itself, I would only want
>>>> > > > > > > > > > > > > to
>>>> > > > > > > > > > introduce nested
>>>> > > > > > > > > > > > > transactions if there were well maintained OGMs
>>>> > > > > > > > > > > > > that
>>>> > > > > > > > > > would have a
>>>> > > > > > > > > > > > > demonstrable benefit from this.
>>>> > > > > > > > > > > > > 2. I like some of your ideas about removing
>>>> > > > > > > > > > > > > Transaction
>>>> > > > > > > > > > as its own
>>>> > > > > > > > > > > > class,
>>>> > > > > > > > > > > > > but I'm not sure GraphTraversalSource is where
>>>> > > > > > > > > > > > > it should
>>>> > > > > > > > > > be instead. Do
>>>> > > > > > > > > > > > you
>>>> > > > > > > > > > > > > have some more details about what you think the
>>>> > > > > > > > > > Transaction API should
>>>> > > > > > > > > > > > look
>>>> > > > > > > > > > > > > like?
>>>> > > > > > > > > > > > > 3. Could you expand a bit more on "Transaction
>>>> > > > > > > > > > > > > scope is
>>>> > > > > > > > > > limited to the
>>>> > > > > > > > > > > > > current thread only". I always get confused by
>>>> > > > > > > > > > TinkerPop's threaded vs
>>>> > > > > > > > > > > > > multithreaded transaction. I find it limiting
>>>> > > > > > > > > > > > > that an
>>>> > > > > > > > > > attempt to open
>>>> > > > > > > > > > > > > multiple transactions from the same thread ends
>>>> > > > > > > > > > > > > up
>>>> > > > > > > > > > returning the same
>>>> > > > > > > > > > > > > Transaction instance. Maybe I didn't quite
>>>> > > > > > > > > > > > > understand
>>>> > > > > > > > > > what you meant by
>>>> > > > > > > > > > > > > this.
>>>> > > > > > > > > > > > > 4. Sessions won't exist moving forward (4.x+)
>>>> > > > > > > > > > > > > as they
>>>> > > > > > > > > > are tied to remote
>>>> > > > > > > > > > > > > Groovy execution. So let's continue this
>>>> > > > > > > > > > > > > conversation as
>>>> > > > > > > > > > simply
>>>> > > > > > > > > > > > > transactions and not refer to sessions at all
>>>> > > > > > > > > > > > > to prevent
>>>> > > > > > > > > > confusion,
>>>> > > > > > > > > > > > unless
>>>> > > > > > > > > > > > > you intend to have these changes in the 3.8.x
>>>> > > > > > > > > > > > > line?
>>>> > > > > > > > > > > > >
>>>> > > > > > > > > > > > > Regards,
>>>> > > > > > > > > > > > > Ken
>>>> > > > > > > > > > > > >
>>>> > > > > > > > > > > > >
>>>> > > > > > > > > > > > > On Tue, Dec 2, 2025 at 3:33 AM Andrii Lomakin
>>>> > > > > > > > > > > > > via dev <
>>>> > > > > > > > > > > > > dev@tinkerpop.apache.org> wrote:
>>>> > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > Small clarification.
>>>> > > > > > > > > > > > > > Implementation notes are related to changes
>>>> > > > > > > > > > > > > > in the
>>>> > > > > > > > > > remote protocol.
>>>> > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > On Tue, Dec 2, 2025 at 8:55 AM Andrii Lomakin
>>>> > > > > > > > > > > > > > <
>>>> > > > > > > > > > > > > > andrey.lomakin@jetbrains.com>
>>>> > > > > > > > > > > > > > wrote:
>>>> > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > To clarify: in this model, 'session' and
>>>> > > > > > > > > > > > > > > 'tx' are
>>>> > > > > > > > > > essentially the
>>>> > > > > > > > > > > > same
>>>> > > > > > > > > > > > > > > concepts.
>>>> > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > I also think it is essential to have a
>>>> > > > > > > > > > > > > > > common
>>>> > > > > > > > > > denominator that
>>>> > > > > > > > > > > > makes TP
>>>> > > > > > > > > > > > > > > applications portable across
>>>> > > > > > > > > > > > > > > implementations.
>>>> > > > > > > > > > > > > > > That is why I also propose to restrict
>>>> > > > > > > > > > > > > > > transaction
>>>> > > > > > > > > > scope to the
>>>> > > > > > > > > > > > thread
>>>> > > > > > > > > > > > > > > scope. It will be a common denominator that
>>>> > > > > > > > > > > > > > > will not
>>>> > > > > > > > > > harm user
>>>> > > > > > > > > > > > > experience
>>>> > > > > > > > > > > > > > > but will ensure uniform and expected API
>>>> > > > > > > > > > > > > > > behavior.
>>>> > > > > > > > > > > > > > > There should not be both `begin` and `open`
>>>> > > > > > > > > > > > > > > methods
>>>> > > > > > > > > > because the
>>>> > > > > > > > > > > > > contract
>>>> > > > > > > > > > > > > > > of the beginning method is vague and
>>>> > > > > > > > > > > > > > > unpredictable.
>>>> > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > On Tue, Dec 2, 2025 at 8:18 AM Andrii
>>>> > > > > > > > > > > > > > > Lomakin <
>>>> > > > > > > > > > > > > > > andrey.lomakin@jetbrains.com> wrote:
>>>> > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > > Good day,
>>>> > > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > > I have a proposal to enhance the
>>>> > > > > > > > > > > > > > > > transaction
>>>> > > > > > > > > > handling logic in TP,
>>>> > > > > > > > > > > > > > > > specifically regarding nested transaction
>>>> > > > > > > > > > > > > > > > calls,
>>>> > > > > > > > > > which are common
>>>> > > > > > > > > > > > in
>>>> > > > > > > > > > > > > EE
>>>> > > > > > > > > > > > > > > > development, particularly when utilizing
>>>> > > > > > > > > > > > > > > > OGM.
>>>> > > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > > Currently, calling open multiple times
>>>> > > > > > > > > > > > > > > > causes an
>>>> > > > > > > > > > exception. This
>>>> > > > > > > > > > > > > > behavior
>>>> > > > > > > > > > > > > > > > is restrictive, and the term "open" can
>>>> > > > > > > > > > > > > > > > be
>>>> > > > > > > > > > confusing.
>>>> > > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > > I propose a design that improves the
>>>> > > > > > > > > > > > > > > > developer
>>>> > > > > > > > > > experience by using
>>>> > > > > > > > > > > > an
>>>> > > > > > > > > > > > > > > > internal counter for transaction calls:
>>>> > > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > > Proposed transaction counter logic
>>>> > > > > > > > > > > > > > > > 1. begin/commit counter:
>>>> > > > > > > > > > > > > > > > - begin would increment an internal
>>>> > > > > > > > > > > > > > > > counter.
>>>> > > > > > > > > > > > > > > > - commit would decrement the internal
>>>> > > > > > > > > > > > > > > > counter,
>>>> > > > > > > > > > causing the actual
>>>> > > > > > > > > > > > > > > > transaction commit only when the counter
>>>> > > > > > > > > > > > > > > > reaches
>>>> > > > > > > > > > zero.
>>>> > > > > > > > > > > > > > > > 2. rollback behavior:
>>>> > > > > > > > > > > > > > > > - rollback would immediately force a
>>>> > > > > > > > > > > > > > > > transaction
>>>> > > > > > > > > > rollback and
>>>> > > > > > > > > > > > > > decrements
>>>> > > > > > > > > > > > > > > > the internal counter
>>>> > > > > > > > > > > > > > > > - the subsequent calls to rollback on
>>>> > > > > > > > > > > > > > > > the same
>>>> > > > > > > > > > transaction object
>>>> > > > > > > > > > > > > > should
>>>> > > > > > > > > > > > > > > > be allowed until the stack trace reaches
>>>> > > > > > > > > > > > > > > > the
>>>> > > > > > > > > > initial begin method
>>>> > > > > > > > > > > > > call,
>>>> > > > > > > > > > > > > > > > after which further rollback calls would
>>>> > > > > > > > > > > > > > > > throw an
>>>> > > > > > > > > > exception.
>>>> > > > > > > > > > > > > > > > 3. Error handling:
>>>> > > > > > > > > > > > > > > > - A commit without a matching begin
>>>> > > > > > > > > > > > > > > > (e.g., counter
>>>> > > > > > > > > > is already zero)
>>>> > > > > > > > > > > > > > > > should throw an exception.
>>>> > > > > > > > > > > > > > > > - A rollback without a matching begin
>>>> > > > > > > > > > > > > > > > should throw
>>>> > > > > > > > > > an exception
>>>> > > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > > I would also consider hiding the
>>>> > > > > > > > > > > > > > > > Transaction object
>>>> > > > > > > > > > altogether and
>>>> > > > > > > > > > > > > > > > delegating the logic of transaction
>>>> > > > > > > > > > > > > > > > management to
>>>> > > > > > > > > > > > GraphTraversalSource
>>>> > > > > > > > > > > > > > it
>>>> > > > > > > > > > > > > > > > will simplify the API a lot, both for
>>>> > > > > > > > > > > > > > > > users and
>>>> > > > > > > > > > vendors.
>>>> > > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > > I also propose to restrict transaction
>>>> > > > > > > > > > > > > > > > scope to the
>>>> > > > > > > > > > thread scope,
>>>> > > > > > > > > > > > it
>>>> > > > > > > > > > > > > > will
>>>> > > > > > > > > > > > > > > > be a common denominator that will not
>>>> > > > > > > > > > > > > > > > harm user
>>>> > > > > > > > > > experience but will
>>>> > > > > > > > > > > > > > ensure
>>>> > > > > > > > > > > > > > > > uniform and expected API behavior across
>>>> > > > > > > > > > > > > > > > all
>>>> > > > > > > > > > implementations.
>>>> > > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > > Proposed API changes
>>>> > > > > > > > > > > > > > > > - I propose deprecating open and using
>>>> > > > > > > > > > > > > > > > begin with
>>>> > > > > > > > > > the semantics
>>>> > > > > > > > > > > > > > > > described above to avoid confusion.
>>>> > > > > > > > > > > > > > > > - Transaction object is deprecated, and
>>>> > > > > > > > > > > > > > > > TX control
>>>> > > > > > > > > > functionality is
>>>> > > > > > > > > > > > > > > > delegated to the GraphTraversal/Graph
>>>> > > > > > > > > > > > > > > > instances.
>>>> > > > > > > > > > > > > > > > - tx() method is deprecated.
>>>> > > > > > > > > > > > > > > > - Transaction scope is limited to the
>>>> > > > > > > > > > > > > > > > current
>>>> > > > > > > > > > thread only.
>>>> > > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > > Behavior for non-direct calls
>>>> > > > > > > > > > > > > > > > When begin/commit/rollback methods are
>>>> > > > > > > > > > > > > > > > not called
>>>> > > > > > > > > > directly, the
>>>> > > > > > > > > > > > > > > > transaction should be automatically
>>>> > > > > > > > > > > > > > > > committed by a
>>>> > > > > > > > > > terminal
>>>> > > > > > > > > > > > operation
>>>> > > > > > > > > > > > > > > > (e.g., when hasNext returns false) in
>>>> > > > > > > > > > > > > > > > both remote
>>>> > > > > > > > > > and embedded
>>>> > > > > > > > > > > > modes.
>>>> > > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > > Implementation notes:
>>>> > > > > > > > > > > > > > > > 1. The begin command should be added.
>>>> > > > > > > > > > > > > > > > Relying on
>>>> > > > > > > > > > the implicit
>>>> > > > > > > > > > > > start of
>>>> > > > > > > > > > > > > > TX
>>>> > > > > > > > > > > > > > > > by traversal can lead to non-controlled
>>>> > > > > > > > > > > > > > > > side
>>>> > > > > > > > > > effects.
>>>> > > > > > > > > > > > > > > > 2. For the sake of optimization of the
>>>> > > > > > > > > > > > > > > > remote
>>>> > > > > > > > > > protocol, while
>>>> > > > > > > > > > > > multiple
>>>> > > > > > > > > > > > > > > > calls of begin/commit/rollback are
>>>> > > > > > > > > > > > > > > > allowed in the
>>>> > > > > > > > > > remote protocol,
>>>> > > > > > > > > > > > > > > > practically, it will mean that we will
>>>> > > > > > > > > > > > > > > > track the
>>>> > > > > > > > > > counter locally on
>>>> > > > > > > > > > > > > the
>>>> > > > > > > > > > > > > > > > client and will send
>>>> > > > > > > > > > > > > > > > begin/commit/rollback only
>>>> > > > > > > > > > once for the TX
>>>> > > > > > > > > > > > > > lifecycle.
>>>> > > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > > This model should be uniform across all
>>>> > > > > > > > > > > > > > > > deployment
>>>> > > > > > > > > > types.
>>>> > > > > > > > > > > > > > > > What are your thoughts on this approach?
>>>> > > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > > -----
>>>> > > > > > > > > > > > > > > > Andrii Lomakin
>>>> > > > > > > > > > > > > > > > YouTrackDB development lead
>>>> > > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > > --
>>>> > > > > > > > > > > > > > > Andrii Lomakin
>>>> > > > > > > > > > > > > > > YouTrackDB development lead
>>>> > > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > >
>>>> > > > > > > > > > > > > > --
>>>> > > > > > > > > > > > > > Andrii Lomakin
>>>> > > > > > > > > > > > > > YouTrackDB development lead
>>>> > > > > > > > > > > > > >
>>>> > > > > > > > > > > > >
>>>> > > > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > >
>>>> > > > > > > > > > > --
>>>> > > > > > > > > > > Andrii Lomakin
>>>> > > > > > > > > > > YouTrackDB development lead
>>>> > > > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > > > > > > >
>>>> > > > >
>>>> > > >
>>>> > >
>>>> > >
>>>> > > --
>>>> > > Andrii Lomakin
>>>> > > YouTrackDB development lead
>>>> > >
>>>>
>>>
Re: [DISCUSS] Handling of transactions - proposal
Posted by Ken Hu <ke...@gmail.com>.
We haven't talked about this for a while but I think there were some good
changes that were proposed that we should go ahead with so I've broken some
of the smaller parts out into their own Jiras.
https://issues.apache.org/jira/browse/TINKERPOP-3252 which is removing
open() since it has the same functionality as begin().
https://issues.apache.org/jira/browse/TINKERPOP-3253 which is adding
convenience lambdas. Manual control of transactions still exist and this is
only an addition.
From what I remember, I believe these were mostly agreed to in some form. I
plan to have pull requests for these soon so please provide feedback if you
have any.
On Wed, Feb 18, 2026 at 6:44 PM Ken Hu <ke...@gmail.com> wrote:
> Sorry for not keeping this thread more up to date. I've been playing
> around with some of the ideas from this thread and ones that Pieter
> suggested in the TinkerPop Gatherings but I haven't yet found something I
> find completely satisfactory. Just wanted to let everyone know that I'm
> still looking into this and will post an update when I've found something
> more promising.
>
> On Sun, Jan 25, 2026 at 10:05 PM Ken Hu <ke...@gmail.com> wrote:
>
>> Given that this is a major version upgrade, I think we can break the core
>> interfaces as much as needed as long as there is a good reason for doing
>> so. I understand the hesitancy to make breaking changes, but it's become
>> apparent that the APIs don't work well for the remote case. I also expect
>> that the 3.x line will be maintained for a while even after the release of
>> TinkerPop 4 so that will give users extra time to upgrade. But yes, we
>> should keep the upgrade path in mind.
>>
>> On Fri, Jan 23, 2026 at 11:49 PM pieter <pi...@riseup.net> wrote:
>>
>>> Hi,
>>>
>>> Before discussing these changes further, I need some clarity on how
>>> much we are willing to break core interfaces in TP 4?
>>>
>>> The core interfaces have been around for more than a decade. i.e.
>>> millions and millions of lines of code in production.
>>>
>>> Personally I am in the 'don't break user space'. camp.
>>>
>>> If we do make drastic changes, we should right from the start think
>>> about the upgrade path and think about the impact on existing code
>>> bases.
>>>
>>> Regards
>>> Pieter
>>>
>>>
>>>
>>>
>>> On Wed, 2026-01-21 at 20:40 -0800, Ken Hu wrote:
>>> > I agree that record consistency is important, but I feel that where
>>> > this
>>> > gets solved depends on how users integrate graph databases into their
>>> > applications. There are probably frameworks and OGM/ORMs that can do
>>> > this
>>> > detaching/unloading that you are mentioning as a solution. If a user
>>> > directly integrates the database as an external service using a
>>> > driver,
>>> > then having it as part of the API could be useful. I need to think
>>> > about
>>> > this more.
>>> >
>>> > To your second point about platform-independence, I think this has
>>> > always
>>> > been a goal of TinkerPop that has never quite been realized. Even
>>> > today,
>>> > there are some examples of Gremlin that return results that are
>>> > provider-dependent and so there are specific cases where it isn't
>>> > truly
>>> > portable. I agree that a difference in read consistency amongst
>>> > vendors/providers means those using transactions can't move between
>>> > different databases unless they find another database with the exact
>>> > same
>>> > transaction behavior. But I'm not entirely sure we can get all
>>> > providers to
>>> > match the same behavior in this case.
>>> >
>>> > On Tue, Jan 20, 2026 at 11:26 PM Andrii Lomakin via dev <
>>> > dev@tinkerpop.apache.org> wrote:
>>> >
>>> > > Hi Ken.
>>> > > Thank you for sharing this with us.
>>> > >
>>> > > After some thought, I agree that we need explicit TX management
>>> > > methods.
>>> > > From my perspective, what I wrote is only the tip of the iceberg,
>>> > > as such
>>> > > questions as consistency of the queries and validity of the data
>>> > > contained
>>> > > in vertices directly bound to the concept of TX lifecycle.
>>> > >
>>> > > Example (it can be applied to any database, not just a graph
>>> > > database, so I
>>> > > will use the general record concept).
>>> > >
>>> > > I have created record A { id: 1, value: "val1"} , committed it, and
>>> > > passed
>>> > > it to another thread.
>>> > > After that, another method updated the same record B {id: 1, value
>>> > > :
>>> > > "val2"} and committed it.
>>> > >
>>> > > A and B in this context object identity, not DB record identity.
>>> > >
>>> > > But I have the record A in another thread with potentially stale
>>> > > data.
>>> > > What then should another thread read when it accesses `A.value`?
>>> > >
>>> > > I doubt it should read stale data. In our database, records are
>>> > > automatically switched to the unloaded state after commit and can
>>> > > be read
>>> > > in a new thread only when a new transaction is started.
>>> > > The question of record consistency, from my point of view, is an
>>> > > important
>>> > > one and cannot be overlooked, as it influences how users interact
>>> > > with the
>>> > > TP API and to what degree it is really platform-independent.
>>> > >
>>> > >
>>> > > On Wed, Jan 21, 2026 at 7:58 AM Ken Hu <ke...@gmail.com> wrote:
>>> > >
>>> > > > Here are some of my current thoughts on the proposed changes and
>>> > > > discussion.
>>> > > >
>>> > > > -deprecate open() and replace with begin()
>>> > > > -I agree with this as having both is confusing and unncessary
>>> > > >
>>> > > > -Transaction class is deprecated, and TX control functionality is
>>> > > delegated
>>> > > > to the GraphTraversal/Graph instances
>>> > > > -I mostly agree with this as well, but I think it's probably
>>> > > > better
>>> > > if
>>> > > > it was delegated to GraphTraversalSource.
>>> > > >
>>> > > > -tx() method is deprecated.
>>> > > > -I think this makes sense as I always found how tx() works
>>> > > > with
>>> > > remote
>>> > > > to be unintuitive.
>>> > > >
>>> > > > -Transaction scope is limited to the current thread only
>>> > > > -I had some questions about this earlier, but having thought
>>> > > > about it
>>> > > > after seeing Andrii's architectural diagram, it's probably fine
>>> > > > since
>>> > > most
>>> > > > developers (in all languages TinkerPop supports) will be used to
>>> > > > this
>>> > > being
>>> > > > how it works as long as we have something that represents what a
>>> > > connection
>>> > > > is in other databases.
>>> > > >
>>> > > > -I believe Non-TX mode is not appropriate for handling graph data
>>> > > > structures.
>>> > > > -Further, I'd say that transactions should be optional. Many
>>> > > > persistent
>>> > > > db/graph systems are eventually consistent and do not support
>>> > > transactions,
>>> > > > so they should not be burdened with concepts foreign to them.
>>> > > > -I tend to agree with Pieter here. My reasoning is different
>>> > > > though,
>>> > > > one of the use cases for graphs is analytics and transactions
>>> > > > aren't
>>> > > really
>>> > > > used in that case.
>>> > > >
>>> > > > -I think we DO need methods for explicit transaction management,
>>> > > > but 90%
>>> > > of
>>> > > > the cases will be covered by implicit transaction management
>>> > > > throughout
>>> > > the
>>> > > > GraphTraversal life cycle.
>>> > > > -What I propose is to use lambda-controlled TXs
>>> > > > -My idea is that the transaction api should be unopinionated.
>>> > > > That means
>>> > > it
>>> > > > should follow closely to `UserTransaction`
>>> > > > -I think having implicit transactions is fine as long as it's
>>> > > > just an
>>> > > > option. I'm not sure which should be the default. I'm not sure we
>>> > > > can
>>> > > make
>>> > > > lambda-controlled transactions work though since lambdas wouldn't
>>> > > > be
>>> > > > compatible with Gremlin scripts. At this point, we probably still
>>> > > > want
>>> > > > parity between scripts and GraphTraversals.
>>> > > >
>>> > > > I think we need some more proposals for what the explicit
>>> > > > transaction
>>> > > > management API could look like because we probably can't move
>>> > > > ahead with
>>> > > > lambda-controlled TX for now and the current tx() solution is
>>> > > > clunky.
>>> > > I'll
>>> > > > try to propose one in the next couple of days.
>>> > > >
>>> > > > On Wed, Jan 7, 2026 at 12:24 PM pieter <pi...@riseup.net>
>>> > > > wrote:
>>> > > >
>>> > > > > Hi,
>>> > > > >
>>> > > > > I had a brief discussion with Andrii in the discord chat
>>> > > > > regarding the
>>> > > > > transaction api.
>>> > > > >
>>> > > > > My idea is that the transaction api should be unopinionated.
>>> > > > > That means it should follow closely to
>>> > > > > `javax.transaction.UserTransaction` which in turn means we have
>>> > > > > a
>>> > > > > dedicated class modelling the transaction with `begin()`,
>>> > > > > `commit()`
>>> > > > > and `rollback()`.
>>> > > > >
>>> > > > > My reasoning is that gremlin is a low level graph language.
>>> > > > > There are
>>> > > > > many ways different developers / architects can go about
>>> > > > > integrating
>>> > > > > gremlin into their stack.
>>> > > > >
>>> > > > > Some like lambdas, some like IOT containers, some have domain
>>> > > > > entity
>>> > > > > frameworks, some use aspect orienated programming and on an on.
>>> > > > >
>>> > > > > It is common for IOT containers to manage transaction
>>> > > > > boundaries and
>>> > > > > lifecycles. Similarly it is common to use the web server's
>>> > > > > interceptors
>>> > > > > to manage transaction boundaries. And it is very common to do
>>> > > > > it all
>>> > > > > manually...
>>> > > > >
>>> > > > > Further, I'd say that transactions should be optional. Many
>>> > > > > persistent
>>> > > > > db/graph systems are eventually consistent and do not support
>>> > > > > transactions, so they should not be burdened with concepts
>>> > > > > foreign to
>>> > > > > them.
>>> > > > >
>>> > > > > Regarding binding the transaction to the thread:
>>> > > > > This is the only way I have ever worked so its entirely natural
>>> > > > > to me.
>>> > > > > However I can image that people want to defince their own
>>> > > > > transaction
>>> > > > > context, so perhaps we should not hardcode that part. Long ago
>>> > > > > Gavin
>>> > > > > King from Hibernate fame had the notion of `conversation`
>>> > > > > context.
>>> > > > > Perhaps such things are still around?
>>> > > > >
>>> > > > > Regards
>>> > > > > Pieter
>>> > > > >
>>> > > > > On Thu, 2025-12-11 at 09:02 +0100, Andrii Lomakin via dev
>>> > > > > wrote:
>>> > > > > > Unfortunately, all images have been removed, so I am sending
>>> > > > > > a link
>>> > > > > > to the diagrams instead
>>> > > > > >
>>> https://miro.com/app/board/uXjVGdd8m44=/?share_link_id=686668490384
>>> > > > > > .
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > > On Thu, Dec 11, 2025 at 8:53 AM Andrii Lomakin
>>> > > > > > <an...@jetbrains.com> wrote:
>>> > > > > > > Good day.
>>> > > > > > >
>>> > > > > > > If I followed Ken's line of thought correctly, I would like
>>> > > > > > > to
>>> > > > > > > expand it to be part of the new proposal for the
>>> > > > > > > transaction API.
>>> > > > > > > First of all, about the model, I think we came to a
>>> > > > > > > consensus that
>>> > > > > > > it should be:
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > image.png
>>> > > > > > > The point is that we do not have a notion of connection in
>>> > > > > > > TinkerPop, but we have a notion of GraphTraversal.
>>> > > > > > > So, from my perspective, this diagram can be changed to
>>> > > > > > > image.png
>>> > > > > > >
>>> > > > > > > There is a corner case for Non-TX databases that we need to
>>> > > > > > > discuss.
>>> > > > > > > I believe Non-TX mode is not appropriate for handling graph
>>> > > > > > > data
>>> > > > > > > structures. GQL, for example, requires serializable
>>> > > > > > > isolation to
>>> > > > > > > satisfy the standard requirements.
>>> > > > > > > Instead, we should require to have presence of a
>>> > > > > > > transaction, which
>>> > > > > > > is started implicitly when GraphTraversal is created and
>>> > > > > > > committed
>>> > > > > > > once it is closed.
>>> > > > > > >
>>> > > > > > > I think we DO need methods for explicit transaction
>>> > > > > > > management, but
>>> > > > > > > 90% of the cases will be covered by implicit transaction
>>> > > > > > > management
>>> > > > > > > throughout the GraphTraversal life cycle.
>>> > > > > > > 1. TX is started when GraphTraversal is created.
>>> > > > > > > 2. TX is committed when GraphTraversal is closed (which
>>> > > > > > > includes
>>> > > > > > > terminal operations like toList or iterate).
>>> > > > > > > 3. TX is rolled back in case of exceptional execution of
>>> > > > > > > GraphTraversal. Users can always roll back a transaction by
>>> > > > > > > using
>>> > > > > > > the fail() step.
>>> > > > > > >
>>> > > > > > > We also provide in the YTDB the following methods that I
>>> > > > > > > hope will
>>> > > > > > > become part of the framework: gSource.computeInTx(g ->{}),
>>> > > > > > > gSource.executeInTx(g - {}), gSource.autoExecuteInTx(g -
>>> > > > > > > {}: g)
>>> > > > > > > (this one executes g.iterate() on the returned
>>> > > > > > > GraphTraversal,
>>> > > > > > > which we find really handy).
>>> > > > > > >
>>> > > > > > > Now, regarding explicit TX management.
>>> > > > > > > I think that explicit TX management is still needed if you
>>> > > > > > > want to
>>> > > > > > > execute several queries and mix them with complex logic,
>>> > > > > > > but I
>>> > > > > > > think that methods that explicitly cause TX
>>> > > > > > > begin/start/commit are
>>> > > > > > > confusing and provide bad DevX.
>>> > > > > > > What I propose is to use lambda-controlled TXs (slightly
>>> > > > > > > different
>>> > > > > > > from the above). I think that an explicit rollback/commit
>>> > > > > > > of TX by
>>> > > > > > > the user is similar to the goto operator in old programming
>>> > > > > > > languages; it gives finer control over TXs, but overall
>>> > > > > > > only adds
>>> > > > > > > confusion and can be completely avoided in modern
>>> > > > > > > languages.
>>> > > > > > >
>>> > > > > > > Explicit control on TXs is proposed to be implemented using
>>> > > > > > > methods
>>> > > > > > > similar to above, with a different signature, namely:
>>> > > > > > > gSource.executeInTx(graphTraversalSupplier -> {
>>> > > > > > > var g = graphTraversalSupplier.get();
>>> > > > > > > var v = g.V().next();
>>> > > > > > > g.close()
>>> > > > > > >
>>> > > > > > > v.property("p", "val");
>>> > > > > > > int i = v.value("int");
>>> > > > > > >
>>> > > > > > > if(i > 0) {
>>> > > > > > > g = graphTraversalSupplier.get();
>>> > > > > > > g.V().has("i2", i).property("i2", i +
>>> > > > > > > 1).iterate();
>>> > > > > > > }
>>> > > > > > > })
>>> > > > > > >
>>> > > > > > > In this case, the transaction is detached from
>>> > > > > > > GraphTraversal, and
>>> > > > > > > its lifecycle is not controlled by GraphTraversal.
>>> > > > > > > Instead, TX is committed if an exception is not thrown from
>>> > > > > > > the
>>> > > > > > > lambda method.
>>> > > > > > >
>>> > > > > > > To inform the user if GraphTraversal controls TX or not, a
>>> > > > > > > simple
>>> > > > > > > isTxAttached method is proposed to be introduced
>>> > > > > > > gSource.<V>computeInTX(graphTraversalSupplier -> {}) works
>>> > > > > > > in
>>> > > > > > > similar way but returns value.
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > So the final diagram could look like the following:
>>> > > > > > >
>>> > > > > > >
>>> > > > > > > image.png
>>> > > > > > >
>>> > > > > > > Please note that in this model, there is no need for an
>>> > > > > > > explicit
>>> > > > > > > transaction object.
>>> > > > > > >
>>> > > > > > > On Fri, Dec 5, 2025 at 10:18 AM Andrii Lomakin
>>> > > > > > > <an...@jetbrains.com> wrote:
>>> > > > > > > > Good day, Ken.
>>> > > > > > > >
>>> > > > > > > > > This could map to multiple threads on the provider
>>> > > > > > > > > side. It's
>>> > > > > > > > probably OK if something like "g.tx()..." returns the
>>> > > > > > > > same
>>> > > > > > > > transaction. We could require users to open multiple
>>> > > > > > > > DriverRemoteConnections/Clients to achieve this and that
>>> > > > > > > > would be
>>> > > > > > > > similar to other data connectivity APIs.
>>> > > > > > > >
>>> > > > > > > > Unfortunately I do not follow, could you elaborate more
>>> > > > > > > > or give
>>> > > > > > > > me example how do you see it ?
>>> > > > > > > >
>>> > > > > > > > On Thu, Dec 4, 2025 at 7:06 PM Ken Hu
>>> > > > > > > > <ke...@gmail.com>
>>> > > wrote:
>>> > > > > > > > > I might be thinking about this too much for just the
>>> > > > > > > > > remote
>>> > > > > > > > > case and not considering the embedded case enough. In
>>> > > > > > > > > my mind,
>>> > > > > > > > > there just needs to be an easy way for users of the
>>> > > > > > > > > language
>>> > > > > > > > > variants (in particular JavaScript because of its
>>> > > > > > > > > single
>>> > > > > > > > > threaded nature) to start multiple transactions from a
>>> > > > > > > > > single
>>> > > > > > > > > thread. This could map to multiple threads on the
>>> > > > > > > > > provider
>>> > > > > > > > > side. It's probably OK if something like "g.tx()..."
>>> > > > > > > > > returns
>>> > > > > > > > > the same transaction. We could require users to open
>>> > > > > > > > > multiple
>>> > > > > > > > > DriverRemoteConnections/Clients to achieve this and
>>> > > > > > > > > that would
>>> > > > > > > > > be similar to other data connectivity APIs.
>>> > > > > > > > >
>>> > > > > > > > > On Thu, Dec 4, 2025 at 2:37 AM Andrii Lomakin via dev
>>> > > > > > > > > <de...@tinkerpop.apache.org> wrote:
>>> > > > > > > > > > The point about having multiple concurrent
>>> > > > > > > > > > transactions
>>> > > > > > > > > > within the same
>>> > > > > > > > > > thread warrants a more detailed discussion.
>>> > > > > > > > > >
>>> > > > > > > > > > To compare this behavior, consider how major SQL
>>> > > > > > > > > > databases
>>> > > > > > > > > > handle attempts
>>> > > > > > > > > > to start a second transaction when one is already
>>> > > > > > > > > > active:
>>> > > > > > > > > >
>>> > > > > > > > > > 1. Error out: The database prevents the operation,
>>> > > > > > > > > > signaling
>>> > > > > > > > > > that a
>>> > > > > > > > > > transaction is already active (e.g., "Transaction
>>> > > > > > > > > > already
>>> > > > > > > > > > active").
>>> > > > > > > > > > 2. Create a nested transaction (savepoint): The
>>> > > > > > > > > > database
>>> > > > > > > > > > creates a
>>> > > > > > > > > > sub-scope of the first transaction, not an
>>> > > > > > > > > > independent
>>> > > > > > > > > > transaction. If the
>>> > > > > > > > > > parent transaction fails, the child scope also fails.
>>> > > > > > > > > > 3. Implicitly commit the first: The database
>>> > > > > > > > > > automatically
>>> > > > > > > > > > commits the
>>> > > > > > > > > > first transaction to open the second one (a common
>>> > > > > > > > > > behavior
>>> > > > > > > > > > in MySQL DDL).
>>> > > > > > > > > >
>>> > > > > > > > > > As far as I know, only Firebird has historically
>>> > > > > > > > > > allowed
>>> > > > > > > > > > several truly
>>> > > > > > > > > > independent transactions concurrently on the same
>>> > > > > > > > > > thread. I
>>> > > > > > > > > > suspect support
>>> > > > > > > > > > for simultaneous transactions may be even more
>>> > > > > > > > > > limited among
>>> > > > > > > > > > graph database
>>> > > > > > > > > > vendors.
>>> > > > > > > > > >
>>> > > > > > > > > > On Thu, Dec 4, 2025 at 10:47 AM Andrii Lomakin
>>> > > > > > > > > > <an...@jetbrains.com>
>>> > > > > > > > > > wrote:
>>> > > > > > > > > >
>>> > > > > > > > > > > Good day.
>>> > > > > > > > > > > As for questions from Ken
>>> > > > > > > > > > > > What OGMs are you referring to here? Regular
>>> > > > > > > > > > > > transactions
>>> > > > > > > > > > are complicated
>>> > > > > > > > > > > enough by itself, I would only want to introduce
>>> > > > > > > > > > > nested
>>> > > > > > > > > > transactions if
>>> > > > > > > > > > > there were well maintained OGMs that would have a
>>> > > > > > > > > > demonstrable benefit from
>>> > > > > > > > > > > this.
>>> > > > > > > > > > >
>>> > > > > > > > > > > We have such an OGM using TinkerPop that is well
>>> > > > > > > > > > maintained, if you mean
>>> > > > > > > > > > > by this years of support and thousands of instances
>>> > > > > > > > > > > running
>>> > > > > > > > > > on OS version.
>>> > > > > > > > > > > However, I have given it thought and believe that
>>> > > > > > > > > > > it is a
>>> > > > > > > > > > relatively rare
>>> > > > > > > > > > > case, and OGM is so complex that it can manage
>>> > > > > > > > > > > nested
>>> > > > > > > > > > transactions itself
>>> > > > > > > > > > > if needed.
>>> > > > > > > > > > > I think we can support nested transactions on pause
>>> > > > > > > > > > > for
>>> > > > > > > > > > now, till we
>>> > > > > > > > > > > receive more feedback from users.
>>> > > > > > > > > > >
>>> > > > > > > > > > > There are many inconsistencies to discuss in the TX
>>> > > > > > > > > > lifecycle, even
>>> > > > > > > > > > > without nested transactions.
>>> > > > > > > > > > >
>>> > > > > > > > > > > > Sessions won't exist moving forward (4.x+) as
>>> > > > > > > > > > > > they are
>>> > > > > > > > > > tied to remote
>>> > > > > > > > > > > Groovy execution. So let's continue this
>>> > > > > > > > > > > conversation as
>>> > > > > > > > > > simply
>>> > > > > > > > > > > transactions and not refer to sessions at all to
>>> > > > > > > > > > > prevent
>>> > > > > > > > > > confusion, unless
>>> > > > > > > > > > > you intend to have these changes in the 3.8.x line?
>>> > > > > > > > > > > Yeah, we already removed Groovy support in our
>>> > > > > > > > > > > fork, so it
>>> > > > > > > > > > is even now in
>>> > > > > > > > > > > our distribution TX, and the session is the same.
>>> > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > > Could you expand a bit more on "Transaction scope
>>> > > > > > > > > > > > is
>>> > > > > > > > > > limited to the
>>> > > > > > > > > > > current thread only".
>>> > > > > > > > > > > I mean that TX visibility and manipulation are
>>> > > > > > > > > > > limited to a
>>> > > > > > > > > > single thread
>>> > > > > > > > > > > where it is created.
>>> > > > > > > > > > > If some of the providers support TX visibility
>>> > > > > > > > > > > between
>>> > > > > > > > > > several threads,
>>> > > > > > > > > > > that will only add a bonus for them if such a need
>>> > > > > > > > > > > arises,
>>> > > > > > > > > > of course,
>>> > > > > > > > > > > without breaking the API.
>>> > > > > > > > > > >
>>> > > > > > > > > > > > I find it limiting that an attempt to open
>>> > > > > > > > > > > > multiple
>>> > > > > > > > > > transactions from
>>> > > > > > > > > > > the same thread ends up returning the same
>>> > > > > > > > > > > Transaction
>>> > > > > > > > > > instance
>>> > > > > > > > > > > Do you propose to support several transactions in
>>> > > > > > > > > > > the same
>>> > > > > > > > > > thread?
>>> > > > > > > > > > > I am afraid not all current vendors can support
>>> > > > > > > > > > > this model.
>>> > > > > > > > > > >
>>> > > > > > > > > > > On Wed, Dec 3, 2025 at 1:23 PM Stephen Mallette
>>> > > > > > > > > > <sp...@gmail.com>
>>> > > > > > > > > > > wrote:
>>> > > > > > > > > > >
>>> > > > > > > > > > > > Andrii, I'd like to just quickly clarify Ken's
>>> > > > > > > > > > > > point
>>> > > > > > > > > > stating " Sessions
>>> > > > > > > > > > > > won't exist moving forward (4.x+) ". He's saying
>>> > > > > > > > > > > > what you
>>> > > > > > > > > > said: "
>>> > > > > > > > > > > > 'session'
>>> > > > > > > > > > > > and 'tx' are essentially the same concepts". We
>>> > > > > > > > > > acknowledged that and
>>> > > > > > > > > > > > decided to stop using "session" terminology and
>>> > > > > > > > > > > > instead
>>> > > > > > > > > > use "transaction"
>>> > > > > > > > > > > > terminology only to avoid confusion.
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > Much of what you wrote is in the spirit of what
>>> > > > > > > > > > > > has been
>>> > > > > > > > > > considered for
>>> > > > > > > > > > > > 4.x, so that is nice alignment. I do think the
>>> > > > > > > > > > > > goal though
>>> > > > > > > > > > is to have a
>>> > > > > > > > > > > > consistent transaction API/system that works for
>>> > > > > > > > > > > > both
>>> > > > > > > > > > embedded and remote
>>> > > > > > > > > > > > cases. It will be interesting to see how that is
>>> > > > > > > > > > > > achieved.
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > On Tue, Dec 2, 2025 at 3:53 PM Ken Hu
>>> > > > > > > > > > > > <kenhuoss@gmail.com
>>> > > >
>>> > > > > > > > > > wrote:
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > > Hi Andrii,
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > Good day to you as well. I have a couple
>>> > > > > > > > > > > > > questions about
>>> > > > > > > > > > this that I
>>> > > > > > > > > > > > would
>>> > > > > > > > > > > > > like some more details on.
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > 1. What OGMs are you referring to here? Regular
>>> > > > > > > > > > transactions are
>>> > > > > > > > > > > > > complicated enough by itself, I would only want
>>> > > > > > > > > > > > > to
>>> > > > > > > > > > introduce nested
>>> > > > > > > > > > > > > transactions if there were well maintained OGMs
>>> > > > > > > > > > > > > that
>>> > > > > > > > > > would have a
>>> > > > > > > > > > > > > demonstrable benefit from this.
>>> > > > > > > > > > > > > 2. I like some of your ideas about removing
>>> > > > > > > > > > > > > Transaction
>>> > > > > > > > > > as its own
>>> > > > > > > > > > > > class,
>>> > > > > > > > > > > > > but I'm not sure GraphTraversalSource is where
>>> > > > > > > > > > > > > it should
>>> > > > > > > > > > be instead. Do
>>> > > > > > > > > > > > you
>>> > > > > > > > > > > > > have some more details about what you think the
>>> > > > > > > > > > Transaction API should
>>> > > > > > > > > > > > look
>>> > > > > > > > > > > > > like?
>>> > > > > > > > > > > > > 3. Could you expand a bit more on "Transaction
>>> > > > > > > > > > > > > scope is
>>> > > > > > > > > > limited to the
>>> > > > > > > > > > > > > current thread only". I always get confused by
>>> > > > > > > > > > TinkerPop's threaded vs
>>> > > > > > > > > > > > > multithreaded transaction. I find it limiting
>>> > > > > > > > > > > > > that an
>>> > > > > > > > > > attempt to open
>>> > > > > > > > > > > > > multiple transactions from the same thread ends
>>> > > > > > > > > > > > > up
>>> > > > > > > > > > returning the same
>>> > > > > > > > > > > > > Transaction instance. Maybe I didn't quite
>>> > > > > > > > > > > > > understand
>>> > > > > > > > > > what you meant by
>>> > > > > > > > > > > > > this.
>>> > > > > > > > > > > > > 4. Sessions won't exist moving forward (4.x+)
>>> > > > > > > > > > > > > as they
>>> > > > > > > > > > are tied to remote
>>> > > > > > > > > > > > > Groovy execution. So let's continue this
>>> > > > > > > > > > > > > conversation as
>>> > > > > > > > > > simply
>>> > > > > > > > > > > > > transactions and not refer to sessions at all
>>> > > > > > > > > > > > > to prevent
>>> > > > > > > > > > confusion,
>>> > > > > > > > > > > > unless
>>> > > > > > > > > > > > > you intend to have these changes in the 3.8.x
>>> > > > > > > > > > > > > line?
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > Regards,
>>> > > > > > > > > > > > > Ken
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > On Tue, Dec 2, 2025 at 3:33 AM Andrii Lomakin
>>> > > > > > > > > > > > > via dev <
>>> > > > > > > > > > > > > dev@tinkerpop.apache.org> wrote:
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > > Small clarification.
>>> > > > > > > > > > > > > > Implementation notes are related to changes
>>> > > > > > > > > > > > > > in the
>>> > > > > > > > > > remote protocol.
>>> > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > On Tue, Dec 2, 2025 at 8:55 AM Andrii Lomakin
>>> > > > > > > > > > > > > > <
>>> > > > > > > > > > > > > > andrey.lomakin@jetbrains.com>
>>> > > > > > > > > > > > > > wrote:
>>> > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > To clarify: in this model, 'session' and
>>> > > > > > > > > > > > > > > 'tx' are
>>> > > > > > > > > > essentially the
>>> > > > > > > > > > > > same
>>> > > > > > > > > > > > > > > concepts.
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > I also think it is essential to have a
>>> > > > > > > > > > > > > > > common
>>> > > > > > > > > > denominator that
>>> > > > > > > > > > > > makes TP
>>> > > > > > > > > > > > > > > applications portable across
>>> > > > > > > > > > > > > > > implementations.
>>> > > > > > > > > > > > > > > That is why I also propose to restrict
>>> > > > > > > > > > > > > > > transaction
>>> > > > > > > > > > scope to the
>>> > > > > > > > > > > > thread
>>> > > > > > > > > > > > > > > scope. It will be a common denominator that
>>> > > > > > > > > > > > > > > will not
>>> > > > > > > > > > harm user
>>> > > > > > > > > > > > > experience
>>> > > > > > > > > > > > > > > but will ensure uniform and expected API
>>> > > > > > > > > > > > > > > behavior.
>>> > > > > > > > > > > > > > > There should not be both `begin` and `open`
>>> > > > > > > > > > > > > > > methods
>>> > > > > > > > > > because the
>>> > > > > > > > > > > > > contract
>>> > > > > > > > > > > > > > > of the beginning method is vague and
>>> > > > > > > > > > > > > > > unpredictable.
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > On Tue, Dec 2, 2025 at 8:18 AM Andrii
>>> > > > > > > > > > > > > > > Lomakin <
>>> > > > > > > > > > > > > > > andrey.lomakin@jetbrains.com> wrote:
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > > Good day,
>>> > > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > > I have a proposal to enhance the
>>> > > > > > > > > > > > > > > > transaction
>>> > > > > > > > > > handling logic in TP,
>>> > > > > > > > > > > > > > > > specifically regarding nested transaction
>>> > > > > > > > > > > > > > > > calls,
>>> > > > > > > > > > which are common
>>> > > > > > > > > > > > in
>>> > > > > > > > > > > > > EE
>>> > > > > > > > > > > > > > > > development, particularly when utilizing
>>> > > > > > > > > > > > > > > > OGM.
>>> > > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > > Currently, calling open multiple times
>>> > > > > > > > > > > > > > > > causes an
>>> > > > > > > > > > exception. This
>>> > > > > > > > > > > > > > behavior
>>> > > > > > > > > > > > > > > > is restrictive, and the term "open" can
>>> > > > > > > > > > > > > > > > be
>>> > > > > > > > > > confusing.
>>> > > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > > I propose a design that improves the
>>> > > > > > > > > > > > > > > > developer
>>> > > > > > > > > > experience by using
>>> > > > > > > > > > > > an
>>> > > > > > > > > > > > > > > > internal counter for transaction calls:
>>> > > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > > Proposed transaction counter logic
>>> > > > > > > > > > > > > > > > 1. begin/commit counter:
>>> > > > > > > > > > > > > > > > - begin would increment an internal
>>> > > > > > > > > > > > > > > > counter.
>>> > > > > > > > > > > > > > > > - commit would decrement the internal
>>> > > > > > > > > > > > > > > > counter,
>>> > > > > > > > > > causing the actual
>>> > > > > > > > > > > > > > > > transaction commit only when the counter
>>> > > > > > > > > > > > > > > > reaches
>>> > > > > > > > > > zero.
>>> > > > > > > > > > > > > > > > 2. rollback behavior:
>>> > > > > > > > > > > > > > > > - rollback would immediately force a
>>> > > > > > > > > > > > > > > > transaction
>>> > > > > > > > > > rollback and
>>> > > > > > > > > > > > > > decrements
>>> > > > > > > > > > > > > > > > the internal counter
>>> > > > > > > > > > > > > > > > - the subsequent calls to rollback on
>>> > > > > > > > > > > > > > > > the same
>>> > > > > > > > > > transaction object
>>> > > > > > > > > > > > > > should
>>> > > > > > > > > > > > > > > > be allowed until the stack trace reaches
>>> > > > > > > > > > > > > > > > the
>>> > > > > > > > > > initial begin method
>>> > > > > > > > > > > > > call,
>>> > > > > > > > > > > > > > > > after which further rollback calls would
>>> > > > > > > > > > > > > > > > throw an
>>> > > > > > > > > > exception.
>>> > > > > > > > > > > > > > > > 3. Error handling:
>>> > > > > > > > > > > > > > > > - A commit without a matching begin
>>> > > > > > > > > > > > > > > > (e.g., counter
>>> > > > > > > > > > is already zero)
>>> > > > > > > > > > > > > > > > should throw an exception.
>>> > > > > > > > > > > > > > > > - A rollback without a matching begin
>>> > > > > > > > > > > > > > > > should throw
>>> > > > > > > > > > an exception
>>> > > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > > I would also consider hiding the
>>> > > > > > > > > > > > > > > > Transaction object
>>> > > > > > > > > > altogether and
>>> > > > > > > > > > > > > > > > delegating the logic of transaction
>>> > > > > > > > > > > > > > > > management to
>>> > > > > > > > > > > > GraphTraversalSource
>>> > > > > > > > > > > > > > it
>>> > > > > > > > > > > > > > > > will simplify the API a lot, both for
>>> > > > > > > > > > > > > > > > users and
>>> > > > > > > > > > vendors.
>>> > > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > > I also propose to restrict transaction
>>> > > > > > > > > > > > > > > > scope to the
>>> > > > > > > > > > thread scope,
>>> > > > > > > > > > > > it
>>> > > > > > > > > > > > > > will
>>> > > > > > > > > > > > > > > > be a common denominator that will not
>>> > > > > > > > > > > > > > > > harm user
>>> > > > > > > > > > experience but will
>>> > > > > > > > > > > > > > ensure
>>> > > > > > > > > > > > > > > > uniform and expected API behavior across
>>> > > > > > > > > > > > > > > > all
>>> > > > > > > > > > implementations.
>>> > > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > > Proposed API changes
>>> > > > > > > > > > > > > > > > - I propose deprecating open and using
>>> > > > > > > > > > > > > > > > begin with
>>> > > > > > > > > > the semantics
>>> > > > > > > > > > > > > > > > described above to avoid confusion.
>>> > > > > > > > > > > > > > > > - Transaction object is deprecated, and
>>> > > > > > > > > > > > > > > > TX control
>>> > > > > > > > > > functionality is
>>> > > > > > > > > > > > > > > > delegated to the GraphTraversal/Graph
>>> > > > > > > > > > > > > > > > instances.
>>> > > > > > > > > > > > > > > > - tx() method is deprecated.
>>> > > > > > > > > > > > > > > > - Transaction scope is limited to the
>>> > > > > > > > > > > > > > > > current
>>> > > > > > > > > > thread only.
>>> > > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > > Behavior for non-direct calls
>>> > > > > > > > > > > > > > > > When begin/commit/rollback methods are
>>> > > > > > > > > > > > > > > > not called
>>> > > > > > > > > > directly, the
>>> > > > > > > > > > > > > > > > transaction should be automatically
>>> > > > > > > > > > > > > > > > committed by a
>>> > > > > > > > > > terminal
>>> > > > > > > > > > > > operation
>>> > > > > > > > > > > > > > > > (e.g., when hasNext returns false) in
>>> > > > > > > > > > > > > > > > both remote
>>> > > > > > > > > > and embedded
>>> > > > > > > > > > > > modes.
>>> > > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > > Implementation notes:
>>> > > > > > > > > > > > > > > > 1. The begin command should be added.
>>> > > > > > > > > > > > > > > > Relying on
>>> > > > > > > > > > the implicit
>>> > > > > > > > > > > > start of
>>> > > > > > > > > > > > > > TX
>>> > > > > > > > > > > > > > > > by traversal can lead to non-controlled
>>> > > > > > > > > > > > > > > > side
>>> > > > > > > > > > effects.
>>> > > > > > > > > > > > > > > > 2. For the sake of optimization of the
>>> > > > > > > > > > > > > > > > remote
>>> > > > > > > > > > protocol, while
>>> > > > > > > > > > > > multiple
>>> > > > > > > > > > > > > > > > calls of begin/commit/rollback are
>>> > > > > > > > > > > > > > > > allowed in the
>>> > > > > > > > > > remote protocol,
>>> > > > > > > > > > > > > > > > practically, it will mean that we will
>>> > > > > > > > > > > > > > > > track the
>>> > > > > > > > > > counter locally on
>>> > > > > > > > > > > > > the
>>> > > > > > > > > > > > > > > > client and will send
>>> > > > > > > > > > > > > > > > begin/commit/rollback only
>>> > > > > > > > > > once for the TX
>>> > > > > > > > > > > > > > lifecycle.
>>> > > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > > This model should be uniform across all
>>> > > > > > > > > > > > > > > > deployment
>>> > > > > > > > > > types.
>>> > > > > > > > > > > > > > > > What are your thoughts on this approach?
>>> > > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > > -----
>>> > > > > > > > > > > > > > > > Andrii Lomakin
>>> > > > > > > > > > > > > > > > YouTrackDB development lead
>>> > > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > > --
>>> > > > > > > > > > > > > > > Andrii Lomakin
>>> > > > > > > > > > > > > > > YouTrackDB development lead
>>> > > > > > > > > > > > > > >
>>> > > > > > > > > > > > > >
>>> > > > > > > > > > > > > >
>>> > > > > > > > > > > > > > --
>>> > > > > > > > > > > > > > Andrii Lomakin
>>> > > > > > > > > > > > > > YouTrackDB development lead
>>> > > > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > --
>>> > > > > > > > > > > Andrii Lomakin
>>> > > > > > > > > > > YouTrackDB development lead
>>> > > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> > >
>>> > > --
>>> > > Andrii Lomakin
>>> > > YouTrackDB development lead
>>> > >
>>>
>>
Re: [DISCUSS] Handling of transactions - proposal
Posted by Ken Hu <ke...@gmail.com>.
Sorry for not keeping this thread more up to date. I've been playing around
with some of the ideas from this thread and ones that Pieter suggested in
the TinkerPop Gatherings but I haven't yet found something I find
completely satisfactory. Just wanted to let everyone know that I'm still
looking into this and will post an update when I've found something more
promising.
On Sun, Jan 25, 2026 at 10:05 PM Ken Hu <ke...@gmail.com> wrote:
> Given that this is a major version upgrade, I think we can break the core
> interfaces as much as needed as long as there is a good reason for doing
> so. I understand the hesitancy to make breaking changes, but it's become
> apparent that the APIs don't work well for the remote case. I also expect
> that the 3.x line will be maintained for a while even after the release of
> TinkerPop 4 so that will give users extra time to upgrade. But yes, we
> should keep the upgrade path in mind.
>
> On Fri, Jan 23, 2026 at 11:49 PM pieter <pi...@riseup.net> wrote:
>
>> Hi,
>>
>> Before discussing these changes further, I need some clarity on how
>> much we are willing to break core interfaces in TP 4?
>>
>> The core interfaces have been around for more than a decade. i.e.
>> millions and millions of lines of code in production.
>>
>> Personally I am in the 'don't break user space'. camp.
>>
>> If we do make drastic changes, we should right from the start think
>> about the upgrade path and think about the impact on existing code
>> bases.
>>
>> Regards
>> Pieter
>>
>>
>>
>>
>> On Wed, 2026-01-21 at 20:40 -0800, Ken Hu wrote:
>> > I agree that record consistency is important, but I feel that where
>> > this
>> > gets solved depends on how users integrate graph databases into their
>> > applications. There are probably frameworks and OGM/ORMs that can do
>> > this
>> > detaching/unloading that you are mentioning as a solution. If a user
>> > directly integrates the database as an external service using a
>> > driver,
>> > then having it as part of the API could be useful. I need to think
>> > about
>> > this more.
>> >
>> > To your second point about platform-independence, I think this has
>> > always
>> > been a goal of TinkerPop that has never quite been realized. Even
>> > today,
>> > there are some examples of Gremlin that return results that are
>> > provider-dependent and so there are specific cases where it isn't
>> > truly
>> > portable. I agree that a difference in read consistency amongst
>> > vendors/providers means those using transactions can't move between
>> > different databases unless they find another database with the exact
>> > same
>> > transaction behavior. But I'm not entirely sure we can get all
>> > providers to
>> > match the same behavior in this case.
>> >
>> > On Tue, Jan 20, 2026 at 11:26 PM Andrii Lomakin via dev <
>> > dev@tinkerpop.apache.org> wrote:
>> >
>> > > Hi Ken.
>> > > Thank you for sharing this with us.
>> > >
>> > > After some thought, I agree that we need explicit TX management
>> > > methods.
>> > > From my perspective, what I wrote is only the tip of the iceberg,
>> > > as such
>> > > questions as consistency of the queries and validity of the data
>> > > contained
>> > > in vertices directly bound to the concept of TX lifecycle.
>> > >
>> > > Example (it can be applied to any database, not just a graph
>> > > database, so I
>> > > will use the general record concept).
>> > >
>> > > I have created record A { id: 1, value: "val1"} , committed it, and
>> > > passed
>> > > it to another thread.
>> > > After that, another method updated the same record B {id: 1, value
>> > > :
>> > > "val2"} and committed it.
>> > >
>> > > A and B in this context object identity, not DB record identity.
>> > >
>> > > But I have the record A in another thread with potentially stale
>> > > data.
>> > > What then should another thread read when it accesses `A.value`?
>> > >
>> > > I doubt it should read stale data. In our database, records are
>> > > automatically switched to the unloaded state after commit and can
>> > > be read
>> > > in a new thread only when a new transaction is started.
>> > > The question of record consistency, from my point of view, is an
>> > > important
>> > > one and cannot be overlooked, as it influences how users interact
>> > > with the
>> > > TP API and to what degree it is really platform-independent.
>> > >
>> > >
>> > > On Wed, Jan 21, 2026 at 7:58 AM Ken Hu <ke...@gmail.com> wrote:
>> > >
>> > > > Here are some of my current thoughts on the proposed changes and
>> > > > discussion.
>> > > >
>> > > > -deprecate open() and replace with begin()
>> > > > -I agree with this as having both is confusing and unncessary
>> > > >
>> > > > -Transaction class is deprecated, and TX control functionality is
>> > > delegated
>> > > > to the GraphTraversal/Graph instances
>> > > > -I mostly agree with this as well, but I think it's probably
>> > > > better
>> > > if
>> > > > it was delegated to GraphTraversalSource.
>> > > >
>> > > > -tx() method is deprecated.
>> > > > -I think this makes sense as I always found how tx() works
>> > > > with
>> > > remote
>> > > > to be unintuitive.
>> > > >
>> > > > -Transaction scope is limited to the current thread only
>> > > > -I had some questions about this earlier, but having thought
>> > > > about it
>> > > > after seeing Andrii's architectural diagram, it's probably fine
>> > > > since
>> > > most
>> > > > developers (in all languages TinkerPop supports) will be used to
>> > > > this
>> > > being
>> > > > how it works as long as we have something that represents what a
>> > > connection
>> > > > is in other databases.
>> > > >
>> > > > -I believe Non-TX mode is not appropriate for handling graph data
>> > > > structures.
>> > > > -Further, I'd say that transactions should be optional. Many
>> > > > persistent
>> > > > db/graph systems are eventually consistent and do not support
>> > > transactions,
>> > > > so they should not be burdened with concepts foreign to them.
>> > > > -I tend to agree with Pieter here. My reasoning is different
>> > > > though,
>> > > > one of the use cases for graphs is analytics and transactions
>> > > > aren't
>> > > really
>> > > > used in that case.
>> > > >
>> > > > -I think we DO need methods for explicit transaction management,
>> > > > but 90%
>> > > of
>> > > > the cases will be covered by implicit transaction management
>> > > > throughout
>> > > the
>> > > > GraphTraversal life cycle.
>> > > > -What I propose is to use lambda-controlled TXs
>> > > > -My idea is that the transaction api should be unopinionated.
>> > > > That means
>> > > it
>> > > > should follow closely to `UserTransaction`
>> > > > -I think having implicit transactions is fine as long as it's
>> > > > just an
>> > > > option. I'm not sure which should be the default. I'm not sure we
>> > > > can
>> > > make
>> > > > lambda-controlled transactions work though since lambdas wouldn't
>> > > > be
>> > > > compatible with Gremlin scripts. At this point, we probably still
>> > > > want
>> > > > parity between scripts and GraphTraversals.
>> > > >
>> > > > I think we need some more proposals for what the explicit
>> > > > transaction
>> > > > management API could look like because we probably can't move
>> > > > ahead with
>> > > > lambda-controlled TX for now and the current tx() solution is
>> > > > clunky.
>> > > I'll
>> > > > try to propose one in the next couple of days.
>> > > >
>> > > > On Wed, Jan 7, 2026 at 12:24 PM pieter <pi...@riseup.net>
>> > > > wrote:
>> > > >
>> > > > > Hi,
>> > > > >
>> > > > > I had a brief discussion with Andrii in the discord chat
>> > > > > regarding the
>> > > > > transaction api.
>> > > > >
>> > > > > My idea is that the transaction api should be unopinionated.
>> > > > > That means it should follow closely to
>> > > > > `javax.transaction.UserTransaction` which in turn means we have
>> > > > > a
>> > > > > dedicated class modelling the transaction with `begin()`,
>> > > > > `commit()`
>> > > > > and `rollback()`.
>> > > > >
>> > > > > My reasoning is that gremlin is a low level graph language.
>> > > > > There are
>> > > > > many ways different developers / architects can go about
>> > > > > integrating
>> > > > > gremlin into their stack.
>> > > > >
>> > > > > Some like lambdas, some like IOT containers, some have domain
>> > > > > entity
>> > > > > frameworks, some use aspect orienated programming and on an on.
>> > > > >
>> > > > > It is common for IOT containers to manage transaction
>> > > > > boundaries and
>> > > > > lifecycles. Similarly it is common to use the web server's
>> > > > > interceptors
>> > > > > to manage transaction boundaries. And it is very common to do
>> > > > > it all
>> > > > > manually...
>> > > > >
>> > > > > Further, I'd say that transactions should be optional. Many
>> > > > > persistent
>> > > > > db/graph systems are eventually consistent and do not support
>> > > > > transactions, so they should not be burdened with concepts
>> > > > > foreign to
>> > > > > them.
>> > > > >
>> > > > > Regarding binding the transaction to the thread:
>> > > > > This is the only way I have ever worked so its entirely natural
>> > > > > to me.
>> > > > > However I can image that people want to defince their own
>> > > > > transaction
>> > > > > context, so perhaps we should not hardcode that part. Long ago
>> > > > > Gavin
>> > > > > King from Hibernate fame had the notion of `conversation`
>> > > > > context.
>> > > > > Perhaps such things are still around?
>> > > > >
>> > > > > Regards
>> > > > > Pieter
>> > > > >
>> > > > > On Thu, 2025-12-11 at 09:02 +0100, Andrii Lomakin via dev
>> > > > > wrote:
>> > > > > > Unfortunately, all images have been removed, so I am sending
>> > > > > > a link
>> > > > > > to the diagrams instead
>> > > > > >
>> https://miro.com/app/board/uXjVGdd8m44=/?share_link_id=686668490384
>> > > > > > .
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Thu, Dec 11, 2025 at 8:53 AM Andrii Lomakin
>> > > > > > <an...@jetbrains.com> wrote:
>> > > > > > > Good day.
>> > > > > > >
>> > > > > > > If I followed Ken's line of thought correctly, I would like
>> > > > > > > to
>> > > > > > > expand it to be part of the new proposal for the
>> > > > > > > transaction API.
>> > > > > > > First of all, about the model, I think we came to a
>> > > > > > > consensus that
>> > > > > > > it should be:
>> > > > > > >
>> > > > > > >
>> > > > > > > image.png
>> > > > > > > The point is that we do not have a notion of connection in
>> > > > > > > TinkerPop, but we have a notion of GraphTraversal.
>> > > > > > > So, from my perspective, this diagram can be changed to
>> > > > > > > image.png
>> > > > > > >
>> > > > > > > There is a corner case for Non-TX databases that we need to
>> > > > > > > discuss.
>> > > > > > > I believe Non-TX mode is not appropriate for handling graph
>> > > > > > > data
>> > > > > > > structures. GQL, for example, requires serializable
>> > > > > > > isolation to
>> > > > > > > satisfy the standard requirements.
>> > > > > > > Instead, we should require to have presence of a
>> > > > > > > transaction, which
>> > > > > > > is started implicitly when GraphTraversal is created and
>> > > > > > > committed
>> > > > > > > once it is closed.
>> > > > > > >
>> > > > > > > I think we DO need methods for explicit transaction
>> > > > > > > management, but
>> > > > > > > 90% of the cases will be covered by implicit transaction
>> > > > > > > management
>> > > > > > > throughout the GraphTraversal life cycle.
>> > > > > > > 1. TX is started when GraphTraversal is created.
>> > > > > > > 2. TX is committed when GraphTraversal is closed (which
>> > > > > > > includes
>> > > > > > > terminal operations like toList or iterate).
>> > > > > > > 3. TX is rolled back in case of exceptional execution of
>> > > > > > > GraphTraversal. Users can always roll back a transaction by
>> > > > > > > using
>> > > > > > > the fail() step.
>> > > > > > >
>> > > > > > > We also provide in the YTDB the following methods that I
>> > > > > > > hope will
>> > > > > > > become part of the framework: gSource.computeInTx(g ->{}),
>> > > > > > > gSource.executeInTx(g - {}), gSource.autoExecuteInTx(g -
>> > > > > > > {}: g)
>> > > > > > > (this one executes g.iterate() on the returned
>> > > > > > > GraphTraversal,
>> > > > > > > which we find really handy).
>> > > > > > >
>> > > > > > > Now, regarding explicit TX management.
>> > > > > > > I think that explicit TX management is still needed if you
>> > > > > > > want to
>> > > > > > > execute several queries and mix them with complex logic,
>> > > > > > > but I
>> > > > > > > think that methods that explicitly cause TX
>> > > > > > > begin/start/commit are
>> > > > > > > confusing and provide bad DevX.
>> > > > > > > What I propose is to use lambda-controlled TXs (slightly
>> > > > > > > different
>> > > > > > > from the above). I think that an explicit rollback/commit
>> > > > > > > of TX by
>> > > > > > > the user is similar to the goto operator in old programming
>> > > > > > > languages; it gives finer control over TXs, but overall
>> > > > > > > only adds
>> > > > > > > confusion and can be completely avoided in modern
>> > > > > > > languages.
>> > > > > > >
>> > > > > > > Explicit control on TXs is proposed to be implemented using
>> > > > > > > methods
>> > > > > > > similar to above, with a different signature, namely:
>> > > > > > > gSource.executeInTx(graphTraversalSupplier -> {
>> > > > > > > var g = graphTraversalSupplier.get();
>> > > > > > > var v = g.V().next();
>> > > > > > > g.close()
>> > > > > > >
>> > > > > > > v.property("p", "val");
>> > > > > > > int i = v.value("int");
>> > > > > > >
>> > > > > > > if(i > 0) {
>> > > > > > > g = graphTraversalSupplier.get();
>> > > > > > > g.V().has("i2", i).property("i2", i +
>> > > > > > > 1).iterate();
>> > > > > > > }
>> > > > > > > })
>> > > > > > >
>> > > > > > > In this case, the transaction is detached from
>> > > > > > > GraphTraversal, and
>> > > > > > > its lifecycle is not controlled by GraphTraversal.
>> > > > > > > Instead, TX is committed if an exception is not thrown from
>> > > > > > > the
>> > > > > > > lambda method.
>> > > > > > >
>> > > > > > > To inform the user if GraphTraversal controls TX or not, a
>> > > > > > > simple
>> > > > > > > isTxAttached method is proposed to be introduced
>> > > > > > > gSource.<V>computeInTX(graphTraversalSupplier -> {}) works
>> > > > > > > in
>> > > > > > > similar way but returns value.
>> > > > > > >
>> > > > > > >
>> > > > > > > So the final diagram could look like the following:
>> > > > > > >
>> > > > > > >
>> > > > > > > image.png
>> > > > > > >
>> > > > > > > Please note that in this model, there is no need for an
>> > > > > > > explicit
>> > > > > > > transaction object.
>> > > > > > >
>> > > > > > > On Fri, Dec 5, 2025 at 10:18 AM Andrii Lomakin
>> > > > > > > <an...@jetbrains.com> wrote:
>> > > > > > > > Good day, Ken.
>> > > > > > > >
>> > > > > > > > > This could map to multiple threads on the provider
>> > > > > > > > > side. It's
>> > > > > > > > probably OK if something like "g.tx()..." returns the
>> > > > > > > > same
>> > > > > > > > transaction. We could require users to open multiple
>> > > > > > > > DriverRemoteConnections/Clients to achieve this and that
>> > > > > > > > would be
>> > > > > > > > similar to other data connectivity APIs.
>> > > > > > > >
>> > > > > > > > Unfortunately I do not follow, could you elaborate more
>> > > > > > > > or give
>> > > > > > > > me example how do you see it ?
>> > > > > > > >
>> > > > > > > > On Thu, Dec 4, 2025 at 7:06 PM Ken Hu
>> > > > > > > > <ke...@gmail.com>
>> > > wrote:
>> > > > > > > > > I might be thinking about this too much for just the
>> > > > > > > > > remote
>> > > > > > > > > case and not considering the embedded case enough. In
>> > > > > > > > > my mind,
>> > > > > > > > > there just needs to be an easy way for users of the
>> > > > > > > > > language
>> > > > > > > > > variants (in particular JavaScript because of its
>> > > > > > > > > single
>> > > > > > > > > threaded nature) to start multiple transactions from a
>> > > > > > > > > single
>> > > > > > > > > thread. This could map to multiple threads on the
>> > > > > > > > > provider
>> > > > > > > > > side. It's probably OK if something like "g.tx()..."
>> > > > > > > > > returns
>> > > > > > > > > the same transaction. We could require users to open
>> > > > > > > > > multiple
>> > > > > > > > > DriverRemoteConnections/Clients to achieve this and
>> > > > > > > > > that would
>> > > > > > > > > be similar to other data connectivity APIs.
>> > > > > > > > >
>> > > > > > > > > On Thu, Dec 4, 2025 at 2:37 AM Andrii Lomakin via dev
>> > > > > > > > > <de...@tinkerpop.apache.org> wrote:
>> > > > > > > > > > The point about having multiple concurrent
>> > > > > > > > > > transactions
>> > > > > > > > > > within the same
>> > > > > > > > > > thread warrants a more detailed discussion.
>> > > > > > > > > >
>> > > > > > > > > > To compare this behavior, consider how major SQL
>> > > > > > > > > > databases
>> > > > > > > > > > handle attempts
>> > > > > > > > > > to start a second transaction when one is already
>> > > > > > > > > > active:
>> > > > > > > > > >
>> > > > > > > > > > 1. Error out: The database prevents the operation,
>> > > > > > > > > > signaling
>> > > > > > > > > > that a
>> > > > > > > > > > transaction is already active (e.g., "Transaction
>> > > > > > > > > > already
>> > > > > > > > > > active").
>> > > > > > > > > > 2. Create a nested transaction (savepoint): The
>> > > > > > > > > > database
>> > > > > > > > > > creates a
>> > > > > > > > > > sub-scope of the first transaction, not an
>> > > > > > > > > > independent
>> > > > > > > > > > transaction. If the
>> > > > > > > > > > parent transaction fails, the child scope also fails.
>> > > > > > > > > > 3. Implicitly commit the first: The database
>> > > > > > > > > > automatically
>> > > > > > > > > > commits the
>> > > > > > > > > > first transaction to open the second one (a common
>> > > > > > > > > > behavior
>> > > > > > > > > > in MySQL DDL).
>> > > > > > > > > >
>> > > > > > > > > > As far as I know, only Firebird has historically
>> > > > > > > > > > allowed
>> > > > > > > > > > several truly
>> > > > > > > > > > independent transactions concurrently on the same
>> > > > > > > > > > thread. I
>> > > > > > > > > > suspect support
>> > > > > > > > > > for simultaneous transactions may be even more
>> > > > > > > > > > limited among
>> > > > > > > > > > graph database
>> > > > > > > > > > vendors.
>> > > > > > > > > >
>> > > > > > > > > > On Thu, Dec 4, 2025 at 10:47 AM Andrii Lomakin
>> > > > > > > > > > <an...@jetbrains.com>
>> > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > Good day.
>> > > > > > > > > > > As for questions from Ken
>> > > > > > > > > > > > What OGMs are you referring to here? Regular
>> > > > > > > > > > > > transactions
>> > > > > > > > > > are complicated
>> > > > > > > > > > > enough by itself, I would only want to introduce
>> > > > > > > > > > > nested
>> > > > > > > > > > transactions if
>> > > > > > > > > > > there were well maintained OGMs that would have a
>> > > > > > > > > > demonstrable benefit from
>> > > > > > > > > > > this.
>> > > > > > > > > > >
>> > > > > > > > > > > We have such an OGM using TinkerPop that is well
>> > > > > > > > > > maintained, if you mean
>> > > > > > > > > > > by this years of support and thousands of instances
>> > > > > > > > > > > running
>> > > > > > > > > > on OS version.
>> > > > > > > > > > > However, I have given it thought and believe that
>> > > > > > > > > > > it is a
>> > > > > > > > > > relatively rare
>> > > > > > > > > > > case, and OGM is so complex that it can manage
>> > > > > > > > > > > nested
>> > > > > > > > > > transactions itself
>> > > > > > > > > > > if needed.
>> > > > > > > > > > > I think we can support nested transactions on pause
>> > > > > > > > > > > for
>> > > > > > > > > > now, till we
>> > > > > > > > > > > receive more feedback from users.
>> > > > > > > > > > >
>> > > > > > > > > > > There are many inconsistencies to discuss in the TX
>> > > > > > > > > > lifecycle, even
>> > > > > > > > > > > without nested transactions.
>> > > > > > > > > > >
>> > > > > > > > > > > > Sessions won't exist moving forward (4.x+) as
>> > > > > > > > > > > > they are
>> > > > > > > > > > tied to remote
>> > > > > > > > > > > Groovy execution. So let's continue this
>> > > > > > > > > > > conversation as
>> > > > > > > > > > simply
>> > > > > > > > > > > transactions and not refer to sessions at all to
>> > > > > > > > > > > prevent
>> > > > > > > > > > confusion, unless
>> > > > > > > > > > > you intend to have these changes in the 3.8.x line?
>> > > > > > > > > > > Yeah, we already removed Groovy support in our
>> > > > > > > > > > > fork, so it
>> > > > > > > > > > is even now in
>> > > > > > > > > > > our distribution TX, and the session is the same.
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > > Could you expand a bit more on "Transaction scope
>> > > > > > > > > > > > is
>> > > > > > > > > > limited to the
>> > > > > > > > > > > current thread only".
>> > > > > > > > > > > I mean that TX visibility and manipulation are
>> > > > > > > > > > > limited to a
>> > > > > > > > > > single thread
>> > > > > > > > > > > where it is created.
>> > > > > > > > > > > If some of the providers support TX visibility
>> > > > > > > > > > > between
>> > > > > > > > > > several threads,
>> > > > > > > > > > > that will only add a bonus for them if such a need
>> > > > > > > > > > > arises,
>> > > > > > > > > > of course,
>> > > > > > > > > > > without breaking the API.
>> > > > > > > > > > >
>> > > > > > > > > > > > I find it limiting that an attempt to open
>> > > > > > > > > > > > multiple
>> > > > > > > > > > transactions from
>> > > > > > > > > > > the same thread ends up returning the same
>> > > > > > > > > > > Transaction
>> > > > > > > > > > instance
>> > > > > > > > > > > Do you propose to support several transactions in
>> > > > > > > > > > > the same
>> > > > > > > > > > thread?
>> > > > > > > > > > > I am afraid not all current vendors can support
>> > > > > > > > > > > this model.
>> > > > > > > > > > >
>> > > > > > > > > > > On Wed, Dec 3, 2025 at 1:23 PM Stephen Mallette
>> > > > > > > > > > <sp...@gmail.com>
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > Andrii, I'd like to just quickly clarify Ken's
>> > > > > > > > > > > > point
>> > > > > > > > > > stating " Sessions
>> > > > > > > > > > > > won't exist moving forward (4.x+) ". He's saying
>> > > > > > > > > > > > what you
>> > > > > > > > > > said: "
>> > > > > > > > > > > > 'session'
>> > > > > > > > > > > > and 'tx' are essentially the same concepts". We
>> > > > > > > > > > acknowledged that and
>> > > > > > > > > > > > decided to stop using "session" terminology and
>> > > > > > > > > > > > instead
>> > > > > > > > > > use "transaction"
>> > > > > > > > > > > > terminology only to avoid confusion.
>> > > > > > > > > > > >
>> > > > > > > > > > > > Much of what you wrote is in the spirit of what
>> > > > > > > > > > > > has been
>> > > > > > > > > > considered for
>> > > > > > > > > > > > 4.x, so that is nice alignment. I do think the
>> > > > > > > > > > > > goal though
>> > > > > > > > > > is to have a
>> > > > > > > > > > > > consistent transaction API/system that works for
>> > > > > > > > > > > > both
>> > > > > > > > > > embedded and remote
>> > > > > > > > > > > > cases. It will be interesting to see how that is
>> > > > > > > > > > > > achieved.
>> > > > > > > > > > > >
>> > > > > > > > > > > > On Tue, Dec 2, 2025 at 3:53 PM Ken Hu
>> > > > > > > > > > > > <kenhuoss@gmail.com
>> > > >
>> > > > > > > > > > wrote:
>> > > > > > > > > > > >
>> > > > > > > > > > > > > Hi Andrii,
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Good day to you as well. I have a couple
>> > > > > > > > > > > > > questions about
>> > > > > > > > > > this that I
>> > > > > > > > > > > > would
>> > > > > > > > > > > > > like some more details on.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > 1. What OGMs are you referring to here? Regular
>> > > > > > > > > > transactions are
>> > > > > > > > > > > > > complicated enough by itself, I would only want
>> > > > > > > > > > > > > to
>> > > > > > > > > > introduce nested
>> > > > > > > > > > > > > transactions if there were well maintained OGMs
>> > > > > > > > > > > > > that
>> > > > > > > > > > would have a
>> > > > > > > > > > > > > demonstrable benefit from this.
>> > > > > > > > > > > > > 2. I like some of your ideas about removing
>> > > > > > > > > > > > > Transaction
>> > > > > > > > > > as its own
>> > > > > > > > > > > > class,
>> > > > > > > > > > > > > but I'm not sure GraphTraversalSource is where
>> > > > > > > > > > > > > it should
>> > > > > > > > > > be instead. Do
>> > > > > > > > > > > > you
>> > > > > > > > > > > > > have some more details about what you think the
>> > > > > > > > > > Transaction API should
>> > > > > > > > > > > > look
>> > > > > > > > > > > > > like?
>> > > > > > > > > > > > > 3. Could you expand a bit more on "Transaction
>> > > > > > > > > > > > > scope is
>> > > > > > > > > > limited to the
>> > > > > > > > > > > > > current thread only". I always get confused by
>> > > > > > > > > > TinkerPop's threaded vs
>> > > > > > > > > > > > > multithreaded transaction. I find it limiting
>> > > > > > > > > > > > > that an
>> > > > > > > > > > attempt to open
>> > > > > > > > > > > > > multiple transactions from the same thread ends
>> > > > > > > > > > > > > up
>> > > > > > > > > > returning the same
>> > > > > > > > > > > > > Transaction instance. Maybe I didn't quite
>> > > > > > > > > > > > > understand
>> > > > > > > > > > what you meant by
>> > > > > > > > > > > > > this.
>> > > > > > > > > > > > > 4. Sessions won't exist moving forward (4.x+)
>> > > > > > > > > > > > > as they
>> > > > > > > > > > are tied to remote
>> > > > > > > > > > > > > Groovy execution. So let's continue this
>> > > > > > > > > > > > > conversation as
>> > > > > > > > > > simply
>> > > > > > > > > > > > > transactions and not refer to sessions at all
>> > > > > > > > > > > > > to prevent
>> > > > > > > > > > confusion,
>> > > > > > > > > > > > unless
>> > > > > > > > > > > > > you intend to have these changes in the 3.8.x
>> > > > > > > > > > > > > line?
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Regards,
>> > > > > > > > > > > > > Ken
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > On Tue, Dec 2, 2025 at 3:33 AM Andrii Lomakin
>> > > > > > > > > > > > > via dev <
>> > > > > > > > > > > > > dev@tinkerpop.apache.org> wrote:
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > > Small clarification.
>> > > > > > > > > > > > > > Implementation notes are related to changes
>> > > > > > > > > > > > > > in the
>> > > > > > > > > > remote protocol.
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > On Tue, Dec 2, 2025 at 8:55 AM Andrii Lomakin
>> > > > > > > > > > > > > > <
>> > > > > > > > > > > > > > andrey.lomakin@jetbrains.com>
>> > > > > > > > > > > > > > wrote:
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > To clarify: in this model, 'session' and
>> > > > > > > > > > > > > > > 'tx' are
>> > > > > > > > > > essentially the
>> > > > > > > > > > > > same
>> > > > > > > > > > > > > > > concepts.
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > I also think it is essential to have a
>> > > > > > > > > > > > > > > common
>> > > > > > > > > > denominator that
>> > > > > > > > > > > > makes TP
>> > > > > > > > > > > > > > > applications portable across
>> > > > > > > > > > > > > > > implementations.
>> > > > > > > > > > > > > > > That is why I also propose to restrict
>> > > > > > > > > > > > > > > transaction
>> > > > > > > > > > scope to the
>> > > > > > > > > > > > thread
>> > > > > > > > > > > > > > > scope. It will be a common denominator that
>> > > > > > > > > > > > > > > will not
>> > > > > > > > > > harm user
>> > > > > > > > > > > > > experience
>> > > > > > > > > > > > > > > but will ensure uniform and expected API
>> > > > > > > > > > > > > > > behavior.
>> > > > > > > > > > > > > > > There should not be both `begin` and `open`
>> > > > > > > > > > > > > > > methods
>> > > > > > > > > > because the
>> > > > > > > > > > > > > contract
>> > > > > > > > > > > > > > > of the beginning method is vague and
>> > > > > > > > > > > > > > > unpredictable.
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > On Tue, Dec 2, 2025 at 8:18 AM Andrii
>> > > > > > > > > > > > > > > Lomakin <
>> > > > > > > > > > > > > > > andrey.lomakin@jetbrains.com> wrote:
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Good day,
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > I have a proposal to enhance the
>> > > > > > > > > > > > > > > > transaction
>> > > > > > > > > > handling logic in TP,
>> > > > > > > > > > > > > > > > specifically regarding nested transaction
>> > > > > > > > > > > > > > > > calls,
>> > > > > > > > > > which are common
>> > > > > > > > > > > > in
>> > > > > > > > > > > > > EE
>> > > > > > > > > > > > > > > > development, particularly when utilizing
>> > > > > > > > > > > > > > > > OGM.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Currently, calling open multiple times
>> > > > > > > > > > > > > > > > causes an
>> > > > > > > > > > exception. This
>> > > > > > > > > > > > > > behavior
>> > > > > > > > > > > > > > > > is restrictive, and the term "open" can
>> > > > > > > > > > > > > > > > be
>> > > > > > > > > > confusing.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > I propose a design that improves the
>> > > > > > > > > > > > > > > > developer
>> > > > > > > > > > experience by using
>> > > > > > > > > > > > an
>> > > > > > > > > > > > > > > > internal counter for transaction calls:
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Proposed transaction counter logic
>> > > > > > > > > > > > > > > > 1. begin/commit counter:
>> > > > > > > > > > > > > > > > - begin would increment an internal
>> > > > > > > > > > > > > > > > counter.
>> > > > > > > > > > > > > > > > - commit would decrement the internal
>> > > > > > > > > > > > > > > > counter,
>> > > > > > > > > > causing the actual
>> > > > > > > > > > > > > > > > transaction commit only when the counter
>> > > > > > > > > > > > > > > > reaches
>> > > > > > > > > > zero.
>> > > > > > > > > > > > > > > > 2. rollback behavior:
>> > > > > > > > > > > > > > > > - rollback would immediately force a
>> > > > > > > > > > > > > > > > transaction
>> > > > > > > > > > rollback and
>> > > > > > > > > > > > > > decrements
>> > > > > > > > > > > > > > > > the internal counter
>> > > > > > > > > > > > > > > > - the subsequent calls to rollback on
>> > > > > > > > > > > > > > > > the same
>> > > > > > > > > > transaction object
>> > > > > > > > > > > > > > should
>> > > > > > > > > > > > > > > > be allowed until the stack trace reaches
>> > > > > > > > > > > > > > > > the
>> > > > > > > > > > initial begin method
>> > > > > > > > > > > > > call,
>> > > > > > > > > > > > > > > > after which further rollback calls would
>> > > > > > > > > > > > > > > > throw an
>> > > > > > > > > > exception.
>> > > > > > > > > > > > > > > > 3. Error handling:
>> > > > > > > > > > > > > > > > - A commit without a matching begin
>> > > > > > > > > > > > > > > > (e.g., counter
>> > > > > > > > > > is already zero)
>> > > > > > > > > > > > > > > > should throw an exception.
>> > > > > > > > > > > > > > > > - A rollback without a matching begin
>> > > > > > > > > > > > > > > > should throw
>> > > > > > > > > > an exception
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > I would also consider hiding the
>> > > > > > > > > > > > > > > > Transaction object
>> > > > > > > > > > altogether and
>> > > > > > > > > > > > > > > > delegating the logic of transaction
>> > > > > > > > > > > > > > > > management to
>> > > > > > > > > > > > GraphTraversalSource
>> > > > > > > > > > > > > > it
>> > > > > > > > > > > > > > > > will simplify the API a lot, both for
>> > > > > > > > > > > > > > > > users and
>> > > > > > > > > > vendors.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > I also propose to restrict transaction
>> > > > > > > > > > > > > > > > scope to the
>> > > > > > > > > > thread scope,
>> > > > > > > > > > > > it
>> > > > > > > > > > > > > > will
>> > > > > > > > > > > > > > > > be a common denominator that will not
>> > > > > > > > > > > > > > > > harm user
>> > > > > > > > > > experience but will
>> > > > > > > > > > > > > > ensure
>> > > > > > > > > > > > > > > > uniform and expected API behavior across
>> > > > > > > > > > > > > > > > all
>> > > > > > > > > > implementations.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Proposed API changes
>> > > > > > > > > > > > > > > > - I propose deprecating open and using
>> > > > > > > > > > > > > > > > begin with
>> > > > > > > > > > the semantics
>> > > > > > > > > > > > > > > > described above to avoid confusion.
>> > > > > > > > > > > > > > > > - Transaction object is deprecated, and
>> > > > > > > > > > > > > > > > TX control
>> > > > > > > > > > functionality is
>> > > > > > > > > > > > > > > > delegated to the GraphTraversal/Graph
>> > > > > > > > > > > > > > > > instances.
>> > > > > > > > > > > > > > > > - tx() method is deprecated.
>> > > > > > > > > > > > > > > > - Transaction scope is limited to the
>> > > > > > > > > > > > > > > > current
>> > > > > > > > > > thread only.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Behavior for non-direct calls
>> > > > > > > > > > > > > > > > When begin/commit/rollback methods are
>> > > > > > > > > > > > > > > > not called
>> > > > > > > > > > directly, the
>> > > > > > > > > > > > > > > > transaction should be automatically
>> > > > > > > > > > > > > > > > committed by a
>> > > > > > > > > > terminal
>> > > > > > > > > > > > operation
>> > > > > > > > > > > > > > > > (e.g., when hasNext returns false) in
>> > > > > > > > > > > > > > > > both remote
>> > > > > > > > > > and embedded
>> > > > > > > > > > > > modes.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > Implementation notes:
>> > > > > > > > > > > > > > > > 1. The begin command should be added.
>> > > > > > > > > > > > > > > > Relying on
>> > > > > > > > > > the implicit
>> > > > > > > > > > > > start of
>> > > > > > > > > > > > > > TX
>> > > > > > > > > > > > > > > > by traversal can lead to non-controlled
>> > > > > > > > > > > > > > > > side
>> > > > > > > > > > effects.
>> > > > > > > > > > > > > > > > 2. For the sake of optimization of the
>> > > > > > > > > > > > > > > > remote
>> > > > > > > > > > protocol, while
>> > > > > > > > > > > > multiple
>> > > > > > > > > > > > > > > > calls of begin/commit/rollback are
>> > > > > > > > > > > > > > > > allowed in the
>> > > > > > > > > > remote protocol,
>> > > > > > > > > > > > > > > > practically, it will mean that we will
>> > > > > > > > > > > > > > > > track the
>> > > > > > > > > > counter locally on
>> > > > > > > > > > > > > the
>> > > > > > > > > > > > > > > > client and will send
>> > > > > > > > > > > > > > > > begin/commit/rollback only
>> > > > > > > > > > once for the TX
>> > > > > > > > > > > > > > lifecycle.
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > This model should be uniform across all
>> > > > > > > > > > > > > > > > deployment
>> > > > > > > > > > types.
>> > > > > > > > > > > > > > > > What are your thoughts on this approach?
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > > -----
>> > > > > > > > > > > > > > > > Andrii Lomakin
>> > > > > > > > > > > > > > > > YouTrackDB development lead
>> > > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > > > --
>> > > > > > > > > > > > > > > Andrii Lomakin
>> > > > > > > > > > > > > > > YouTrackDB development lead
>> > > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > > > --
>> > > > > > > > > > > > > > Andrii Lomakin
>> > > > > > > > > > > > > > YouTrackDB development lead
>> > > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > --
>> > > > > > > > > > > Andrii Lomakin
>> > > > > > > > > > > YouTrackDB development lead
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> > > --
>> > > Andrii Lomakin
>> > > YouTrackDB development lead
>> > >
>>
>
Re: [DISCUSS] Handling of transactions - proposal
Posted by Ken Hu <ke...@gmail.com>.
Given that this is a major version upgrade, I think we can break the core
interfaces as much as needed as long as there is a good reason for doing
so. I understand the hesitancy to make breaking changes, but it's become
apparent that the APIs don't work well for the remote case. I also expect
that the 3.x line will be maintained for a while even after the release of
TinkerPop 4 so that will give users extra time to upgrade. But yes, we
should keep the upgrade path in mind.
On Fri, Jan 23, 2026 at 11:49 PM pieter <pi...@riseup.net> wrote:
> Hi,
>
> Before discussing these changes further, I need some clarity on how
> much we are willing to break core interfaces in TP 4?
>
> The core interfaces have been around for more than a decade. i.e.
> millions and millions of lines of code in production.
>
> Personally I am in the 'don't break user space'. camp.
>
> If we do make drastic changes, we should right from the start think
> about the upgrade path and think about the impact on existing code
> bases.
>
> Regards
> Pieter
>
>
>
>
> On Wed, 2026-01-21 at 20:40 -0800, Ken Hu wrote:
> > I agree that record consistency is important, but I feel that where
> > this
> > gets solved depends on how users integrate graph databases into their
> > applications. There are probably frameworks and OGM/ORMs that can do
> > this
> > detaching/unloading that you are mentioning as a solution. If a user
> > directly integrates the database as an external service using a
> > driver,
> > then having it as part of the API could be useful. I need to think
> > about
> > this more.
> >
> > To your second point about platform-independence, I think this has
> > always
> > been a goal of TinkerPop that has never quite been realized. Even
> > today,
> > there are some examples of Gremlin that return results that are
> > provider-dependent and so there are specific cases where it isn't
> > truly
> > portable. I agree that a difference in read consistency amongst
> > vendors/providers means those using transactions can't move between
> > different databases unless they find another database with the exact
> > same
> > transaction behavior. But I'm not entirely sure we can get all
> > providers to
> > match the same behavior in this case.
> >
> > On Tue, Jan 20, 2026 at 11:26 PM Andrii Lomakin via dev <
> > dev@tinkerpop.apache.org> wrote:
> >
> > > Hi Ken.
> > > Thank you for sharing this with us.
> > >
> > > After some thought, I agree that we need explicit TX management
> > > methods.
> > > From my perspective, what I wrote is only the tip of the iceberg,
> > > as such
> > > questions as consistency of the queries and validity of the data
> > > contained
> > > in vertices directly bound to the concept of TX lifecycle.
> > >
> > > Example (it can be applied to any database, not just a graph
> > > database, so I
> > > will use the general record concept).
> > >
> > > I have created record A { id: 1, value: "val1"} , committed it, and
> > > passed
> > > it to another thread.
> > > After that, another method updated the same record B {id: 1, value
> > > :
> > > "val2"} and committed it.
> > >
> > > A and B in this context object identity, not DB record identity.
> > >
> > > But I have the record A in another thread with potentially stale
> > > data.
> > > What then should another thread read when it accesses `A.value`?
> > >
> > > I doubt it should read stale data. In our database, records are
> > > automatically switched to the unloaded state after commit and can
> > > be read
> > > in a new thread only when a new transaction is started.
> > > The question of record consistency, from my point of view, is an
> > > important
> > > one and cannot be overlooked, as it influences how users interact
> > > with the
> > > TP API and to what degree it is really platform-independent.
> > >
> > >
> > > On Wed, Jan 21, 2026 at 7:58 AM Ken Hu <ke...@gmail.com> wrote:
> > >
> > > > Here are some of my current thoughts on the proposed changes and
> > > > discussion.
> > > >
> > > > -deprecate open() and replace with begin()
> > > > -I agree with this as having both is confusing and unncessary
> > > >
> > > > -Transaction class is deprecated, and TX control functionality is
> > > delegated
> > > > to the GraphTraversal/Graph instances
> > > > -I mostly agree with this as well, but I think it's probably
> > > > better
> > > if
> > > > it was delegated to GraphTraversalSource.
> > > >
> > > > -tx() method is deprecated.
> > > > -I think this makes sense as I always found how tx() works
> > > > with
> > > remote
> > > > to be unintuitive.
> > > >
> > > > -Transaction scope is limited to the current thread only
> > > > -I had some questions about this earlier, but having thought
> > > > about it
> > > > after seeing Andrii's architectural diagram, it's probably fine
> > > > since
> > > most
> > > > developers (in all languages TinkerPop supports) will be used to
> > > > this
> > > being
> > > > how it works as long as we have something that represents what a
> > > connection
> > > > is in other databases.
> > > >
> > > > -I believe Non-TX mode is not appropriate for handling graph data
> > > > structures.
> > > > -Further, I'd say that transactions should be optional. Many
> > > > persistent
> > > > db/graph systems are eventually consistent and do not support
> > > transactions,
> > > > so they should not be burdened with concepts foreign to them.
> > > > -I tend to agree with Pieter here. My reasoning is different
> > > > though,
> > > > one of the use cases for graphs is analytics and transactions
> > > > aren't
> > > really
> > > > used in that case.
> > > >
> > > > -I think we DO need methods for explicit transaction management,
> > > > but 90%
> > > of
> > > > the cases will be covered by implicit transaction management
> > > > throughout
> > > the
> > > > GraphTraversal life cycle.
> > > > -What I propose is to use lambda-controlled TXs
> > > > -My idea is that the transaction api should be unopinionated.
> > > > That means
> > > it
> > > > should follow closely to `UserTransaction`
> > > > -I think having implicit transactions is fine as long as it's
> > > > just an
> > > > option. I'm not sure which should be the default. I'm not sure we
> > > > can
> > > make
> > > > lambda-controlled transactions work though since lambdas wouldn't
> > > > be
> > > > compatible with Gremlin scripts. At this point, we probably still
> > > > want
> > > > parity between scripts and GraphTraversals.
> > > >
> > > > I think we need some more proposals for what the explicit
> > > > transaction
> > > > management API could look like because we probably can't move
> > > > ahead with
> > > > lambda-controlled TX for now and the current tx() solution is
> > > > clunky.
> > > I'll
> > > > try to propose one in the next couple of days.
> > > >
> > > > On Wed, Jan 7, 2026 at 12:24 PM pieter <pi...@riseup.net>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > I had a brief discussion with Andrii in the discord chat
> > > > > regarding the
> > > > > transaction api.
> > > > >
> > > > > My idea is that the transaction api should be unopinionated.
> > > > > That means it should follow closely to
> > > > > `javax.transaction.UserTransaction` which in turn means we have
> > > > > a
> > > > > dedicated class modelling the transaction with `begin()`,
> > > > > `commit()`
> > > > > and `rollback()`.
> > > > >
> > > > > My reasoning is that gremlin is a low level graph language.
> > > > > There are
> > > > > many ways different developers / architects can go about
> > > > > integrating
> > > > > gremlin into their stack.
> > > > >
> > > > > Some like lambdas, some like IOT containers, some have domain
> > > > > entity
> > > > > frameworks, some use aspect orienated programming and on an on.
> > > > >
> > > > > It is common for IOT containers to manage transaction
> > > > > boundaries and
> > > > > lifecycles. Similarly it is common to use the web server's
> > > > > interceptors
> > > > > to manage transaction boundaries. And it is very common to do
> > > > > it all
> > > > > manually...
> > > > >
> > > > > Further, I'd say that transactions should be optional. Many
> > > > > persistent
> > > > > db/graph systems are eventually consistent and do not support
> > > > > transactions, so they should not be burdened with concepts
> > > > > foreign to
> > > > > them.
> > > > >
> > > > > Regarding binding the transaction to the thread:
> > > > > This is the only way I have ever worked so its entirely natural
> > > > > to me.
> > > > > However I can image that people want to defince their own
> > > > > transaction
> > > > > context, so perhaps we should not hardcode that part. Long ago
> > > > > Gavin
> > > > > King from Hibernate fame had the notion of `conversation`
> > > > > context.
> > > > > Perhaps such things are still around?
> > > > >
> > > > > Regards
> > > > > Pieter
> > > > >
> > > > > On Thu, 2025-12-11 at 09:02 +0100, Andrii Lomakin via dev
> > > > > wrote:
> > > > > > Unfortunately, all images have been removed, so I am sending
> > > > > > a link
> > > > > > to the diagrams instead
> > > > > >
> https://miro.com/app/board/uXjVGdd8m44=/?share_link_id=686668490384
> > > > > > .
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Thu, Dec 11, 2025 at 8:53 AM Andrii Lomakin
> > > > > > <an...@jetbrains.com> wrote:
> > > > > > > Good day.
> > > > > > >
> > > > > > > If I followed Ken's line of thought correctly, I would like
> > > > > > > to
> > > > > > > expand it to be part of the new proposal for the
> > > > > > > transaction API.
> > > > > > > First of all, about the model, I think we came to a
> > > > > > > consensus that
> > > > > > > it should be:
> > > > > > >
> > > > > > >
> > > > > > > image.png
> > > > > > > The point is that we do not have a notion of connection in
> > > > > > > TinkerPop, but we have a notion of GraphTraversal.
> > > > > > > So, from my perspective, this diagram can be changed to
> > > > > > > image.png
> > > > > > >
> > > > > > > There is a corner case for Non-TX databases that we need to
> > > > > > > discuss.
> > > > > > > I believe Non-TX mode is not appropriate for handling graph
> > > > > > > data
> > > > > > > structures. GQL, for example, requires serializable
> > > > > > > isolation to
> > > > > > > satisfy the standard requirements.
> > > > > > > Instead, we should require to have presence of a
> > > > > > > transaction, which
> > > > > > > is started implicitly when GraphTraversal is created and
> > > > > > > committed
> > > > > > > once it is closed.
> > > > > > >
> > > > > > > I think we DO need methods for explicit transaction
> > > > > > > management, but
> > > > > > > 90% of the cases will be covered by implicit transaction
> > > > > > > management
> > > > > > > throughout the GraphTraversal life cycle.
> > > > > > > 1. TX is started when GraphTraversal is created.
> > > > > > > 2. TX is committed when GraphTraversal is closed (which
> > > > > > > includes
> > > > > > > terminal operations like toList or iterate).
> > > > > > > 3. TX is rolled back in case of exceptional execution of
> > > > > > > GraphTraversal. Users can always roll back a transaction by
> > > > > > > using
> > > > > > > the fail() step.
> > > > > > >
> > > > > > > We also provide in the YTDB the following methods that I
> > > > > > > hope will
> > > > > > > become part of the framework: gSource.computeInTx(g ->{}),
> > > > > > > gSource.executeInTx(g - {}), gSource.autoExecuteInTx(g -
> > > > > > > {}: g)
> > > > > > > (this one executes g.iterate() on the returned
> > > > > > > GraphTraversal,
> > > > > > > which we find really handy).
> > > > > > >
> > > > > > > Now, regarding explicit TX management.
> > > > > > > I think that explicit TX management is still needed if you
> > > > > > > want to
> > > > > > > execute several queries and mix them with complex logic,
> > > > > > > but I
> > > > > > > think that methods that explicitly cause TX
> > > > > > > begin/start/commit are
> > > > > > > confusing and provide bad DevX.
> > > > > > > What I propose is to use lambda-controlled TXs (slightly
> > > > > > > different
> > > > > > > from the above). I think that an explicit rollback/commit
> > > > > > > of TX by
> > > > > > > the user is similar to the goto operator in old programming
> > > > > > > languages; it gives finer control over TXs, but overall
> > > > > > > only adds
> > > > > > > confusion and can be completely avoided in modern
> > > > > > > languages.
> > > > > > >
> > > > > > > Explicit control on TXs is proposed to be implemented using
> > > > > > > methods
> > > > > > > similar to above, with a different signature, namely:
> > > > > > > gSource.executeInTx(graphTraversalSupplier -> {
> > > > > > > var g = graphTraversalSupplier.get();
> > > > > > > var v = g.V().next();
> > > > > > > g.close()
> > > > > > >
> > > > > > > v.property("p", "val");
> > > > > > > int i = v.value("int");
> > > > > > >
> > > > > > > if(i > 0) {
> > > > > > > g = graphTraversalSupplier.get();
> > > > > > > g.V().has("i2", i).property("i2", i +
> > > > > > > 1).iterate();
> > > > > > > }
> > > > > > > })
> > > > > > >
> > > > > > > In this case, the transaction is detached from
> > > > > > > GraphTraversal, and
> > > > > > > its lifecycle is not controlled by GraphTraversal.
> > > > > > > Instead, TX is committed if an exception is not thrown from
> > > > > > > the
> > > > > > > lambda method.
> > > > > > >
> > > > > > > To inform the user if GraphTraversal controls TX or not, a
> > > > > > > simple
> > > > > > > isTxAttached method is proposed to be introduced
> > > > > > > gSource.<V>computeInTX(graphTraversalSupplier -> {}) works
> > > > > > > in
> > > > > > > similar way but returns value.
> > > > > > >
> > > > > > >
> > > > > > > So the final diagram could look like the following:
> > > > > > >
> > > > > > >
> > > > > > > image.png
> > > > > > >
> > > > > > > Please note that in this model, there is no need for an
> > > > > > > explicit
> > > > > > > transaction object.
> > > > > > >
> > > > > > > On Fri, Dec 5, 2025 at 10:18 AM Andrii Lomakin
> > > > > > > <an...@jetbrains.com> wrote:
> > > > > > > > Good day, Ken.
> > > > > > > >
> > > > > > > > > This could map to multiple threads on the provider
> > > > > > > > > side. It's
> > > > > > > > probably OK if something like "g.tx()..." returns the
> > > > > > > > same
> > > > > > > > transaction. We could require users to open multiple
> > > > > > > > DriverRemoteConnections/Clients to achieve this and that
> > > > > > > > would be
> > > > > > > > similar to other data connectivity APIs.
> > > > > > > >
> > > > > > > > Unfortunately I do not follow, could you elaborate more
> > > > > > > > or give
> > > > > > > > me example how do you see it ?
> > > > > > > >
> > > > > > > > On Thu, Dec 4, 2025 at 7:06 PM Ken Hu
> > > > > > > > <ke...@gmail.com>
> > > wrote:
> > > > > > > > > I might be thinking about this too much for just the
> > > > > > > > > remote
> > > > > > > > > case and not considering the embedded case enough. In
> > > > > > > > > my mind,
> > > > > > > > > there just needs to be an easy way for users of the
> > > > > > > > > language
> > > > > > > > > variants (in particular JavaScript because of its
> > > > > > > > > single
> > > > > > > > > threaded nature) to start multiple transactions from a
> > > > > > > > > single
> > > > > > > > > thread. This could map to multiple threads on the
> > > > > > > > > provider
> > > > > > > > > side. It's probably OK if something like "g.tx()..."
> > > > > > > > > returns
> > > > > > > > > the same transaction. We could require users to open
> > > > > > > > > multiple
> > > > > > > > > DriverRemoteConnections/Clients to achieve this and
> > > > > > > > > that would
> > > > > > > > > be similar to other data connectivity APIs.
> > > > > > > > >
> > > > > > > > > On Thu, Dec 4, 2025 at 2:37 AM Andrii Lomakin via dev
> > > > > > > > > <de...@tinkerpop.apache.org> wrote:
> > > > > > > > > > The point about having multiple concurrent
> > > > > > > > > > transactions
> > > > > > > > > > within the same
> > > > > > > > > > thread warrants a more detailed discussion.
> > > > > > > > > >
> > > > > > > > > > To compare this behavior, consider how major SQL
> > > > > > > > > > databases
> > > > > > > > > > handle attempts
> > > > > > > > > > to start a second transaction when one is already
> > > > > > > > > > active:
> > > > > > > > > >
> > > > > > > > > > 1. Error out: The database prevents the operation,
> > > > > > > > > > signaling
> > > > > > > > > > that a
> > > > > > > > > > transaction is already active (e.g., "Transaction
> > > > > > > > > > already
> > > > > > > > > > active").
> > > > > > > > > > 2. Create a nested transaction (savepoint): The
> > > > > > > > > > database
> > > > > > > > > > creates a
> > > > > > > > > > sub-scope of the first transaction, not an
> > > > > > > > > > independent
> > > > > > > > > > transaction. If the
> > > > > > > > > > parent transaction fails, the child scope also fails.
> > > > > > > > > > 3. Implicitly commit the first: The database
> > > > > > > > > > automatically
> > > > > > > > > > commits the
> > > > > > > > > > first transaction to open the second one (a common
> > > > > > > > > > behavior
> > > > > > > > > > in MySQL DDL).
> > > > > > > > > >
> > > > > > > > > > As far as I know, only Firebird has historically
> > > > > > > > > > allowed
> > > > > > > > > > several truly
> > > > > > > > > > independent transactions concurrently on the same
> > > > > > > > > > thread. I
> > > > > > > > > > suspect support
> > > > > > > > > > for simultaneous transactions may be even more
> > > > > > > > > > limited among
> > > > > > > > > > graph database
> > > > > > > > > > vendors.
> > > > > > > > > >
> > > > > > > > > > On Thu, Dec 4, 2025 at 10:47 AM Andrii Lomakin
> > > > > > > > > > <an...@jetbrains.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Good day.
> > > > > > > > > > > As for questions from Ken
> > > > > > > > > > > > What OGMs are you referring to here? Regular
> > > > > > > > > > > > transactions
> > > > > > > > > > are complicated
> > > > > > > > > > > enough by itself, I would only want to introduce
> > > > > > > > > > > nested
> > > > > > > > > > transactions if
> > > > > > > > > > > there were well maintained OGMs that would have a
> > > > > > > > > > demonstrable benefit from
> > > > > > > > > > > this.
> > > > > > > > > > >
> > > > > > > > > > > We have such an OGM using TinkerPop that is well
> > > > > > > > > > maintained, if you mean
> > > > > > > > > > > by this years of support and thousands of instances
> > > > > > > > > > > running
> > > > > > > > > > on OS version.
> > > > > > > > > > > However, I have given it thought and believe that
> > > > > > > > > > > it is a
> > > > > > > > > > relatively rare
> > > > > > > > > > > case, and OGM is so complex that it can manage
> > > > > > > > > > > nested
> > > > > > > > > > transactions itself
> > > > > > > > > > > if needed.
> > > > > > > > > > > I think we can support nested transactions on pause
> > > > > > > > > > > for
> > > > > > > > > > now, till we
> > > > > > > > > > > receive more feedback from users.
> > > > > > > > > > >
> > > > > > > > > > > There are many inconsistencies to discuss in the TX
> > > > > > > > > > lifecycle, even
> > > > > > > > > > > without nested transactions.
> > > > > > > > > > >
> > > > > > > > > > > > Sessions won't exist moving forward (4.x+) as
> > > > > > > > > > > > they are
> > > > > > > > > > tied to remote
> > > > > > > > > > > Groovy execution. So let's continue this
> > > > > > > > > > > conversation as
> > > > > > > > > > simply
> > > > > > > > > > > transactions and not refer to sessions at all to
> > > > > > > > > > > prevent
> > > > > > > > > > confusion, unless
> > > > > > > > > > > you intend to have these changes in the 3.8.x line?
> > > > > > > > > > > Yeah, we already removed Groovy support in our
> > > > > > > > > > > fork, so it
> > > > > > > > > > is even now in
> > > > > > > > > > > our distribution TX, and the session is the same.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > > Could you expand a bit more on "Transaction scope
> > > > > > > > > > > > is
> > > > > > > > > > limited to the
> > > > > > > > > > > current thread only".
> > > > > > > > > > > I mean that TX visibility and manipulation are
> > > > > > > > > > > limited to a
> > > > > > > > > > single thread
> > > > > > > > > > > where it is created.
> > > > > > > > > > > If some of the providers support TX visibility
> > > > > > > > > > > between
> > > > > > > > > > several threads,
> > > > > > > > > > > that will only add a bonus for them if such a need
> > > > > > > > > > > arises,
> > > > > > > > > > of course,
> > > > > > > > > > > without breaking the API.
> > > > > > > > > > >
> > > > > > > > > > > > I find it limiting that an attempt to open
> > > > > > > > > > > > multiple
> > > > > > > > > > transactions from
> > > > > > > > > > > the same thread ends up returning the same
> > > > > > > > > > > Transaction
> > > > > > > > > > instance
> > > > > > > > > > > Do you propose to support several transactions in
> > > > > > > > > > > the same
> > > > > > > > > > thread?
> > > > > > > > > > > I am afraid not all current vendors can support
> > > > > > > > > > > this model.
> > > > > > > > > > >
> > > > > > > > > > > On Wed, Dec 3, 2025 at 1:23 PM Stephen Mallette
> > > > > > > > > > <sp...@gmail.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Andrii, I'd like to just quickly clarify Ken's
> > > > > > > > > > > > point
> > > > > > > > > > stating " Sessions
> > > > > > > > > > > > won't exist moving forward (4.x+) ". He's saying
> > > > > > > > > > > > what you
> > > > > > > > > > said: "
> > > > > > > > > > > > 'session'
> > > > > > > > > > > > and 'tx' are essentially the same concepts". We
> > > > > > > > > > acknowledged that and
> > > > > > > > > > > > decided to stop using "session" terminology and
> > > > > > > > > > > > instead
> > > > > > > > > > use "transaction"
> > > > > > > > > > > > terminology only to avoid confusion.
> > > > > > > > > > > >
> > > > > > > > > > > > Much of what you wrote is in the spirit of what
> > > > > > > > > > > > has been
> > > > > > > > > > considered for
> > > > > > > > > > > > 4.x, so that is nice alignment. I do think the
> > > > > > > > > > > > goal though
> > > > > > > > > > is to have a
> > > > > > > > > > > > consistent transaction API/system that works for
> > > > > > > > > > > > both
> > > > > > > > > > embedded and remote
> > > > > > > > > > > > cases. It will be interesting to see how that is
> > > > > > > > > > > > achieved.
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, Dec 2, 2025 at 3:53 PM Ken Hu
> > > > > > > > > > > > <kenhuoss@gmail.com
> > > >
> > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi Andrii,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Good day to you as well. I have a couple
> > > > > > > > > > > > > questions about
> > > > > > > > > > this that I
> > > > > > > > > > > > would
> > > > > > > > > > > > > like some more details on.
> > > > > > > > > > > > >
> > > > > > > > > > > > > 1. What OGMs are you referring to here? Regular
> > > > > > > > > > transactions are
> > > > > > > > > > > > > complicated enough by itself, I would only want
> > > > > > > > > > > > > to
> > > > > > > > > > introduce nested
> > > > > > > > > > > > > transactions if there were well maintained OGMs
> > > > > > > > > > > > > that
> > > > > > > > > > would have a
> > > > > > > > > > > > > demonstrable benefit from this.
> > > > > > > > > > > > > 2. I like some of your ideas about removing
> > > > > > > > > > > > > Transaction
> > > > > > > > > > as its own
> > > > > > > > > > > > class,
> > > > > > > > > > > > > but I'm not sure GraphTraversalSource is where
> > > > > > > > > > > > > it should
> > > > > > > > > > be instead. Do
> > > > > > > > > > > > you
> > > > > > > > > > > > > have some more details about what you think the
> > > > > > > > > > Transaction API should
> > > > > > > > > > > > look
> > > > > > > > > > > > > like?
> > > > > > > > > > > > > 3. Could you expand a bit more on "Transaction
> > > > > > > > > > > > > scope is
> > > > > > > > > > limited to the
> > > > > > > > > > > > > current thread only". I always get confused by
> > > > > > > > > > TinkerPop's threaded vs
> > > > > > > > > > > > > multithreaded transaction. I find it limiting
> > > > > > > > > > > > > that an
> > > > > > > > > > attempt to open
> > > > > > > > > > > > > multiple transactions from the same thread ends
> > > > > > > > > > > > > up
> > > > > > > > > > returning the same
> > > > > > > > > > > > > Transaction instance. Maybe I didn't quite
> > > > > > > > > > > > > understand
> > > > > > > > > > what you meant by
> > > > > > > > > > > > > this.
> > > > > > > > > > > > > 4. Sessions won't exist moving forward (4.x+)
> > > > > > > > > > > > > as they
> > > > > > > > > > are tied to remote
> > > > > > > > > > > > > Groovy execution. So let's continue this
> > > > > > > > > > > > > conversation as
> > > > > > > > > > simply
> > > > > > > > > > > > > transactions and not refer to sessions at all
> > > > > > > > > > > > > to prevent
> > > > > > > > > > confusion,
> > > > > > > > > > > > unless
> > > > > > > > > > > > > you intend to have these changes in the 3.8.x
> > > > > > > > > > > > > line?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > Ken
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Tue, Dec 2, 2025 at 3:33 AM Andrii Lomakin
> > > > > > > > > > > > > via dev <
> > > > > > > > > > > > > dev@tinkerpop.apache.org> wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Small clarification.
> > > > > > > > > > > > > > Implementation notes are related to changes
> > > > > > > > > > > > > > in the
> > > > > > > > > > remote protocol.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Tue, Dec 2, 2025 at 8:55 AM Andrii Lomakin
> > > > > > > > > > > > > > <
> > > > > > > > > > > > > > andrey.lomakin@jetbrains.com>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > To clarify: in this model, 'session' and
> > > > > > > > > > > > > > > 'tx' are
> > > > > > > > > > essentially the
> > > > > > > > > > > > same
> > > > > > > > > > > > > > > concepts.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I also think it is essential to have a
> > > > > > > > > > > > > > > common
> > > > > > > > > > denominator that
> > > > > > > > > > > > makes TP
> > > > > > > > > > > > > > > applications portable across
> > > > > > > > > > > > > > > implementations.
> > > > > > > > > > > > > > > That is why I also propose to restrict
> > > > > > > > > > > > > > > transaction
> > > > > > > > > > scope to the
> > > > > > > > > > > > thread
> > > > > > > > > > > > > > > scope. It will be a common denominator that
> > > > > > > > > > > > > > > will not
> > > > > > > > > > harm user
> > > > > > > > > > > > > experience
> > > > > > > > > > > > > > > but will ensure uniform and expected API
> > > > > > > > > > > > > > > behavior.
> > > > > > > > > > > > > > > There should not be both `begin` and `open`
> > > > > > > > > > > > > > > methods
> > > > > > > > > > because the
> > > > > > > > > > > > > contract
> > > > > > > > > > > > > > > of the beginning method is vague and
> > > > > > > > > > > > > > > unpredictable.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Tue, Dec 2, 2025 at 8:18 AM Andrii
> > > > > > > > > > > > > > > Lomakin <
> > > > > > > > > > > > > > > andrey.lomakin@jetbrains.com> wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Good day,
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I have a proposal to enhance the
> > > > > > > > > > > > > > > > transaction
> > > > > > > > > > handling logic in TP,
> > > > > > > > > > > > > > > > specifically regarding nested transaction
> > > > > > > > > > > > > > > > calls,
> > > > > > > > > > which are common
> > > > > > > > > > > > in
> > > > > > > > > > > > > EE
> > > > > > > > > > > > > > > > development, particularly when utilizing
> > > > > > > > > > > > > > > > OGM.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Currently, calling open multiple times
> > > > > > > > > > > > > > > > causes an
> > > > > > > > > > exception. This
> > > > > > > > > > > > > > behavior
> > > > > > > > > > > > > > > > is restrictive, and the term "open" can
> > > > > > > > > > > > > > > > be
> > > > > > > > > > confusing.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I propose a design that improves the
> > > > > > > > > > > > > > > > developer
> > > > > > > > > > experience by using
> > > > > > > > > > > > an
> > > > > > > > > > > > > > > > internal counter for transaction calls:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Proposed transaction counter logic
> > > > > > > > > > > > > > > > 1. begin/commit counter:
> > > > > > > > > > > > > > > > - begin would increment an internal
> > > > > > > > > > > > > > > > counter.
> > > > > > > > > > > > > > > > - commit would decrement the internal
> > > > > > > > > > > > > > > > counter,
> > > > > > > > > > causing the actual
> > > > > > > > > > > > > > > > transaction commit only when the counter
> > > > > > > > > > > > > > > > reaches
> > > > > > > > > > zero.
> > > > > > > > > > > > > > > > 2. rollback behavior:
> > > > > > > > > > > > > > > > - rollback would immediately force a
> > > > > > > > > > > > > > > > transaction
> > > > > > > > > > rollback and
> > > > > > > > > > > > > > decrements
> > > > > > > > > > > > > > > > the internal counter
> > > > > > > > > > > > > > > > - the subsequent calls to rollback on
> > > > > > > > > > > > > > > > the same
> > > > > > > > > > transaction object
> > > > > > > > > > > > > > should
> > > > > > > > > > > > > > > > be allowed until the stack trace reaches
> > > > > > > > > > > > > > > > the
> > > > > > > > > > initial begin method
> > > > > > > > > > > > > call,
> > > > > > > > > > > > > > > > after which further rollback calls would
> > > > > > > > > > > > > > > > throw an
> > > > > > > > > > exception.
> > > > > > > > > > > > > > > > 3. Error handling:
> > > > > > > > > > > > > > > > - A commit without a matching begin
> > > > > > > > > > > > > > > > (e.g., counter
> > > > > > > > > > is already zero)
> > > > > > > > > > > > > > > > should throw an exception.
> > > > > > > > > > > > > > > > - A rollback without a matching begin
> > > > > > > > > > > > > > > > should throw
> > > > > > > > > > an exception
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I would also consider hiding the
> > > > > > > > > > > > > > > > Transaction object
> > > > > > > > > > altogether and
> > > > > > > > > > > > > > > > delegating the logic of transaction
> > > > > > > > > > > > > > > > management to
> > > > > > > > > > > > GraphTraversalSource
> > > > > > > > > > > > > > it
> > > > > > > > > > > > > > > > will simplify the API a lot, both for
> > > > > > > > > > > > > > > > users and
> > > > > > > > > > vendors.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > I also propose to restrict transaction
> > > > > > > > > > > > > > > > scope to the
> > > > > > > > > > thread scope,
> > > > > > > > > > > > it
> > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > be a common denominator that will not
> > > > > > > > > > > > > > > > harm user
> > > > > > > > > > experience but will
> > > > > > > > > > > > > > ensure
> > > > > > > > > > > > > > > > uniform and expected API behavior across
> > > > > > > > > > > > > > > > all
> > > > > > > > > > implementations.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Proposed API changes
> > > > > > > > > > > > > > > > - I propose deprecating open and using
> > > > > > > > > > > > > > > > begin with
> > > > > > > > > > the semantics
> > > > > > > > > > > > > > > > described above to avoid confusion.
> > > > > > > > > > > > > > > > - Transaction object is deprecated, and
> > > > > > > > > > > > > > > > TX control
> > > > > > > > > > functionality is
> > > > > > > > > > > > > > > > delegated to the GraphTraversal/Graph
> > > > > > > > > > > > > > > > instances.
> > > > > > > > > > > > > > > > - tx() method is deprecated.
> > > > > > > > > > > > > > > > - Transaction scope is limited to the
> > > > > > > > > > > > > > > > current
> > > > > > > > > > thread only.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Behavior for non-direct calls
> > > > > > > > > > > > > > > > When begin/commit/rollback methods are
> > > > > > > > > > > > > > > > not called
> > > > > > > > > > directly, the
> > > > > > > > > > > > > > > > transaction should be automatically
> > > > > > > > > > > > > > > > committed by a
> > > > > > > > > > terminal
> > > > > > > > > > > > operation
> > > > > > > > > > > > > > > > (e.g., when hasNext returns false) in
> > > > > > > > > > > > > > > > both remote
> > > > > > > > > > and embedded
> > > > > > > > > > > > modes.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Implementation notes:
> > > > > > > > > > > > > > > > 1. The begin command should be added.
> > > > > > > > > > > > > > > > Relying on
> > > > > > > > > > the implicit
> > > > > > > > > > > > start of
> > > > > > > > > > > > > > TX
> > > > > > > > > > > > > > > > by traversal can lead to non-controlled
> > > > > > > > > > > > > > > > side
> > > > > > > > > > effects.
> > > > > > > > > > > > > > > > 2. For the sake of optimization of the
> > > > > > > > > > > > > > > > remote
> > > > > > > > > > protocol, while
> > > > > > > > > > > > multiple
> > > > > > > > > > > > > > > > calls of begin/commit/rollback are
> > > > > > > > > > > > > > > > allowed in the
> > > > > > > > > > remote protocol,
> > > > > > > > > > > > > > > > practically, it will mean that we will
> > > > > > > > > > > > > > > > track the
> > > > > > > > > > counter locally on
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > client and will send
> > > > > > > > > > > > > > > > begin/commit/rollback only
> > > > > > > > > > once for the TX
> > > > > > > > > > > > > > lifecycle.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > This model should be uniform across all
> > > > > > > > > > > > > > > > deployment
> > > > > > > > > > types.
> > > > > > > > > > > > > > > > What are your thoughts on this approach?
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > -----
> > > > > > > > > > > > > > > > Andrii Lomakin
> > > > > > > > > > > > > > > > YouTrackDB development lead
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > Andrii Lomakin
> > > > > > > > > > > > > > > YouTrackDB development lead
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > Andrii Lomakin
> > > > > > > > > > > > > > YouTrackDB development lead
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Andrii Lomakin
> > > > > > > > > > > YouTrackDB development lead
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > > Andrii Lomakin
> > > YouTrackDB development lead
> > >
>
Re: [DISCUSS] Handling of transactions - proposal
Posted by pieter <pi...@riseup.net>.
Hi,
Before discussing these changes further, I need some clarity on how
much we are willing to break core interfaces in TP 4?
The core interfaces have been around for more than a decade. i.e.
millions and millions of lines of code in production.
Personally I am in the 'don't break user space'. camp.
If we do make drastic changes, we should right from the start think
about the upgrade path and think about the impact on existing code
bases.
Regards
Pieter
On Wed, 2026-01-21 at 20:40 -0800, Ken Hu wrote:
> I agree that record consistency is important, but I feel that where
> this
> gets solved depends on how users integrate graph databases into their
> applications. There are probably frameworks and OGM/ORMs that can do
> this
> detaching/unloading that you are mentioning as a solution. If a user
> directly integrates the database as an external service using a
> driver,
> then having it as part of the API could be useful. I need to think
> about
> this more.
>
> To your second point about platform-independence, I think this has
> always
> been a goal of TinkerPop that has never quite been realized. Even
> today,
> there are some examples of Gremlin that return results that are
> provider-dependent and so there are specific cases where it isn't
> truly
> portable. I agree that a difference in read consistency amongst
> vendors/providers means those using transactions can't move between
> different databases unless they find another database with the exact
> same
> transaction behavior. But I'm not entirely sure we can get all
> providers to
> match the same behavior in this case.
>
> On Tue, Jan 20, 2026 at 11:26 PM Andrii Lomakin via dev <
> dev@tinkerpop.apache.org> wrote:
>
> > Hi Ken.
> > Thank you for sharing this with us.
> >
> > After some thought, I agree that we need explicit TX management
> > methods.
> > From my perspective, what I wrote is only the tip of the iceberg,
> > as such
> > questions as consistency of the queries and validity of the data
> > contained
> > in vertices directly bound to the concept of TX lifecycle.
> >
> > Example (it can be applied to any database, not just a graph
> > database, so I
> > will use the general record concept).
> >
> > I have created record A { id: 1, value: "val1"} , committed it, and
> > passed
> > it to another thread.
> > After that, another method updated the same record B {id: 1, value
> > :
> > "val2"} and committed it.
> >
> > A and B in this context object identity, not DB record identity.
> >
> > But I have the record A in another thread with potentially stale
> > data.
> > What then should another thread read when it accesses `A.value`?
> >
> > I doubt it should read stale data. In our database, records are
> > automatically switched to the unloaded state after commit and can
> > be read
> > in a new thread only when a new transaction is started.
> > The question of record consistency, from my point of view, is an
> > important
> > one and cannot be overlooked, as it influences how users interact
> > with the
> > TP API and to what degree it is really platform-independent.
> >
> >
> > On Wed, Jan 21, 2026 at 7:58 AM Ken Hu <ke...@gmail.com> wrote:
> >
> > > Here are some of my current thoughts on the proposed changes and
> > > discussion.
> > >
> > > -deprecate open() and replace with begin()
> > > -I agree with this as having both is confusing and unncessary
> > >
> > > -Transaction class is deprecated, and TX control functionality is
> > delegated
> > > to the GraphTraversal/Graph instances
> > > -I mostly agree with this as well, but I think it's probably
> > > better
> > if
> > > it was delegated to GraphTraversalSource.
> > >
> > > -tx() method is deprecated.
> > > -I think this makes sense as I always found how tx() works
> > > with
> > remote
> > > to be unintuitive.
> > >
> > > -Transaction scope is limited to the current thread only
> > > -I had some questions about this earlier, but having thought
> > > about it
> > > after seeing Andrii's architectural diagram, it's probably fine
> > > since
> > most
> > > developers (in all languages TinkerPop supports) will be used to
> > > this
> > being
> > > how it works as long as we have something that represents what a
> > connection
> > > is in other databases.
> > >
> > > -I believe Non-TX mode is not appropriate for handling graph data
> > > structures.
> > > -Further, I'd say that transactions should be optional. Many
> > > persistent
> > > db/graph systems are eventually consistent and do not support
> > transactions,
> > > so they should not be burdened with concepts foreign to them.
> > > -I tend to agree with Pieter here. My reasoning is different
> > > though,
> > > one of the use cases for graphs is analytics and transactions
> > > aren't
> > really
> > > used in that case.
> > >
> > > -I think we DO need methods for explicit transaction management,
> > > but 90%
> > of
> > > the cases will be covered by implicit transaction management
> > > throughout
> > the
> > > GraphTraversal life cycle.
> > > -What I propose is to use lambda-controlled TXs
> > > -My idea is that the transaction api should be unopinionated.
> > > That means
> > it
> > > should follow closely to `UserTransaction`
> > > -I think having implicit transactions is fine as long as it's
> > > just an
> > > option. I'm not sure which should be the default. I'm not sure we
> > > can
> > make
> > > lambda-controlled transactions work though since lambdas wouldn't
> > > be
> > > compatible with Gremlin scripts. At this point, we probably still
> > > want
> > > parity between scripts and GraphTraversals.
> > >
> > > I think we need some more proposals for what the explicit
> > > transaction
> > > management API could look like because we probably can't move
> > > ahead with
> > > lambda-controlled TX for now and the current tx() solution is
> > > clunky.
> > I'll
> > > try to propose one in the next couple of days.
> > >
> > > On Wed, Jan 7, 2026 at 12:24 PM pieter <pi...@riseup.net>
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > I had a brief discussion with Andrii in the discord chat
> > > > regarding the
> > > > transaction api.
> > > >
> > > > My idea is that the transaction api should be unopinionated.
> > > > That means it should follow closely to
> > > > `javax.transaction.UserTransaction` which in turn means we have
> > > > a
> > > > dedicated class modelling the transaction with `begin()`,
> > > > `commit()`
> > > > and `rollback()`.
> > > >
> > > > My reasoning is that gremlin is a low level graph language.
> > > > There are
> > > > many ways different developers / architects can go about
> > > > integrating
> > > > gremlin into their stack.
> > > >
> > > > Some like lambdas, some like IOT containers, some have domain
> > > > entity
> > > > frameworks, some use aspect orienated programming and on an on.
> > > >
> > > > It is common for IOT containers to manage transaction
> > > > boundaries and
> > > > lifecycles. Similarly it is common to use the web server's
> > > > interceptors
> > > > to manage transaction boundaries. And it is very common to do
> > > > it all
> > > > manually...
> > > >
> > > > Further, I'd say that transactions should be optional. Many
> > > > persistent
> > > > db/graph systems are eventually consistent and do not support
> > > > transactions, so they should not be burdened with concepts
> > > > foreign to
> > > > them.
> > > >
> > > > Regarding binding the transaction to the thread:
> > > > This is the only way I have ever worked so its entirely natural
> > > > to me.
> > > > However I can image that people want to defince their own
> > > > transaction
> > > > context, so perhaps we should not hardcode that part. Long ago
> > > > Gavin
> > > > King from Hibernate fame had the notion of `conversation`
> > > > context.
> > > > Perhaps such things are still around?
> > > >
> > > > Regards
> > > > Pieter
> > > >
> > > > On Thu, 2025-12-11 at 09:02 +0100, Andrii Lomakin via dev
> > > > wrote:
> > > > > Unfortunately, all images have been removed, so I am sending
> > > > > a link
> > > > > to the diagrams instead
> > > > > https://miro.com/app/board/uXjVGdd8m44=/?share_link_id=686668490384
> > > > > .
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Dec 11, 2025 at 8:53 AM Andrii Lomakin
> > > > > <an...@jetbrains.com> wrote:
> > > > > > Good day.
> > > > > >
> > > > > > If I followed Ken's line of thought correctly, I would like
> > > > > > to
> > > > > > expand it to be part of the new proposal for the
> > > > > > transaction API.
> > > > > > First of all, about the model, I think we came to a
> > > > > > consensus that
> > > > > > it should be:
> > > > > >
> > > > > >
> > > > > > image.png
> > > > > > The point is that we do not have a notion of connection in
> > > > > > TinkerPop, but we have a notion of GraphTraversal.
> > > > > > So, from my perspective, this diagram can be changed to
> > > > > > image.png
> > > > > >
> > > > > > There is a corner case for Non-TX databases that we need to
> > > > > > discuss.
> > > > > > I believe Non-TX mode is not appropriate for handling graph
> > > > > > data
> > > > > > structures. GQL, for example, requires serializable
> > > > > > isolation to
> > > > > > satisfy the standard requirements.
> > > > > > Instead, we should require to have presence of a
> > > > > > transaction, which
> > > > > > is started implicitly when GraphTraversal is created and
> > > > > > committed
> > > > > > once it is closed.
> > > > > >
> > > > > > I think we DO need methods for explicit transaction
> > > > > > management, but
> > > > > > 90% of the cases will be covered by implicit transaction
> > > > > > management
> > > > > > throughout the GraphTraversal life cycle.
> > > > > > 1. TX is started when GraphTraversal is created.
> > > > > > 2. TX is committed when GraphTraversal is closed (which
> > > > > > includes
> > > > > > terminal operations like toList or iterate).
> > > > > > 3. TX is rolled back in case of exceptional execution of
> > > > > > GraphTraversal. Users can always roll back a transaction by
> > > > > > using
> > > > > > the fail() step.
> > > > > >
> > > > > > We also provide in the YTDB the following methods that I
> > > > > > hope will
> > > > > > become part of the framework: gSource.computeInTx(g ->{}),
> > > > > > gSource.executeInTx(g - {}), gSource.autoExecuteInTx(g -
> > > > > > {}: g)
> > > > > > (this one executes g.iterate() on the returned
> > > > > > GraphTraversal,
> > > > > > which we find really handy).
> > > > > >
> > > > > > Now, regarding explicit TX management.
> > > > > > I think that explicit TX management is still needed if you
> > > > > > want to
> > > > > > execute several queries and mix them with complex logic,
> > > > > > but I
> > > > > > think that methods that explicitly cause TX
> > > > > > begin/start/commit are
> > > > > > confusing and provide bad DevX.
> > > > > > What I propose is to use lambda-controlled TXs (slightly
> > > > > > different
> > > > > > from the above). I think that an explicit rollback/commit
> > > > > > of TX by
> > > > > > the user is similar to the goto operator in old programming
> > > > > > languages; it gives finer control over TXs, but overall
> > > > > > only adds
> > > > > > confusion and can be completely avoided in modern
> > > > > > languages.
> > > > > >
> > > > > > Explicit control on TXs is proposed to be implemented using
> > > > > > methods
> > > > > > similar to above, with a different signature, namely:
> > > > > > gSource.executeInTx(graphTraversalSupplier -> {
> > > > > > var g = graphTraversalSupplier.get();
> > > > > > var v = g.V().next();
> > > > > > g.close()
> > > > > >
> > > > > > v.property("p", "val");
> > > > > > int i = v.value("int");
> > > > > >
> > > > > > if(i > 0) {
> > > > > > g = graphTraversalSupplier.get();
> > > > > > g.V().has("i2", i).property("i2", i +
> > > > > > 1).iterate();
> > > > > > }
> > > > > > })
> > > > > >
> > > > > > In this case, the transaction is detached from
> > > > > > GraphTraversal, and
> > > > > > its lifecycle is not controlled by GraphTraversal.
> > > > > > Instead, TX is committed if an exception is not thrown from
> > > > > > the
> > > > > > lambda method.
> > > > > >
> > > > > > To inform the user if GraphTraversal controls TX or not, a
> > > > > > simple
> > > > > > isTxAttached method is proposed to be introduced
> > > > > > gSource.<V>computeInTX(graphTraversalSupplier -> {}) works
> > > > > > in
> > > > > > similar way but returns value.
> > > > > >
> > > > > >
> > > > > > So the final diagram could look like the following:
> > > > > >
> > > > > >
> > > > > > image.png
> > > > > >
> > > > > > Please note that in this model, there is no need for an
> > > > > > explicit
> > > > > > transaction object.
> > > > > >
> > > > > > On Fri, Dec 5, 2025 at 10:18 AM Andrii Lomakin
> > > > > > <an...@jetbrains.com> wrote:
> > > > > > > Good day, Ken.
> > > > > > >
> > > > > > > > This could map to multiple threads on the provider
> > > > > > > > side. It's
> > > > > > > probably OK if something like "g.tx()..." returns the
> > > > > > > same
> > > > > > > transaction. We could require users to open multiple
> > > > > > > DriverRemoteConnections/Clients to achieve this and that
> > > > > > > would be
> > > > > > > similar to other data connectivity APIs.
> > > > > > >
> > > > > > > Unfortunately I do not follow, could you elaborate more
> > > > > > > or give
> > > > > > > me example how do you see it ?
> > > > > > >
> > > > > > > On Thu, Dec 4, 2025 at 7:06 PM Ken Hu
> > > > > > > <ke...@gmail.com>
> > wrote:
> > > > > > > > I might be thinking about this too much for just the
> > > > > > > > remote
> > > > > > > > case and not considering the embedded case enough. In
> > > > > > > > my mind,
> > > > > > > > there just needs to be an easy way for users of the
> > > > > > > > language
> > > > > > > > variants (in particular JavaScript because of its
> > > > > > > > single
> > > > > > > > threaded nature) to start multiple transactions from a
> > > > > > > > single
> > > > > > > > thread. This could map to multiple threads on the
> > > > > > > > provider
> > > > > > > > side. It's probably OK if something like "g.tx()..."
> > > > > > > > returns
> > > > > > > > the same transaction. We could require users to open
> > > > > > > > multiple
> > > > > > > > DriverRemoteConnections/Clients to achieve this and
> > > > > > > > that would
> > > > > > > > be similar to other data connectivity APIs.
> > > > > > > >
> > > > > > > > On Thu, Dec 4, 2025 at 2:37 AM Andrii Lomakin via dev
> > > > > > > > <de...@tinkerpop.apache.org> wrote:
> > > > > > > > > The point about having multiple concurrent
> > > > > > > > > transactions
> > > > > > > > > within the same
> > > > > > > > > thread warrants a more detailed discussion.
> > > > > > > > >
> > > > > > > > > To compare this behavior, consider how major SQL
> > > > > > > > > databases
> > > > > > > > > handle attempts
> > > > > > > > > to start a second transaction when one is already
> > > > > > > > > active:
> > > > > > > > >
> > > > > > > > > 1. Error out: The database prevents the operation,
> > > > > > > > > signaling
> > > > > > > > > that a
> > > > > > > > > transaction is already active (e.g., "Transaction
> > > > > > > > > already
> > > > > > > > > active").
> > > > > > > > > 2. Create a nested transaction (savepoint): The
> > > > > > > > > database
> > > > > > > > > creates a
> > > > > > > > > sub-scope of the first transaction, not an
> > > > > > > > > independent
> > > > > > > > > transaction. If the
> > > > > > > > > parent transaction fails, the child scope also fails.
> > > > > > > > > 3. Implicitly commit the first: The database
> > > > > > > > > automatically
> > > > > > > > > commits the
> > > > > > > > > first transaction to open the second one (a common
> > > > > > > > > behavior
> > > > > > > > > in MySQL DDL).
> > > > > > > > >
> > > > > > > > > As far as I know, only Firebird has historically
> > > > > > > > > allowed
> > > > > > > > > several truly
> > > > > > > > > independent transactions concurrently on the same
> > > > > > > > > thread. I
> > > > > > > > > suspect support
> > > > > > > > > for simultaneous transactions may be even more
> > > > > > > > > limited among
> > > > > > > > > graph database
> > > > > > > > > vendors.
> > > > > > > > >
> > > > > > > > > On Thu, Dec 4, 2025 at 10:47 AM Andrii Lomakin
> > > > > > > > > <an...@jetbrains.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Good day.
> > > > > > > > > > As for questions from Ken
> > > > > > > > > > > What OGMs are you referring to here? Regular
> > > > > > > > > > > transactions
> > > > > > > > > are complicated
> > > > > > > > > > enough by itself, I would only want to introduce
> > > > > > > > > > nested
> > > > > > > > > transactions if
> > > > > > > > > > there were well maintained OGMs that would have a
> > > > > > > > > demonstrable benefit from
> > > > > > > > > > this.
> > > > > > > > > >
> > > > > > > > > > We have such an OGM using TinkerPop that is well
> > > > > > > > > maintained, if you mean
> > > > > > > > > > by this years of support and thousands of instances
> > > > > > > > > > running
> > > > > > > > > on OS version.
> > > > > > > > > > However, I have given it thought and believe that
> > > > > > > > > > it is a
> > > > > > > > > relatively rare
> > > > > > > > > > case, and OGM is so complex that it can manage
> > > > > > > > > > nested
> > > > > > > > > transactions itself
> > > > > > > > > > if needed.
> > > > > > > > > > I think we can support nested transactions on pause
> > > > > > > > > > for
> > > > > > > > > now, till we
> > > > > > > > > > receive more feedback from users.
> > > > > > > > > >
> > > > > > > > > > There are many inconsistencies to discuss in the TX
> > > > > > > > > lifecycle, even
> > > > > > > > > > without nested transactions.
> > > > > > > > > >
> > > > > > > > > > > Sessions won't exist moving forward (4.x+) as
> > > > > > > > > > > they are
> > > > > > > > > tied to remote
> > > > > > > > > > Groovy execution. So let's continue this
> > > > > > > > > > conversation as
> > > > > > > > > simply
> > > > > > > > > > transactions and not refer to sessions at all to
> > > > > > > > > > prevent
> > > > > > > > > confusion, unless
> > > > > > > > > > you intend to have these changes in the 3.8.x line?
> > > > > > > > > > Yeah, we already removed Groovy support in our
> > > > > > > > > > fork, so it
> > > > > > > > > is even now in
> > > > > > > > > > our distribution TX, and the session is the same.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > > Could you expand a bit more on "Transaction scope
> > > > > > > > > > > is
> > > > > > > > > limited to the
> > > > > > > > > > current thread only".
> > > > > > > > > > I mean that TX visibility and manipulation are
> > > > > > > > > > limited to a
> > > > > > > > > single thread
> > > > > > > > > > where it is created.
> > > > > > > > > > If some of the providers support TX visibility
> > > > > > > > > > between
> > > > > > > > > several threads,
> > > > > > > > > > that will only add a bonus for them if such a need
> > > > > > > > > > arises,
> > > > > > > > > of course,
> > > > > > > > > > without breaking the API.
> > > > > > > > > >
> > > > > > > > > > > I find it limiting that an attempt to open
> > > > > > > > > > > multiple
> > > > > > > > > transactions from
> > > > > > > > > > the same thread ends up returning the same
> > > > > > > > > > Transaction
> > > > > > > > > instance
> > > > > > > > > > Do you propose to support several transactions in
> > > > > > > > > > the same
> > > > > > > > > thread?
> > > > > > > > > > I am afraid not all current vendors can support
> > > > > > > > > > this model.
> > > > > > > > > >
> > > > > > > > > > On Wed, Dec 3, 2025 at 1:23 PM Stephen Mallette
> > > > > > > > > <sp...@gmail.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Andrii, I'd like to just quickly clarify Ken's
> > > > > > > > > > > point
> > > > > > > > > stating " Sessions
> > > > > > > > > > > won't exist moving forward (4.x+) ". He's saying
> > > > > > > > > > > what you
> > > > > > > > > said: "
> > > > > > > > > > > 'session'
> > > > > > > > > > > and 'tx' are essentially the same concepts". We
> > > > > > > > > acknowledged that and
> > > > > > > > > > > decided to stop using "session" terminology and
> > > > > > > > > > > instead
> > > > > > > > > use "transaction"
> > > > > > > > > > > terminology only to avoid confusion.
> > > > > > > > > > >
> > > > > > > > > > > Much of what you wrote is in the spirit of what
> > > > > > > > > > > has been
> > > > > > > > > considered for
> > > > > > > > > > > 4.x, so that is nice alignment. I do think the
> > > > > > > > > > > goal though
> > > > > > > > > is to have a
> > > > > > > > > > > consistent transaction API/system that works for
> > > > > > > > > > > both
> > > > > > > > > embedded and remote
> > > > > > > > > > > cases. It will be interesting to see how that is
> > > > > > > > > > > achieved.
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Dec 2, 2025 at 3:53 PM Ken Hu
> > > > > > > > > > > <kenhuoss@gmail.com
> > >
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Andrii,
> > > > > > > > > > > >
> > > > > > > > > > > > Good day to you as well. I have a couple
> > > > > > > > > > > > questions about
> > > > > > > > > this that I
> > > > > > > > > > > would
> > > > > > > > > > > > like some more details on.
> > > > > > > > > > > >
> > > > > > > > > > > > 1. What OGMs are you referring to here? Regular
> > > > > > > > > transactions are
> > > > > > > > > > > > complicated enough by itself, I would only want
> > > > > > > > > > > > to
> > > > > > > > > introduce nested
> > > > > > > > > > > > transactions if there were well maintained OGMs
> > > > > > > > > > > > that
> > > > > > > > > would have a
> > > > > > > > > > > > demonstrable benefit from this.
> > > > > > > > > > > > 2. I like some of your ideas about removing
> > > > > > > > > > > > Transaction
> > > > > > > > > as its own
> > > > > > > > > > > class,
> > > > > > > > > > > > but I'm not sure GraphTraversalSource is where
> > > > > > > > > > > > it should
> > > > > > > > > be instead. Do
> > > > > > > > > > > you
> > > > > > > > > > > > have some more details about what you think the
> > > > > > > > > Transaction API should
> > > > > > > > > > > look
> > > > > > > > > > > > like?
> > > > > > > > > > > > 3. Could you expand a bit more on "Transaction
> > > > > > > > > > > > scope is
> > > > > > > > > limited to the
> > > > > > > > > > > > current thread only". I always get confused by
> > > > > > > > > TinkerPop's threaded vs
> > > > > > > > > > > > multithreaded transaction. I find it limiting
> > > > > > > > > > > > that an
> > > > > > > > > attempt to open
> > > > > > > > > > > > multiple transactions from the same thread ends
> > > > > > > > > > > > up
> > > > > > > > > returning the same
> > > > > > > > > > > > Transaction instance. Maybe I didn't quite
> > > > > > > > > > > > understand
> > > > > > > > > what you meant by
> > > > > > > > > > > > this.
> > > > > > > > > > > > 4. Sessions won't exist moving forward (4.x+)
> > > > > > > > > > > > as they
> > > > > > > > > are tied to remote
> > > > > > > > > > > > Groovy execution. So let's continue this
> > > > > > > > > > > > conversation as
> > > > > > > > > simply
> > > > > > > > > > > > transactions and not refer to sessions at all
> > > > > > > > > > > > to prevent
> > > > > > > > > confusion,
> > > > > > > > > > > unless
> > > > > > > > > > > > you intend to have these changes in the 3.8.x
> > > > > > > > > > > > line?
> > > > > > > > > > > >
> > > > > > > > > > > > Regards,
> > > > > > > > > > > > Ken
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, Dec 2, 2025 at 3:33 AM Andrii Lomakin
> > > > > > > > > > > > via dev <
> > > > > > > > > > > > dev@tinkerpop.apache.org> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Small clarification.
> > > > > > > > > > > > > Implementation notes are related to changes
> > > > > > > > > > > > > in the
> > > > > > > > > remote protocol.
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Tue, Dec 2, 2025 at 8:55 AM Andrii Lomakin
> > > > > > > > > > > > > <
> > > > > > > > > > > > > andrey.lomakin@jetbrains.com>
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > To clarify: in this model, 'session' and
> > > > > > > > > > > > > > 'tx' are
> > > > > > > > > essentially the
> > > > > > > > > > > same
> > > > > > > > > > > > > > concepts.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I also think it is essential to have a
> > > > > > > > > > > > > > common
> > > > > > > > > denominator that
> > > > > > > > > > > makes TP
> > > > > > > > > > > > > > applications portable across
> > > > > > > > > > > > > > implementations.
> > > > > > > > > > > > > > That is why I also propose to restrict
> > > > > > > > > > > > > > transaction
> > > > > > > > > scope to the
> > > > > > > > > > > thread
> > > > > > > > > > > > > > scope. It will be a common denominator that
> > > > > > > > > > > > > > will not
> > > > > > > > > harm user
> > > > > > > > > > > > experience
> > > > > > > > > > > > > > but will ensure uniform and expected API
> > > > > > > > > > > > > > behavior.
> > > > > > > > > > > > > > There should not be both `begin` and `open`
> > > > > > > > > > > > > > methods
> > > > > > > > > because the
> > > > > > > > > > > > contract
> > > > > > > > > > > > > > of the beginning method is vague and
> > > > > > > > > > > > > > unpredictable.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Tue, Dec 2, 2025 at 8:18 AM Andrii
> > > > > > > > > > > > > > Lomakin <
> > > > > > > > > > > > > > andrey.lomakin@jetbrains.com> wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Good day,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I have a proposal to enhance the
> > > > > > > > > > > > > > > transaction
> > > > > > > > > handling logic in TP,
> > > > > > > > > > > > > > > specifically regarding nested transaction
> > > > > > > > > > > > > > > calls,
> > > > > > > > > which are common
> > > > > > > > > > > in
> > > > > > > > > > > > EE
> > > > > > > > > > > > > > > development, particularly when utilizing
> > > > > > > > > > > > > > > OGM.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Currently, calling open multiple times
> > > > > > > > > > > > > > > causes an
> > > > > > > > > exception. This
> > > > > > > > > > > > > behavior
> > > > > > > > > > > > > > > is restrictive, and the term "open" can
> > > > > > > > > > > > > > > be
> > > > > > > > > confusing.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I propose a design that improves the
> > > > > > > > > > > > > > > developer
> > > > > > > > > experience by using
> > > > > > > > > > > an
> > > > > > > > > > > > > > > internal counter for transaction calls:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Proposed transaction counter logic
> > > > > > > > > > > > > > > 1. begin/commit counter:
> > > > > > > > > > > > > > > - begin would increment an internal
> > > > > > > > > > > > > > > counter.
> > > > > > > > > > > > > > > - commit would decrement the internal
> > > > > > > > > > > > > > > counter,
> > > > > > > > > causing the actual
> > > > > > > > > > > > > > > transaction commit only when the counter
> > > > > > > > > > > > > > > reaches
> > > > > > > > > zero.
> > > > > > > > > > > > > > > 2. rollback behavior:
> > > > > > > > > > > > > > > - rollback would immediately force a
> > > > > > > > > > > > > > > transaction
> > > > > > > > > rollback and
> > > > > > > > > > > > > decrements
> > > > > > > > > > > > > > > the internal counter
> > > > > > > > > > > > > > > - the subsequent calls to rollback on
> > > > > > > > > > > > > > > the same
> > > > > > > > > transaction object
> > > > > > > > > > > > > should
> > > > > > > > > > > > > > > be allowed until the stack trace reaches
> > > > > > > > > > > > > > > the
> > > > > > > > > initial begin method
> > > > > > > > > > > > call,
> > > > > > > > > > > > > > > after which further rollback calls would
> > > > > > > > > > > > > > > throw an
> > > > > > > > > exception.
> > > > > > > > > > > > > > > 3. Error handling:
> > > > > > > > > > > > > > > - A commit without a matching begin
> > > > > > > > > > > > > > > (e.g., counter
> > > > > > > > > is already zero)
> > > > > > > > > > > > > > > should throw an exception.
> > > > > > > > > > > > > > > - A rollback without a matching begin
> > > > > > > > > > > > > > > should throw
> > > > > > > > > an exception
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I would also consider hiding the
> > > > > > > > > > > > > > > Transaction object
> > > > > > > > > altogether and
> > > > > > > > > > > > > > > delegating the logic of transaction
> > > > > > > > > > > > > > > management to
> > > > > > > > > > > GraphTraversalSource
> > > > > > > > > > > > > it
> > > > > > > > > > > > > > > will simplify the API a lot, both for
> > > > > > > > > > > > > > > users and
> > > > > > > > > vendors.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I also propose to restrict transaction
> > > > > > > > > > > > > > > scope to the
> > > > > > > > > thread scope,
> > > > > > > > > > > it
> > > > > > > > > > > > > will
> > > > > > > > > > > > > > > be a common denominator that will not
> > > > > > > > > > > > > > > harm user
> > > > > > > > > experience but will
> > > > > > > > > > > > > ensure
> > > > > > > > > > > > > > > uniform and expected API behavior across
> > > > > > > > > > > > > > > all
> > > > > > > > > implementations.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Proposed API changes
> > > > > > > > > > > > > > > - I propose deprecating open and using
> > > > > > > > > > > > > > > begin with
> > > > > > > > > the semantics
> > > > > > > > > > > > > > > described above to avoid confusion.
> > > > > > > > > > > > > > > - Transaction object is deprecated, and
> > > > > > > > > > > > > > > TX control
> > > > > > > > > functionality is
> > > > > > > > > > > > > > > delegated to the GraphTraversal/Graph
> > > > > > > > > > > > > > > instances.
> > > > > > > > > > > > > > > - tx() method is deprecated.
> > > > > > > > > > > > > > > - Transaction scope is limited to the
> > > > > > > > > > > > > > > current
> > > > > > > > > thread only.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Behavior for non-direct calls
> > > > > > > > > > > > > > > When begin/commit/rollback methods are
> > > > > > > > > > > > > > > not called
> > > > > > > > > directly, the
> > > > > > > > > > > > > > > transaction should be automatically
> > > > > > > > > > > > > > > committed by a
> > > > > > > > > terminal
> > > > > > > > > > > operation
> > > > > > > > > > > > > > > (e.g., when hasNext returns false) in
> > > > > > > > > > > > > > > both remote
> > > > > > > > > and embedded
> > > > > > > > > > > modes.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Implementation notes:
> > > > > > > > > > > > > > > 1. The begin command should be added.
> > > > > > > > > > > > > > > Relying on
> > > > > > > > > the implicit
> > > > > > > > > > > start of
> > > > > > > > > > > > > TX
> > > > > > > > > > > > > > > by traversal can lead to non-controlled
> > > > > > > > > > > > > > > side
> > > > > > > > > effects.
> > > > > > > > > > > > > > > 2. For the sake of optimization of the
> > > > > > > > > > > > > > > remote
> > > > > > > > > protocol, while
> > > > > > > > > > > multiple
> > > > > > > > > > > > > > > calls of begin/commit/rollback are
> > > > > > > > > > > > > > > allowed in the
> > > > > > > > > remote protocol,
> > > > > > > > > > > > > > > practically, it will mean that we will
> > > > > > > > > > > > > > > track the
> > > > > > > > > counter locally on
> > > > > > > > > > > > the
> > > > > > > > > > > > > > > client and will send
> > > > > > > > > > > > > > > begin/commit/rollback only
> > > > > > > > > once for the TX
> > > > > > > > > > > > > lifecycle.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > This model should be uniform across all
> > > > > > > > > > > > > > > deployment
> > > > > > > > > types.
> > > > > > > > > > > > > > > What are your thoughts on this approach?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > -----
> > > > > > > > > > > > > > > Andrii Lomakin
> > > > > > > > > > > > > > > YouTrackDB development lead
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > Andrii Lomakin
> > > > > > > > > > > > > > YouTrackDB development lead
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > Andrii Lomakin
> > > > > > > > > > > > > YouTrackDB development lead
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Andrii Lomakin
> > > > > > > > > > YouTrackDB development lead
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > >
> > >
> >
> >
> > --
> > Andrii Lomakin
> > YouTrackDB development lead
> >
Re: [DISCUSS] Handling of transactions - proposal
Posted by Andrii Lomakin via dev <de...@tinkerpop.apache.org>.
Good day, Ken.
Thank you for your prompt response.
The more I think about it and see discussions around this topic, the more I
think that, because TP supports a very wide variety of DBs, it is, in some
sense, soft-locked the evolution of this topic.
I am more inclined toward a minimal but consistent API rather than a strict
life-cycle definition.
But I suggest implementing the lambda-controlled TXs as a usability
feature, since our users (not just YTDB) treat it as a good DevX experience.
On Thu, Jan 22, 2026 at 5:40 AM Ken Hu <ke...@gmail.com> wrote:
> I agree that record consistency is important, but I feel that where this
> gets solved depends on how users integrate graph databases into their
> applications. There are probably frameworks and OGM/ORMs that can do this
> detaching/unloading that you are mentioning as a solution. If a user
> directly integrates the database as an external service using a driver,
> then having it as part of the API could be useful. I need to think about
> this more.
>
> To your second point about platform-independence, I think this has always
> been a goal of TinkerPop that has never quite been realized. Even today,
> there are some examples of Gremlin that return results that are
> provider-dependent and so there are specific cases where it isn't truly
> portable. I agree that a difference in read consistency amongst
> vendors/providers means those using transactions can't move between
> different databases unless they find another database with the exact same
> transaction behavior. But I'm not entirely sure we can get all providers to
> match the same behavior in this case.
>
> On Tue, Jan 20, 2026 at 11:26 PM Andrii Lomakin via dev <
> dev@tinkerpop.apache.org> wrote:
>
>> Hi Ken.
>> Thank you for sharing this with us.
>>
>> After some thought, I agree that we need explicit TX management methods.
>> From my perspective, what I wrote is only the tip of the iceberg, as such
>> questions as consistency of the queries and validity of the data contained
>> in vertices directly bound to the concept of TX lifecycle.
>>
>> Example (it can be applied to any database, not just a graph database, so
>> I
>> will use the general record concept).
>>
>> I have created record A { id: 1, value: "val1"} , committed it, and passed
>> it to another thread.
>> After that, another method updated the same record B {id: 1, value :
>> "val2"} and committed it.
>>
>> A and B in this context object identity, not DB record identity.
>>
>> But I have the record A in another thread with potentially stale data.
>> What then should another thread read when it accesses `A.value`?
>>
>> I doubt it should read stale data. In our database, records are
>> automatically switched to the unloaded state after commit and can be read
>> in a new thread only when a new transaction is started.
>> The question of record consistency, from my point of view, is an important
>> one and cannot be overlooked, as it influences how users interact with the
>> TP API and to what degree it is really platform-independent.
>>
>>
>> On Wed, Jan 21, 2026 at 7:58 AM Ken Hu <ke...@gmail.com> wrote:
>>
>> > Here are some of my current thoughts on the proposed changes and
>> > discussion.
>> >
>> > -deprecate open() and replace with begin()
>> > -I agree with this as having both is confusing and unncessary
>> >
>> > -Transaction class is deprecated, and TX control functionality is
>> delegated
>> > to the GraphTraversal/Graph instances
>> > -I mostly agree with this as well, but I think it's probably better
>> if
>> > it was delegated to GraphTraversalSource.
>> >
>> > -tx() method is deprecated.
>> > -I think this makes sense as I always found how tx() works with
>> remote
>> > to be unintuitive.
>> >
>> > -Transaction scope is limited to the current thread only
>> > -I had some questions about this earlier, but having thought about
>> it
>> > after seeing Andrii's architectural diagram, it's probably fine since
>> most
>> > developers (in all languages TinkerPop supports) will be used to this
>> being
>> > how it works as long as we have something that represents what a
>> connection
>> > is in other databases.
>> >
>> > -I believe Non-TX mode is not appropriate for handling graph data
>> > structures.
>> > -Further, I'd say that transactions should be optional. Many persistent
>> > db/graph systems are eventually consistent and do not support
>> transactions,
>> > so they should not be burdened with concepts foreign to them.
>> > -I tend to agree with Pieter here. My reasoning is different though,
>> > one of the use cases for graphs is analytics and transactions aren't
>> really
>> > used in that case.
>> >
>> > -I think we DO need methods for explicit transaction management, but
>> 90% of
>> > the cases will be covered by implicit transaction management throughout
>> the
>> > GraphTraversal life cycle.
>> > -What I propose is to use lambda-controlled TXs
>> > -My idea is that the transaction api should be unopinionated. That
>> means it
>> > should follow closely to `UserTransaction`
>> > -I think having implicit transactions is fine as long as it's just
>> an
>> > option. I'm not sure which should be the default. I'm not sure we can
>> make
>> > lambda-controlled transactions work though since lambdas wouldn't be
>> > compatible with Gremlin scripts. At this point, we probably still want
>> > parity between scripts and GraphTraversals.
>> >
>> > I think we need some more proposals for what the explicit transaction
>> > management API could look like because we probably can't move ahead with
>> > lambda-controlled TX for now and the current tx() solution is clunky.
>> I'll
>> > try to propose one in the next couple of days.
>> >
>> > On Wed, Jan 7, 2026 at 12:24 PM pieter <pi...@riseup.net>
>> wrote:
>> >
>> > > Hi,
>> > >
>> > > I had a brief discussion with Andrii in the discord chat regarding the
>> > > transaction api.
>> > >
>> > > My idea is that the transaction api should be unopinionated.
>> > > That means it should follow closely to
>> > > `javax.transaction.UserTransaction` which in turn means we have a
>> > > dedicated class modelling the transaction with `begin()`, `commit()`
>> > > and `rollback()`.
>> > >
>> > > My reasoning is that gremlin is a low level graph language. There are
>> > > many ways different developers / architects can go about integrating
>> > > gremlin into their stack.
>> > >
>> > > Some like lambdas, some like IOT containers, some have domain entity
>> > > frameworks, some use aspect orienated programming and on an on.
>> > >
>> > > It is common for IOT containers to manage transaction boundaries and
>> > > lifecycles. Similarly it is common to use the web server's
>> interceptors
>> > > to manage transaction boundaries. And it is very common to do it all
>> > > manually...
>> > >
>> > > Further, I'd say that transactions should be optional. Many persistent
>> > > db/graph systems are eventually consistent and do not support
>> > > transactions, so they should not be burdened with concepts foreign to
>> > > them.
>> > >
>> > > Regarding binding the transaction to the thread:
>> > > This is the only way I have ever worked so its entirely natural to me.
>> > > However I can image that people want to defince their own transaction
>> > > context, so perhaps we should not hardcode that part. Long ago Gavin
>> > > King from Hibernate fame had the notion of `conversation` context.
>> > > Perhaps such things are still around?
>> > >
>> > > Regards
>> > > Pieter
>> > >
>> > > On Thu, 2025-12-11 at 09:02 +0100, Andrii Lomakin via dev wrote:
>> > > > Unfortunately, all images have been removed, so I am sending a link
>> > > > to the diagrams instead
>> > > > https://miro.com/app/board/uXjVGdd8m44=/?share_link_id=686668490384
>> > > > .
>> > > >
>> > > >
>> > > >
>> > > > On Thu, Dec 11, 2025 at 8:53 AM Andrii Lomakin
>> > > > <an...@jetbrains.com> wrote:
>> > > > > Good day.
>> > > > >
>> > > > > If I followed Ken's line of thought correctly, I would like to
>> > > > > expand it to be part of the new proposal for the transaction API.
>> > > > > First of all, about the model, I think we came to a consensus that
>> > > > > it should be:
>> > > > >
>> > > > >
>> > > > > image.png
>> > > > > The point is that we do not have a notion of connection in
>> > > > > TinkerPop, but we have a notion of GraphTraversal.
>> > > > > So, from my perspective, this diagram can be changed to
>> > > > > image.png
>> > > > >
>> > > > > There is a corner case for Non-TX databases that we need to
>> > > > > discuss.
>> > > > > I believe Non-TX mode is not appropriate for handling graph data
>> > > > > structures. GQL, for example, requires serializable isolation to
>> > > > > satisfy the standard requirements.
>> > > > > Instead, we should require to have presence of a transaction,
>> which
>> > > > > is started implicitly when GraphTraversal is created and committed
>> > > > > once it is closed.
>> > > > >
>> > > > > I think we DO need methods for explicit transaction management,
>> but
>> > > > > 90% of the cases will be covered by implicit transaction
>> management
>> > > > > throughout the GraphTraversal life cycle.
>> > > > > 1. TX is started when GraphTraversal is created.
>> > > > > 2. TX is committed when GraphTraversal is closed (which includes
>> > > > > terminal operations like toList or iterate).
>> > > > > 3. TX is rolled back in case of exceptional execution of
>> > > > > GraphTraversal. Users can always roll back a transaction by using
>> > > > > the fail() step.
>> > > > >
>> > > > > We also provide in the YTDB the following methods that I hope will
>> > > > > become part of the framework: gSource.computeInTx(g ->{}),
>> > > > > gSource.executeInTx(g - {}), gSource.autoExecuteInTx(g - {}: g)
>> > > > > (this one executes g.iterate() on the returned GraphTraversal,
>> > > > > which we find really handy).
>> > > > >
>> > > > > Now, regarding explicit TX management.
>> > > > > I think that explicit TX management is still needed if you want to
>> > > > > execute several queries and mix them with complex logic, but I
>> > > > > think that methods that explicitly cause TX begin/start/commit are
>> > > > > confusing and provide bad DevX.
>> > > > > What I propose is to use lambda-controlled TXs (slightly different
>> > > > > from the above). I think that an explicit rollback/commit of TX by
>> > > > > the user is similar to the goto operator in old programming
>> > > > > languages; it gives finer control over TXs, but overall only adds
>> > > > > confusion and can be completely avoided in modern languages.
>> > > > >
>> > > > > Explicit control on TXs is proposed to be implemented using
>> methods
>> > > > > similar to above, with a different signature, namely:
>> > > > > gSource.executeInTx(graphTraversalSupplier -> {
>> > > > > var g = graphTraversalSupplier.get();
>> > > > > var v = g.V().next();
>> > > > > g.close()
>> > > > >
>> > > > > v.property("p", "val");
>> > > > > int i = v.value("int");
>> > > > >
>> > > > > if(i > 0) {
>> > > > > g = graphTraversalSupplier.get();
>> > > > > g.V().has("i2", i).property("i2", i + 1).iterate();
>> > > > > }
>> > > > > })
>> > > > >
>> > > > > In this case, the transaction is detached from GraphTraversal, and
>> > > > > its lifecycle is not controlled by GraphTraversal.
>> > > > > Instead, TX is committed if an exception is not thrown from the
>> > > > > lambda method.
>> > > > >
>> > > > > To inform the user if GraphTraversal controls TX or not, a simple
>> > > > > isTxAttached method is proposed to be introduced
>> > > > > gSource.<V>computeInTX(graphTraversalSupplier -> {}) works in
>> > > > > similar way but returns value.
>> > > > >
>> > > > >
>> > > > > So the final diagram could look like the following:
>> > > > >
>> > > > >
>> > > > > image.png
>> > > > >
>> > > > > Please note that in this model, there is no need for an explicit
>> > > > > transaction object.
>> > > > >
>> > > > > On Fri, Dec 5, 2025 at 10:18 AM Andrii Lomakin
>> > > > > <an...@jetbrains.com> wrote:
>> > > > > > Good day, Ken.
>> > > > > >
>> > > > > > >This could map to multiple threads on the provider side. It's
>> > > > > > probably OK if something like "g.tx()..." returns the same
>> > > > > > transaction. We could require users to open multiple
>> > > > > > DriverRemoteConnections/Clients to achieve this and that would
>> be
>> > > > > > similar to other data connectivity APIs.
>> > > > > >
>> > > > > > Unfortunately I do not follow, could you elaborate more or give
>> > > > > > me example how do you see it ?
>> > > > > >
>> > > > > > On Thu, Dec 4, 2025 at 7:06 PM Ken Hu <ke...@gmail.com>
>> wrote:
>> > > > > > > I might be thinking about this too much for just the remote
>> > > > > > > case and not considering the embedded case enough. In my mind,
>> > > > > > > there just needs to be an easy way for users of the language
>> > > > > > > variants (in particular JavaScript because of its single
>> > > > > > > threaded nature) to start multiple transactions from a single
>> > > > > > > thread. This could map to multiple threads on the provider
>> > > > > > > side. It's probably OK if something like "g.tx()..." returns
>> > > > > > > the same transaction. We could require users to open multiple
>> > > > > > > DriverRemoteConnections/Clients to achieve this and that would
>> > > > > > > be similar to other data connectivity APIs.
>> > > > > > >
>> > > > > > > On Thu, Dec 4, 2025 at 2:37 AM Andrii Lomakin via dev
>> > > > > > > <de...@tinkerpop.apache.org> wrote:
>> > > > > > > > The point about having multiple concurrent transactions
>> > > > > > > > within the same
>> > > > > > > > thread warrants a more detailed discussion.
>> > > > > > > >
>> > > > > > > > To compare this behavior, consider how major SQL databases
>> > > > > > > > handle attempts
>> > > > > > > > to start a second transaction when one is already active:
>> > > > > > > >
>> > > > > > > > 1. Error out: The database prevents the operation, signaling
>> > > > > > > > that a
>> > > > > > > > transaction is already active (e.g., "Transaction already
>> > > > > > > > active").
>> > > > > > > > 2. Create a nested transaction (savepoint): The database
>> > > > > > > > creates a
>> > > > > > > > sub-scope of the first transaction, not an independent
>> > > > > > > > transaction. If the
>> > > > > > > > parent transaction fails, the child scope also fails.
>> > > > > > > > 3. Implicitly commit the first: The database automatically
>> > > > > > > > commits the
>> > > > > > > > first transaction to open the second one (a common behavior
>> > > > > > > > in MySQL DDL).
>> > > > > > > >
>> > > > > > > > As far as I know, only Firebird has historically allowed
>> > > > > > > > several truly
>> > > > > > > > independent transactions concurrently on the same thread. I
>> > > > > > > > suspect support
>> > > > > > > > for simultaneous transactions may be even more limited among
>> > > > > > > > graph database
>> > > > > > > > vendors.
>> > > > > > > >
>> > > > > > > > On Thu, Dec 4, 2025 at 10:47 AM Andrii Lomakin
>> > > > > > > > <an...@jetbrains.com>
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Good day.
>> > > > > > > > > As for questions from Ken
>> > > > > > > > > >What OGMs are you referring to here? Regular transactions
>> > > > > > > > are complicated
>> > > > > > > > > enough by itself, I would only want to introduce nested
>> > > > > > > > transactions if
>> > > > > > > > > there were well maintained OGMs that would have a
>> > > > > > > > demonstrable benefit from
>> > > > > > > > > this.
>> > > > > > > > >
>> > > > > > > > > We have such an OGM using TinkerPop that is well
>> > > > > > > > maintained, if you mean
>> > > > > > > > > by this years of support and thousands of instances
>> running
>> > > > > > > > on OS version.
>> > > > > > > > > However, I have given it thought and believe that it is a
>> > > > > > > > relatively rare
>> > > > > > > > > case, and OGM is so complex that it can manage nested
>> > > > > > > > transactions itself
>> > > > > > > > > if needed.
>> > > > > > > > > I think we can support nested transactions on pause for
>> > > > > > > > now, till we
>> > > > > > > > > receive more feedback from users.
>> > > > > > > > >
>> > > > > > > > > There are many inconsistencies to discuss in the TX
>> > > > > > > > lifecycle, even
>> > > > > > > > > without nested transactions.
>> > > > > > > > >
>> > > > > > > > > >Sessions won't exist moving forward (4.x+) as they are
>> > > > > > > > tied to remote
>> > > > > > > > > Groovy execution. So let's continue this conversation as
>> > > > > > > > simply
>> > > > > > > > > transactions and not refer to sessions at all to prevent
>> > > > > > > > confusion, unless
>> > > > > > > > > you intend to have these changes in the 3.8.x line?
>> > > > > > > > > Yeah, we already removed Groovy support in our fork, so it
>> > > > > > > > is even now in
>> > > > > > > > > our distribution TX, and the session is the same.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > >Could you expand a bit more on "Transaction scope is
>> > > > > > > > limited to the
>> > > > > > > > > current thread only".
>> > > > > > > > > I mean that TX visibility and manipulation are limited to
>> a
>> > > > > > > > single thread
>> > > > > > > > > where it is created.
>> > > > > > > > > If some of the providers support TX visibility between
>> > > > > > > > several threads,
>> > > > > > > > > that will only add a bonus for them if such a need arises,
>> > > > > > > > of course,
>> > > > > > > > > without breaking the API.
>> > > > > > > > >
>> > > > > > > > > > I find it limiting that an attempt to open multiple
>> > > > > > > > transactions from
>> > > > > > > > > the same thread ends up returning the same Transaction
>> > > > > > > > instance
>> > > > > > > > > Do you propose to support several transactions in the same
>> > > > > > > > thread?
>> > > > > > > > > I am afraid not all current vendors can support this
>> model.
>> > > > > > > > >
>> > > > > > > > > On Wed, Dec 3, 2025 at 1:23 PM Stephen Mallette
>> > > > > > > > <sp...@gmail.com>
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > >> Andrii, I'd like to just quickly clarify Ken's point
>> > > > > > > > stating " Sessions
>> > > > > > > > >> won't exist moving forward (4.x+) ". He's saying what you
>> > > > > > > > said: "
>> > > > > > > > >> 'session'
>> > > > > > > > >> and 'tx' are essentially the same concepts". We
>> > > > > > > > acknowledged that and
>> > > > > > > > >> decided to stop using "session" terminology and instead
>> > > > > > > > use "transaction"
>> > > > > > > > >> terminology only to avoid confusion.
>> > > > > > > > >>
>> > > > > > > > >> Much of what you wrote is in the spirit of what has been
>> > > > > > > > considered for
>> > > > > > > > >> 4.x, so that is nice alignment. I do think the goal
>> though
>> > > > > > > > is to have a
>> > > > > > > > >> consistent transaction API/system that works for both
>> > > > > > > > embedded and remote
>> > > > > > > > >> cases. It will be interesting to see how that is
>> achieved.
>> > > > > > > > >>
>> > > > > > > > >> On Tue, Dec 2, 2025 at 3:53 PM Ken Hu <
>> kenhuoss@gmail.com>
>> > > > > > > > wrote:
>> > > > > > > > >>
>> > > > > > > > >> > Hi Andrii,
>> > > > > > > > >> >
>> > > > > > > > >> > Good day to you as well. I have a couple questions
>> about
>> > > > > > > > this that I
>> > > > > > > > >> would
>> > > > > > > > >> > like some more details on.
>> > > > > > > > >> >
>> > > > > > > > >> > 1. What OGMs are you referring to here? Regular
>> > > > > > > > transactions are
>> > > > > > > > >> > complicated enough by itself, I would only want to
>> > > > > > > > introduce nested
>> > > > > > > > >> > transactions if there were well maintained OGMs that
>> > > > > > > > would have a
>> > > > > > > > >> > demonstrable benefit from this.
>> > > > > > > > >> > 2. I like some of your ideas about removing Transaction
>> > > > > > > > as its own
>> > > > > > > > >> class,
>> > > > > > > > >> > but I'm not sure GraphTraversalSource is where it
>> should
>> > > > > > > > be instead. Do
>> > > > > > > > >> you
>> > > > > > > > >> > have some more details about what you think the
>> > > > > > > > Transaction API should
>> > > > > > > > >> look
>> > > > > > > > >> > like?
>> > > > > > > > >> > 3. Could you expand a bit more on "Transaction scope is
>> > > > > > > > limited to the
>> > > > > > > > >> > current thread only". I always get confused by
>> > > > > > > > TinkerPop's threaded vs
>> > > > > > > > >> > multithreaded transaction. I find it limiting that an
>> > > > > > > > attempt to open
>> > > > > > > > >> > multiple transactions from the same thread ends up
>> > > > > > > > returning the same
>> > > > > > > > >> > Transaction instance. Maybe I didn't quite understand
>> > > > > > > > what you meant by
>> > > > > > > > >> > this.
>> > > > > > > > >> > 4. Sessions won't exist moving forward (4.x+) as they
>> > > > > > > > are tied to remote
>> > > > > > > > >> > Groovy execution. So let's continue this conversation
>> as
>> > > > > > > > simply
>> > > > > > > > >> > transactions and not refer to sessions at all to
>> prevent
>> > > > > > > > confusion,
>> > > > > > > > >> unless
>> > > > > > > > >> > you intend to have these changes in the 3.8.x line?
>> > > > > > > > >> >
>> > > > > > > > >> > Regards,
>> > > > > > > > >> > Ken
>> > > > > > > > >> >
>> > > > > > > > >> >
>> > > > > > > > >> > On Tue, Dec 2, 2025 at 3:33 AM Andrii Lomakin via dev <
>> > > > > > > > >> > dev@tinkerpop.apache.org> wrote:
>> > > > > > > > >> >
>> > > > > > > > >> > > Small clarification.
>> > > > > > > > >> > > Implementation notes are related to changes in the
>> > > > > > > > remote protocol.
>> > > > > > > > >> > >
>> > > > > > > > >> > > On Tue, Dec 2, 2025 at 8:55 AM Andrii Lomakin <
>> > > > > > > > >> > > andrey.lomakin@jetbrains.com>
>> > > > > > > > >> > > wrote:
>> > > > > > > > >> > >
>> > > > > > > > >> > > > To clarify: in this model, 'session' and 'tx' are
>> > > > > > > > essentially the
>> > > > > > > > >> same
>> > > > > > > > >> > > > concepts.
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > I also think it is essential to have a common
>> > > > > > > > denominator that
>> > > > > > > > >> makes TP
>> > > > > > > > >> > > > applications portable across implementations.
>> > > > > > > > >> > > > That is why I also propose to restrict transaction
>> > > > > > > > scope to the
>> > > > > > > > >> thread
>> > > > > > > > >> > > > scope. It will be a common denominator that will
>> not
>> > > > > > > > harm user
>> > > > > > > > >> > experience
>> > > > > > > > >> > > > but will ensure uniform and expected API behavior.
>> > > > > > > > >> > > > There should not be both `begin` and `open` methods
>> > > > > > > > because the
>> > > > > > > > >> > contract
>> > > > > > > > >> > > > of the beginning method is vague and unpredictable.
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > On Tue, Dec 2, 2025 at 8:18 AM Andrii Lomakin <
>> > > > > > > > >> > > > andrey.lomakin@jetbrains.com> wrote:
>> > > > > > > > >> > > >
>> > > > > > > > >> > > >> Good day,
>> > > > > > > > >> > > >>
>> > > > > > > > >> > > >> I have a proposal to enhance the transaction
>> > > > > > > > handling logic in TP,
>> > > > > > > > >> > > >> specifically regarding nested transaction calls,
>> > > > > > > > which are common
>> > > > > > > > >> in
>> > > > > > > > >> > EE
>> > > > > > > > >> > > >> development, particularly when utilizing OGM.
>> > > > > > > > >> > > >>
>> > > > > > > > >> > > >> Currently, calling open multiple times causes an
>> > > > > > > > exception. This
>> > > > > > > > >> > > behavior
>> > > > > > > > >> > > >> is restrictive, and the term "open" can be
>> > > > > > > > confusing.
>> > > > > > > > >> > > >>
>> > > > > > > > >> > > >> I propose a design that improves the developer
>> > > > > > > > experience by using
>> > > > > > > > >> an
>> > > > > > > > >> > > >> internal counter for transaction calls:
>> > > > > > > > >> > > >>
>> > > > > > > > >> > > >> Proposed transaction counter logic
>> > > > > > > > >> > > >> 1. begin/commit counter:
>> > > > > > > > >> > > >> - begin would increment an internal counter.
>> > > > > > > > >> > > >> - commit would decrement the internal counter,
>> > > > > > > > causing the actual
>> > > > > > > > >> > > >> transaction commit only when the counter reaches
>> > > > > > > > zero.
>> > > > > > > > >> > > >> 2. rollback behavior:
>> > > > > > > > >> > > >> - rollback would immediately force a transaction
>> > > > > > > > rollback and
>> > > > > > > > >> > > decrements
>> > > > > > > > >> > > >> the internal counter
>> > > > > > > > >> > > >> - the subsequent calls to rollback on the same
>> > > > > > > > transaction object
>> > > > > > > > >> > > should
>> > > > > > > > >> > > >> be allowed until the stack trace reaches the
>> > > > > > > > initial begin method
>> > > > > > > > >> > call,
>> > > > > > > > >> > > >> after which further rollback calls would throw an
>> > > > > > > > exception.
>> > > > > > > > >> > > >> 3. Error handling:
>> > > > > > > > >> > > >> - A commit without a matching begin (e.g., counter
>> > > > > > > > is already zero)
>> > > > > > > > >> > > >> should throw an exception.
>> > > > > > > > >> > > >> - A rollback without a matching begin should throw
>> > > > > > > > an exception
>> > > > > > > > >> > > >>
>> > > > > > > > >> > > >> I would also consider hiding the Transaction
>> object
>> > > > > > > > altogether and
>> > > > > > > > >> > > >> delegating the logic of transaction management to
>> > > > > > > > >> GraphTraversalSource
>> > > > > > > > >> > > it
>> > > > > > > > >> > > >> will simplify the API a lot, both for users and
>> > > > > > > > vendors.
>> > > > > > > > >> > > >>
>> > > > > > > > >> > > >> I also propose to restrict transaction scope to
>> the
>> > > > > > > > thread scope,
>> > > > > > > > >> it
>> > > > > > > > >> > > will
>> > > > > > > > >> > > >> be a common denominator that will not harm user
>> > > > > > > > experience but will
>> > > > > > > > >> > > ensure
>> > > > > > > > >> > > >> uniform and expected API behavior across all
>> > > > > > > > implementations.
>> > > > > > > > >> > > >>
>> > > > > > > > >> > > >> Proposed API changes
>> > > > > > > > >> > > >> - I propose deprecating open and using begin with
>> > > > > > > > the semantics
>> > > > > > > > >> > > >> described above to avoid confusion.
>> > > > > > > > >> > > >> - Transaction object is deprecated, and TX control
>> > > > > > > > functionality is
>> > > > > > > > >> > > >> delegated to the GraphTraversal/Graph instances.
>> > > > > > > > >> > > >> - tx() method is deprecated.
>> > > > > > > > >> > > >> - Transaction scope is limited to the current
>> > > > > > > > thread only.
>> > > > > > > > >> > > >>
>> > > > > > > > >> > > >> Behavior for non-direct calls
>> > > > > > > > >> > > >> When begin/commit/rollback methods are not called
>> > > > > > > > directly, the
>> > > > > > > > >> > > >> transaction should be automatically committed by a
>> > > > > > > > terminal
>> > > > > > > > >> operation
>> > > > > > > > >> > > >> (e.g., when hasNext returns false) in both remote
>> > > > > > > > and embedded
>> > > > > > > > >> modes.
>> > > > > > > > >> > > >>
>> > > > > > > > >> > > >> Implementation notes:
>> > > > > > > > >> > > >> 1. The begin command should be added. Relying on
>> > > > > > > > the implicit
>> > > > > > > > >> start of
>> > > > > > > > >> > > TX
>> > > > > > > > >> > > >> by traversal can lead to non-controlled side
>> > > > > > > > effects.
>> > > > > > > > >> > > >> 2. For the sake of optimization of the remote
>> > > > > > > > protocol, while
>> > > > > > > > >> multiple
>> > > > > > > > >> > > >> calls of begin/commit/rollback are allowed in the
>> > > > > > > > remote protocol,
>> > > > > > > > >> > > >> practically, it will mean that we will track the
>> > > > > > > > counter locally on
>> > > > > > > > >> > the
>> > > > > > > > >> > > >> client and will send begin/commit/rollback only
>> > > > > > > > once for the TX
>> > > > > > > > >> > > lifecycle.
>> > > > > > > > >> > > >>
>> > > > > > > > >> > > >> This model should be uniform across all deployment
>> > > > > > > > types.
>> > > > > > > > >> > > >> What are your thoughts on this approach?
>> > > > > > > > >> > > >>
>> > > > > > > > >> > > >> -----
>> > > > > > > > >> > > >> Andrii Lomakin
>> > > > > > > > >> > > >> YouTrackDB development lead
>> > > > > > > > >> > > >>
>> > > > > > > > >> > > >
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > --
>> > > > > > > > >> > > > Andrii Lomakin
>> > > > > > > > >> > > > YouTrackDB development lead
>> > > > > > > > >> > > >
>> > > > > > > > >> > >
>> > > > > > > > >> > >
>> > > > > > > > >> > > --
>> > > > > > > > >> > > Andrii Lomakin
>> > > > > > > > >> > > YouTrackDB development lead
>> > > > > > > > >> > >
>> > > > > > > > >> >
>> > > > > > > > >>
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > --
>> > > > > > > > > Andrii Lomakin
>> > > > > > > > > YouTrackDB development lead
>> > > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > >
>> >
>>
>>
>> --
>> Andrii Lomakin
>> YouTrackDB development lead
>>
>
--
Andrii Lomakin
YouTrackDB development lead
Re: [DISCUSS] Handling of transactions - proposal
Posted by Ken Hu <ke...@gmail.com>.
I agree that record consistency is important, but I feel that where this
gets solved depends on how users integrate graph databases into their
applications. There are probably frameworks and OGM/ORMs that can do this
detaching/unloading that you are mentioning as a solution. If a user
directly integrates the database as an external service using a driver,
then having it as part of the API could be useful. I need to think about
this more.
To your second point about platform-independence, I think this has always
been a goal of TinkerPop that has never quite been realized. Even today,
there are some examples of Gremlin that return results that are
provider-dependent and so there are specific cases where it isn't truly
portable. I agree that a difference in read consistency amongst
vendors/providers means those using transactions can't move between
different databases unless they find another database with the exact same
transaction behavior. But I'm not entirely sure we can get all providers to
match the same behavior in this case.
On Tue, Jan 20, 2026 at 11:26 PM Andrii Lomakin via dev <
dev@tinkerpop.apache.org> wrote:
> Hi Ken.
> Thank you for sharing this with us.
>
> After some thought, I agree that we need explicit TX management methods.
> From my perspective, what I wrote is only the tip of the iceberg, as such
> questions as consistency of the queries and validity of the data contained
> in vertices directly bound to the concept of TX lifecycle.
>
> Example (it can be applied to any database, not just a graph database, so I
> will use the general record concept).
>
> I have created record A { id: 1, value: "val1"} , committed it, and passed
> it to another thread.
> After that, another method updated the same record B {id: 1, value :
> "val2"} and committed it.
>
> A and B in this context object identity, not DB record identity.
>
> But I have the record A in another thread with potentially stale data.
> What then should another thread read when it accesses `A.value`?
>
> I doubt it should read stale data. In our database, records are
> automatically switched to the unloaded state after commit and can be read
> in a new thread only when a new transaction is started.
> The question of record consistency, from my point of view, is an important
> one and cannot be overlooked, as it influences how users interact with the
> TP API and to what degree it is really platform-independent.
>
>
> On Wed, Jan 21, 2026 at 7:58 AM Ken Hu <ke...@gmail.com> wrote:
>
> > Here are some of my current thoughts on the proposed changes and
> > discussion.
> >
> > -deprecate open() and replace with begin()
> > -I agree with this as having both is confusing and unncessary
> >
> > -Transaction class is deprecated, and TX control functionality is
> delegated
> > to the GraphTraversal/Graph instances
> > -I mostly agree with this as well, but I think it's probably better
> if
> > it was delegated to GraphTraversalSource.
> >
> > -tx() method is deprecated.
> > -I think this makes sense as I always found how tx() works with
> remote
> > to be unintuitive.
> >
> > -Transaction scope is limited to the current thread only
> > -I had some questions about this earlier, but having thought about it
> > after seeing Andrii's architectural diagram, it's probably fine since
> most
> > developers (in all languages TinkerPop supports) will be used to this
> being
> > how it works as long as we have something that represents what a
> connection
> > is in other databases.
> >
> > -I believe Non-TX mode is not appropriate for handling graph data
> > structures.
> > -Further, I'd say that transactions should be optional. Many persistent
> > db/graph systems are eventually consistent and do not support
> transactions,
> > so they should not be burdened with concepts foreign to them.
> > -I tend to agree with Pieter here. My reasoning is different though,
> > one of the use cases for graphs is analytics and transactions aren't
> really
> > used in that case.
> >
> > -I think we DO need methods for explicit transaction management, but 90%
> of
> > the cases will be covered by implicit transaction management throughout
> the
> > GraphTraversal life cycle.
> > -What I propose is to use lambda-controlled TXs
> > -My idea is that the transaction api should be unopinionated. That means
> it
> > should follow closely to `UserTransaction`
> > -I think having implicit transactions is fine as long as it's just an
> > option. I'm not sure which should be the default. I'm not sure we can
> make
> > lambda-controlled transactions work though since lambdas wouldn't be
> > compatible with Gremlin scripts. At this point, we probably still want
> > parity between scripts and GraphTraversals.
> >
> > I think we need some more proposals for what the explicit transaction
> > management API could look like because we probably can't move ahead with
> > lambda-controlled TX for now and the current tx() solution is clunky.
> I'll
> > try to propose one in the next couple of days.
> >
> > On Wed, Jan 7, 2026 at 12:24 PM pieter <pi...@riseup.net> wrote:
> >
> > > Hi,
> > >
> > > I had a brief discussion with Andrii in the discord chat regarding the
> > > transaction api.
> > >
> > > My idea is that the transaction api should be unopinionated.
> > > That means it should follow closely to
> > > `javax.transaction.UserTransaction` which in turn means we have a
> > > dedicated class modelling the transaction with `begin()`, `commit()`
> > > and `rollback()`.
> > >
> > > My reasoning is that gremlin is a low level graph language. There are
> > > many ways different developers / architects can go about integrating
> > > gremlin into their stack.
> > >
> > > Some like lambdas, some like IOT containers, some have domain entity
> > > frameworks, some use aspect orienated programming and on an on.
> > >
> > > It is common for IOT containers to manage transaction boundaries and
> > > lifecycles. Similarly it is common to use the web server's interceptors
> > > to manage transaction boundaries. And it is very common to do it all
> > > manually...
> > >
> > > Further, I'd say that transactions should be optional. Many persistent
> > > db/graph systems are eventually consistent and do not support
> > > transactions, so they should not be burdened with concepts foreign to
> > > them.
> > >
> > > Regarding binding the transaction to the thread:
> > > This is the only way I have ever worked so its entirely natural to me.
> > > However I can image that people want to defince their own transaction
> > > context, so perhaps we should not hardcode that part. Long ago Gavin
> > > King from Hibernate fame had the notion of `conversation` context.
> > > Perhaps such things are still around?
> > >
> > > Regards
> > > Pieter
> > >
> > > On Thu, 2025-12-11 at 09:02 +0100, Andrii Lomakin via dev wrote:
> > > > Unfortunately, all images have been removed, so I am sending a link
> > > > to the diagrams instead
> > > > https://miro.com/app/board/uXjVGdd8m44=/?share_link_id=686668490384
> > > > .
> > > >
> > > >
> > > >
> > > > On Thu, Dec 11, 2025 at 8:53 AM Andrii Lomakin
> > > > <an...@jetbrains.com> wrote:
> > > > > Good day.
> > > > >
> > > > > If I followed Ken's line of thought correctly, I would like to
> > > > > expand it to be part of the new proposal for the transaction API.
> > > > > First of all, about the model, I think we came to a consensus that
> > > > > it should be:
> > > > >
> > > > >
> > > > > image.png
> > > > > The point is that we do not have a notion of connection in
> > > > > TinkerPop, but we have a notion of GraphTraversal.
> > > > > So, from my perspective, this diagram can be changed to
> > > > > image.png
> > > > >
> > > > > There is a corner case for Non-TX databases that we need to
> > > > > discuss.
> > > > > I believe Non-TX mode is not appropriate for handling graph data
> > > > > structures. GQL, for example, requires serializable isolation to
> > > > > satisfy the standard requirements.
> > > > > Instead, we should require to have presence of a transaction, which
> > > > > is started implicitly when GraphTraversal is created and committed
> > > > > once it is closed.
> > > > >
> > > > > I think we DO need methods for explicit transaction management, but
> > > > > 90% of the cases will be covered by implicit transaction management
> > > > > throughout the GraphTraversal life cycle.
> > > > > 1. TX is started when GraphTraversal is created.
> > > > > 2. TX is committed when GraphTraversal is closed (which includes
> > > > > terminal operations like toList or iterate).
> > > > > 3. TX is rolled back in case of exceptional execution of
> > > > > GraphTraversal. Users can always roll back a transaction by using
> > > > > the fail() step.
> > > > >
> > > > > We also provide in the YTDB the following methods that I hope will
> > > > > become part of the framework: gSource.computeInTx(g ->{}),
> > > > > gSource.executeInTx(g - {}), gSource.autoExecuteInTx(g - {}: g)
> > > > > (this one executes g.iterate() on the returned GraphTraversal,
> > > > > which we find really handy).
> > > > >
> > > > > Now, regarding explicit TX management.
> > > > > I think that explicit TX management is still needed if you want to
> > > > > execute several queries and mix them with complex logic, but I
> > > > > think that methods that explicitly cause TX begin/start/commit are
> > > > > confusing and provide bad DevX.
> > > > > What I propose is to use lambda-controlled TXs (slightly different
> > > > > from the above). I think that an explicit rollback/commit of TX by
> > > > > the user is similar to the goto operator in old programming
> > > > > languages; it gives finer control over TXs, but overall only adds
> > > > > confusion and can be completely avoided in modern languages.
> > > > >
> > > > > Explicit control on TXs is proposed to be implemented using methods
> > > > > similar to above, with a different signature, namely:
> > > > > gSource.executeInTx(graphTraversalSupplier -> {
> > > > > var g = graphTraversalSupplier.get();
> > > > > var v = g.V().next();
> > > > > g.close()
> > > > >
> > > > > v.property("p", "val");
> > > > > int i = v.value("int");
> > > > >
> > > > > if(i > 0) {
> > > > > g = graphTraversalSupplier.get();
> > > > > g.V().has("i2", i).property("i2", i + 1).iterate();
> > > > > }
> > > > > })
> > > > >
> > > > > In this case, the transaction is detached from GraphTraversal, and
> > > > > its lifecycle is not controlled by GraphTraversal.
> > > > > Instead, TX is committed if an exception is not thrown from the
> > > > > lambda method.
> > > > >
> > > > > To inform the user if GraphTraversal controls TX or not, a simple
> > > > > isTxAttached method is proposed to be introduced
> > > > > gSource.<V>computeInTX(graphTraversalSupplier -> {}) works in
> > > > > similar way but returns value.
> > > > >
> > > > >
> > > > > So the final diagram could look like the following:
> > > > >
> > > > >
> > > > > image.png
> > > > >
> > > > > Please note that in this model, there is no need for an explicit
> > > > > transaction object.
> > > > >
> > > > > On Fri, Dec 5, 2025 at 10:18 AM Andrii Lomakin
> > > > > <an...@jetbrains.com> wrote:
> > > > > > Good day, Ken.
> > > > > >
> > > > > > >This could map to multiple threads on the provider side. It's
> > > > > > probably OK if something like "g.tx()..." returns the same
> > > > > > transaction. We could require users to open multiple
> > > > > > DriverRemoteConnections/Clients to achieve this and that would be
> > > > > > similar to other data connectivity APIs.
> > > > > >
> > > > > > Unfortunately I do not follow, could you elaborate more or give
> > > > > > me example how do you see it ?
> > > > > >
> > > > > > On Thu, Dec 4, 2025 at 7:06 PM Ken Hu <ke...@gmail.com>
> wrote:
> > > > > > > I might be thinking about this too much for just the remote
> > > > > > > case and not considering the embedded case enough. In my mind,
> > > > > > > there just needs to be an easy way for users of the language
> > > > > > > variants (in particular JavaScript because of its single
> > > > > > > threaded nature) to start multiple transactions from a single
> > > > > > > thread. This could map to multiple threads on the provider
> > > > > > > side. It's probably OK if something like "g.tx()..." returns
> > > > > > > the same transaction. We could require users to open multiple
> > > > > > > DriverRemoteConnections/Clients to achieve this and that would
> > > > > > > be similar to other data connectivity APIs.
> > > > > > >
> > > > > > > On Thu, Dec 4, 2025 at 2:37 AM Andrii Lomakin via dev
> > > > > > > <de...@tinkerpop.apache.org> wrote:
> > > > > > > > The point about having multiple concurrent transactions
> > > > > > > > within the same
> > > > > > > > thread warrants a more detailed discussion.
> > > > > > > >
> > > > > > > > To compare this behavior, consider how major SQL databases
> > > > > > > > handle attempts
> > > > > > > > to start a second transaction when one is already active:
> > > > > > > >
> > > > > > > > 1. Error out: The database prevents the operation, signaling
> > > > > > > > that a
> > > > > > > > transaction is already active (e.g., "Transaction already
> > > > > > > > active").
> > > > > > > > 2. Create a nested transaction (savepoint): The database
> > > > > > > > creates a
> > > > > > > > sub-scope of the first transaction, not an independent
> > > > > > > > transaction. If the
> > > > > > > > parent transaction fails, the child scope also fails.
> > > > > > > > 3. Implicitly commit the first: The database automatically
> > > > > > > > commits the
> > > > > > > > first transaction to open the second one (a common behavior
> > > > > > > > in MySQL DDL).
> > > > > > > >
> > > > > > > > As far as I know, only Firebird has historically allowed
> > > > > > > > several truly
> > > > > > > > independent transactions concurrently on the same thread. I
> > > > > > > > suspect support
> > > > > > > > for simultaneous transactions may be even more limited among
> > > > > > > > graph database
> > > > > > > > vendors.
> > > > > > > >
> > > > > > > > On Thu, Dec 4, 2025 at 10:47 AM Andrii Lomakin
> > > > > > > > <an...@jetbrains.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Good day.
> > > > > > > > > As for questions from Ken
> > > > > > > > > >What OGMs are you referring to here? Regular transactions
> > > > > > > > are complicated
> > > > > > > > > enough by itself, I would only want to introduce nested
> > > > > > > > transactions if
> > > > > > > > > there were well maintained OGMs that would have a
> > > > > > > > demonstrable benefit from
> > > > > > > > > this.
> > > > > > > > >
> > > > > > > > > We have such an OGM using TinkerPop that is well
> > > > > > > > maintained, if you mean
> > > > > > > > > by this years of support and thousands of instances running
> > > > > > > > on OS version.
> > > > > > > > > However, I have given it thought and believe that it is a
> > > > > > > > relatively rare
> > > > > > > > > case, and OGM is so complex that it can manage nested
> > > > > > > > transactions itself
> > > > > > > > > if needed.
> > > > > > > > > I think we can support nested transactions on pause for
> > > > > > > > now, till we
> > > > > > > > > receive more feedback from users.
> > > > > > > > >
> > > > > > > > > There are many inconsistencies to discuss in the TX
> > > > > > > > lifecycle, even
> > > > > > > > > without nested transactions.
> > > > > > > > >
> > > > > > > > > >Sessions won't exist moving forward (4.x+) as they are
> > > > > > > > tied to remote
> > > > > > > > > Groovy execution. So let's continue this conversation as
> > > > > > > > simply
> > > > > > > > > transactions and not refer to sessions at all to prevent
> > > > > > > > confusion, unless
> > > > > > > > > you intend to have these changes in the 3.8.x line?
> > > > > > > > > Yeah, we already removed Groovy support in our fork, so it
> > > > > > > > is even now in
> > > > > > > > > our distribution TX, and the session is the same.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > >Could you expand a bit more on "Transaction scope is
> > > > > > > > limited to the
> > > > > > > > > current thread only".
> > > > > > > > > I mean that TX visibility and manipulation are limited to a
> > > > > > > > single thread
> > > > > > > > > where it is created.
> > > > > > > > > If some of the providers support TX visibility between
> > > > > > > > several threads,
> > > > > > > > > that will only add a bonus for them if such a need arises,
> > > > > > > > of course,
> > > > > > > > > without breaking the API.
> > > > > > > > >
> > > > > > > > > > I find it limiting that an attempt to open multiple
> > > > > > > > transactions from
> > > > > > > > > the same thread ends up returning the same Transaction
> > > > > > > > instance
> > > > > > > > > Do you propose to support several transactions in the same
> > > > > > > > thread?
> > > > > > > > > I am afraid not all current vendors can support this model.
> > > > > > > > >
> > > > > > > > > On Wed, Dec 3, 2025 at 1:23 PM Stephen Mallette
> > > > > > > > <sp...@gmail.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> Andrii, I'd like to just quickly clarify Ken's point
> > > > > > > > stating " Sessions
> > > > > > > > >> won't exist moving forward (4.x+) ". He's saying what you
> > > > > > > > said: "
> > > > > > > > >> 'session'
> > > > > > > > >> and 'tx' are essentially the same concepts". We
> > > > > > > > acknowledged that and
> > > > > > > > >> decided to stop using "session" terminology and instead
> > > > > > > > use "transaction"
> > > > > > > > >> terminology only to avoid confusion.
> > > > > > > > >>
> > > > > > > > >> Much of what you wrote is in the spirit of what has been
> > > > > > > > considered for
> > > > > > > > >> 4.x, so that is nice alignment. I do think the goal though
> > > > > > > > is to have a
> > > > > > > > >> consistent transaction API/system that works for both
> > > > > > > > embedded and remote
> > > > > > > > >> cases. It will be interesting to see how that is achieved.
> > > > > > > > >>
> > > > > > > > >> On Tue, Dec 2, 2025 at 3:53 PM Ken Hu <kenhuoss@gmail.com
> >
> > > > > > > > wrote:
> > > > > > > > >>
> > > > > > > > >> > Hi Andrii,
> > > > > > > > >> >
> > > > > > > > >> > Good day to you as well. I have a couple questions about
> > > > > > > > this that I
> > > > > > > > >> would
> > > > > > > > >> > like some more details on.
> > > > > > > > >> >
> > > > > > > > >> > 1. What OGMs are you referring to here? Regular
> > > > > > > > transactions are
> > > > > > > > >> > complicated enough by itself, I would only want to
> > > > > > > > introduce nested
> > > > > > > > >> > transactions if there were well maintained OGMs that
> > > > > > > > would have a
> > > > > > > > >> > demonstrable benefit from this.
> > > > > > > > >> > 2. I like some of your ideas about removing Transaction
> > > > > > > > as its own
> > > > > > > > >> class,
> > > > > > > > >> > but I'm not sure GraphTraversalSource is where it should
> > > > > > > > be instead. Do
> > > > > > > > >> you
> > > > > > > > >> > have some more details about what you think the
> > > > > > > > Transaction API should
> > > > > > > > >> look
> > > > > > > > >> > like?
> > > > > > > > >> > 3. Could you expand a bit more on "Transaction scope is
> > > > > > > > limited to the
> > > > > > > > >> > current thread only". I always get confused by
> > > > > > > > TinkerPop's threaded vs
> > > > > > > > >> > multithreaded transaction. I find it limiting that an
> > > > > > > > attempt to open
> > > > > > > > >> > multiple transactions from the same thread ends up
> > > > > > > > returning the same
> > > > > > > > >> > Transaction instance. Maybe I didn't quite understand
> > > > > > > > what you meant by
> > > > > > > > >> > this.
> > > > > > > > >> > 4. Sessions won't exist moving forward (4.x+) as they
> > > > > > > > are tied to remote
> > > > > > > > >> > Groovy execution. So let's continue this conversation as
> > > > > > > > simply
> > > > > > > > >> > transactions and not refer to sessions at all to prevent
> > > > > > > > confusion,
> > > > > > > > >> unless
> > > > > > > > >> > you intend to have these changes in the 3.8.x line?
> > > > > > > > >> >
> > > > > > > > >> > Regards,
> > > > > > > > >> > Ken
> > > > > > > > >> >
> > > > > > > > >> >
> > > > > > > > >> > On Tue, Dec 2, 2025 at 3:33 AM Andrii Lomakin via dev <
> > > > > > > > >> > dev@tinkerpop.apache.org> wrote:
> > > > > > > > >> >
> > > > > > > > >> > > Small clarification.
> > > > > > > > >> > > Implementation notes are related to changes in the
> > > > > > > > remote protocol.
> > > > > > > > >> > >
> > > > > > > > >> > > On Tue, Dec 2, 2025 at 8:55 AM Andrii Lomakin <
> > > > > > > > >> > > andrey.lomakin@jetbrains.com>
> > > > > > > > >> > > wrote:
> > > > > > > > >> > >
> > > > > > > > >> > > > To clarify: in this model, 'session' and 'tx' are
> > > > > > > > essentially the
> > > > > > > > >> same
> > > > > > > > >> > > > concepts.
> > > > > > > > >> > > >
> > > > > > > > >> > > > I also think it is essential to have a common
> > > > > > > > denominator that
> > > > > > > > >> makes TP
> > > > > > > > >> > > > applications portable across implementations.
> > > > > > > > >> > > > That is why I also propose to restrict transaction
> > > > > > > > scope to the
> > > > > > > > >> thread
> > > > > > > > >> > > > scope. It will be a common denominator that will not
> > > > > > > > harm user
> > > > > > > > >> > experience
> > > > > > > > >> > > > but will ensure uniform and expected API behavior.
> > > > > > > > >> > > > There should not be both `begin` and `open` methods
> > > > > > > > because the
> > > > > > > > >> > contract
> > > > > > > > >> > > > of the beginning method is vague and unpredictable.
> > > > > > > > >> > > >
> > > > > > > > >> > > > On Tue, Dec 2, 2025 at 8:18 AM Andrii Lomakin <
> > > > > > > > >> > > > andrey.lomakin@jetbrains.com> wrote:
> > > > > > > > >> > > >
> > > > > > > > >> > > >> Good day,
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> I have a proposal to enhance the transaction
> > > > > > > > handling logic in TP,
> > > > > > > > >> > > >> specifically regarding nested transaction calls,
> > > > > > > > which are common
> > > > > > > > >> in
> > > > > > > > >> > EE
> > > > > > > > >> > > >> development, particularly when utilizing OGM.
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> Currently, calling open multiple times causes an
> > > > > > > > exception. This
> > > > > > > > >> > > behavior
> > > > > > > > >> > > >> is restrictive, and the term "open" can be
> > > > > > > > confusing.
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> I propose a design that improves the developer
> > > > > > > > experience by using
> > > > > > > > >> an
> > > > > > > > >> > > >> internal counter for transaction calls:
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> Proposed transaction counter logic
> > > > > > > > >> > > >> 1. begin/commit counter:
> > > > > > > > >> > > >> - begin would increment an internal counter.
> > > > > > > > >> > > >> - commit would decrement the internal counter,
> > > > > > > > causing the actual
> > > > > > > > >> > > >> transaction commit only when the counter reaches
> > > > > > > > zero.
> > > > > > > > >> > > >> 2. rollback behavior:
> > > > > > > > >> > > >> - rollback would immediately force a transaction
> > > > > > > > rollback and
> > > > > > > > >> > > decrements
> > > > > > > > >> > > >> the internal counter
> > > > > > > > >> > > >> - the subsequent calls to rollback on the same
> > > > > > > > transaction object
> > > > > > > > >> > > should
> > > > > > > > >> > > >> be allowed until the stack trace reaches the
> > > > > > > > initial begin method
> > > > > > > > >> > call,
> > > > > > > > >> > > >> after which further rollback calls would throw an
> > > > > > > > exception.
> > > > > > > > >> > > >> 3. Error handling:
> > > > > > > > >> > > >> - A commit without a matching begin (e.g., counter
> > > > > > > > is already zero)
> > > > > > > > >> > > >> should throw an exception.
> > > > > > > > >> > > >> - A rollback without a matching begin should throw
> > > > > > > > an exception
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> I would also consider hiding the Transaction object
> > > > > > > > altogether and
> > > > > > > > >> > > >> delegating the logic of transaction management to
> > > > > > > > >> GraphTraversalSource
> > > > > > > > >> > > it
> > > > > > > > >> > > >> will simplify the API a lot, both for users and
> > > > > > > > vendors.
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> I also propose to restrict transaction scope to the
> > > > > > > > thread scope,
> > > > > > > > >> it
> > > > > > > > >> > > will
> > > > > > > > >> > > >> be a common denominator that will not harm user
> > > > > > > > experience but will
> > > > > > > > >> > > ensure
> > > > > > > > >> > > >> uniform and expected API behavior across all
> > > > > > > > implementations.
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> Proposed API changes
> > > > > > > > >> > > >> - I propose deprecating open and using begin with
> > > > > > > > the semantics
> > > > > > > > >> > > >> described above to avoid confusion.
> > > > > > > > >> > > >> - Transaction object is deprecated, and TX control
> > > > > > > > functionality is
> > > > > > > > >> > > >> delegated to the GraphTraversal/Graph instances.
> > > > > > > > >> > > >> - tx() method is deprecated.
> > > > > > > > >> > > >> - Transaction scope is limited to the current
> > > > > > > > thread only.
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> Behavior for non-direct calls
> > > > > > > > >> > > >> When begin/commit/rollback methods are not called
> > > > > > > > directly, the
> > > > > > > > >> > > >> transaction should be automatically committed by a
> > > > > > > > terminal
> > > > > > > > >> operation
> > > > > > > > >> > > >> (e.g., when hasNext returns false) in both remote
> > > > > > > > and embedded
> > > > > > > > >> modes.
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> Implementation notes:
> > > > > > > > >> > > >> 1. The begin command should be added. Relying on
> > > > > > > > the implicit
> > > > > > > > >> start of
> > > > > > > > >> > > TX
> > > > > > > > >> > > >> by traversal can lead to non-controlled side
> > > > > > > > effects.
> > > > > > > > >> > > >> 2. For the sake of optimization of the remote
> > > > > > > > protocol, while
> > > > > > > > >> multiple
> > > > > > > > >> > > >> calls of begin/commit/rollback are allowed in the
> > > > > > > > remote protocol,
> > > > > > > > >> > > >> practically, it will mean that we will track the
> > > > > > > > counter locally on
> > > > > > > > >> > the
> > > > > > > > >> > > >> client and will send begin/commit/rollback only
> > > > > > > > once for the TX
> > > > > > > > >> > > lifecycle.
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> This model should be uniform across all deployment
> > > > > > > > types.
> > > > > > > > >> > > >> What are your thoughts on this approach?
> > > > > > > > >> > > >>
> > > > > > > > >> > > >> -----
> > > > > > > > >> > > >> Andrii Lomakin
> > > > > > > > >> > > >> YouTrackDB development lead
> > > > > > > > >> > > >>
> > > > > > > > >> > > >
> > > > > > > > >> > > >
> > > > > > > > >> > > > --
> > > > > > > > >> > > > Andrii Lomakin
> > > > > > > > >> > > > YouTrackDB development lead
> > > > > > > > >> > > >
> > > > > > > > >> > >
> > > > > > > > >> > >
> > > > > > > > >> > > --
> > > > > > > > >> > > Andrii Lomakin
> > > > > > > > >> > > YouTrackDB development lead
> > > > > > > > >> > >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Andrii Lomakin
> > > > > > > > > YouTrackDB development lead
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > >
> >
>
>
> --
> Andrii Lomakin
> YouTrackDB development lead
>
Re: [DISCUSS] Handling of transactions - proposal
Posted by Andrii Lomakin via dev <de...@tinkerpop.apache.org>.
Hi Ken.
Thank you for sharing this with us.
After some thought, I agree that we need explicit TX management methods.
From my perspective, what I wrote is only the tip of the iceberg, as such
questions as consistency of the queries and validity of the data contained
in vertices directly bound to the concept of TX lifecycle.
Example (it can be applied to any database, not just a graph database, so I
will use the general record concept).
I have created record A { id: 1, value: "val1"} , committed it, and passed
it to another thread.
After that, another method updated the same record B {id: 1, value :
"val2"} and committed it.
A and B in this context object identity, not DB record identity.
But I have the record A in another thread with potentially stale data.
What then should another thread read when it accesses `A.value`?
I doubt it should read stale data. In our database, records are
automatically switched to the unloaded state after commit and can be read
in a new thread only when a new transaction is started.
The question of record consistency, from my point of view, is an important
one and cannot be overlooked, as it influences how users interact with the
TP API and to what degree it is really platform-independent.
On Wed, Jan 21, 2026 at 7:58 AM Ken Hu <ke...@gmail.com> wrote:
> Here are some of my current thoughts on the proposed changes and
> discussion.
>
> -deprecate open() and replace with begin()
> -I agree with this as having both is confusing and unncessary
>
> -Transaction class is deprecated, and TX control functionality is delegated
> to the GraphTraversal/Graph instances
> -I mostly agree with this as well, but I think it's probably better if
> it was delegated to GraphTraversalSource.
>
> -tx() method is deprecated.
> -I think this makes sense as I always found how tx() works with remote
> to be unintuitive.
>
> -Transaction scope is limited to the current thread only
> -I had some questions about this earlier, but having thought about it
> after seeing Andrii's architectural diagram, it's probably fine since most
> developers (in all languages TinkerPop supports) will be used to this being
> how it works as long as we have something that represents what a connection
> is in other databases.
>
> -I believe Non-TX mode is not appropriate for handling graph data
> structures.
> -Further, I'd say that transactions should be optional. Many persistent
> db/graph systems are eventually consistent and do not support transactions,
> so they should not be burdened with concepts foreign to them.
> -I tend to agree with Pieter here. My reasoning is different though,
> one of the use cases for graphs is analytics and transactions aren't really
> used in that case.
>
> -I think we DO need methods for explicit transaction management, but 90% of
> the cases will be covered by implicit transaction management throughout the
> GraphTraversal life cycle.
> -What I propose is to use lambda-controlled TXs
> -My idea is that the transaction api should be unopinionated. That means it
> should follow closely to `UserTransaction`
> -I think having implicit transactions is fine as long as it's just an
> option. I'm not sure which should be the default. I'm not sure we can make
> lambda-controlled transactions work though since lambdas wouldn't be
> compatible with Gremlin scripts. At this point, we probably still want
> parity between scripts and GraphTraversals.
>
> I think we need some more proposals for what the explicit transaction
> management API could look like because we probably can't move ahead with
> lambda-controlled TX for now and the current tx() solution is clunky. I'll
> try to propose one in the next couple of days.
>
> On Wed, Jan 7, 2026 at 12:24 PM pieter <pi...@riseup.net> wrote:
>
> > Hi,
> >
> > I had a brief discussion with Andrii in the discord chat regarding the
> > transaction api.
> >
> > My idea is that the transaction api should be unopinionated.
> > That means it should follow closely to
> > `javax.transaction.UserTransaction` which in turn means we have a
> > dedicated class modelling the transaction with `begin()`, `commit()`
> > and `rollback()`.
> >
> > My reasoning is that gremlin is a low level graph language. There are
> > many ways different developers / architects can go about integrating
> > gremlin into their stack.
> >
> > Some like lambdas, some like IOT containers, some have domain entity
> > frameworks, some use aspect orienated programming and on an on.
> >
> > It is common for IOT containers to manage transaction boundaries and
> > lifecycles. Similarly it is common to use the web server's interceptors
> > to manage transaction boundaries. And it is very common to do it all
> > manually...
> >
> > Further, I'd say that transactions should be optional. Many persistent
> > db/graph systems are eventually consistent and do not support
> > transactions, so they should not be burdened with concepts foreign to
> > them.
> >
> > Regarding binding the transaction to the thread:
> > This is the only way I have ever worked so its entirely natural to me.
> > However I can image that people want to defince their own transaction
> > context, so perhaps we should not hardcode that part. Long ago Gavin
> > King from Hibernate fame had the notion of `conversation` context.
> > Perhaps such things are still around?
> >
> > Regards
> > Pieter
> >
> > On Thu, 2025-12-11 at 09:02 +0100, Andrii Lomakin via dev wrote:
> > > Unfortunately, all images have been removed, so I am sending a link
> > > to the diagrams instead
> > > https://miro.com/app/board/uXjVGdd8m44=/?share_link_id=686668490384
> > > .
> > >
> > >
> > >
> > > On Thu, Dec 11, 2025 at 8:53 AM Andrii Lomakin
> > > <an...@jetbrains.com> wrote:
> > > > Good day.
> > > >
> > > > If I followed Ken's line of thought correctly, I would like to
> > > > expand it to be part of the new proposal for the transaction API.
> > > > First of all, about the model, I think we came to a consensus that
> > > > it should be:
> > > >
> > > >
> > > > image.png
> > > > The point is that we do not have a notion of connection in
> > > > TinkerPop, but we have a notion of GraphTraversal.
> > > > So, from my perspective, this diagram can be changed to
> > > > image.png
> > > >
> > > > There is a corner case for Non-TX databases that we need to
> > > > discuss.
> > > > I believe Non-TX mode is not appropriate for handling graph data
> > > > structures. GQL, for example, requires serializable isolation to
> > > > satisfy the standard requirements.
> > > > Instead, we should require to have presence of a transaction, which
> > > > is started implicitly when GraphTraversal is created and committed
> > > > once it is closed.
> > > >
> > > > I think we DO need methods for explicit transaction management, but
> > > > 90% of the cases will be covered by implicit transaction management
> > > > throughout the GraphTraversal life cycle.
> > > > 1. TX is started when GraphTraversal is created.
> > > > 2. TX is committed when GraphTraversal is closed (which includes
> > > > terminal operations like toList or iterate).
> > > > 3. TX is rolled back in case of exceptional execution of
> > > > GraphTraversal. Users can always roll back a transaction by using
> > > > the fail() step.
> > > >
> > > > We also provide in the YTDB the following methods that I hope will
> > > > become part of the framework: gSource.computeInTx(g ->{}),
> > > > gSource.executeInTx(g - {}), gSource.autoExecuteInTx(g - {}: g)
> > > > (this one executes g.iterate() on the returned GraphTraversal,
> > > > which we find really handy).
> > > >
> > > > Now, regarding explicit TX management.
> > > > I think that explicit TX management is still needed if you want to
> > > > execute several queries and mix them with complex logic, but I
> > > > think that methods that explicitly cause TX begin/start/commit are
> > > > confusing and provide bad DevX.
> > > > What I propose is to use lambda-controlled TXs (slightly different
> > > > from the above). I think that an explicit rollback/commit of TX by
> > > > the user is similar to the goto operator in old programming
> > > > languages; it gives finer control over TXs, but overall only adds
> > > > confusion and can be completely avoided in modern languages.
> > > >
> > > > Explicit control on TXs is proposed to be implemented using methods
> > > > similar to above, with a different signature, namely:
> > > > gSource.executeInTx(graphTraversalSupplier -> {
> > > > var g = graphTraversalSupplier.get();
> > > > var v = g.V().next();
> > > > g.close()
> > > >
> > > > v.property("p", "val");
> > > > int i = v.value("int");
> > > >
> > > > if(i > 0) {
> > > > g = graphTraversalSupplier.get();
> > > > g.V().has("i2", i).property("i2", i + 1).iterate();
> > > > }
> > > > })
> > > >
> > > > In this case, the transaction is detached from GraphTraversal, and
> > > > its lifecycle is not controlled by GraphTraversal.
> > > > Instead, TX is committed if an exception is not thrown from the
> > > > lambda method.
> > > >
> > > > To inform the user if GraphTraversal controls TX or not, a simple
> > > > isTxAttached method is proposed to be introduced
> > > > gSource.<V>computeInTX(graphTraversalSupplier -> {}) works in
> > > > similar way but returns value.
> > > >
> > > >
> > > > So the final diagram could look like the following:
> > > >
> > > >
> > > > image.png
> > > >
> > > > Please note that in this model, there is no need for an explicit
> > > > transaction object.
> > > >
> > > > On Fri, Dec 5, 2025 at 10:18 AM Andrii Lomakin
> > > > <an...@jetbrains.com> wrote:
> > > > > Good day, Ken.
> > > > >
> > > > > >This could map to multiple threads on the provider side. It's
> > > > > probably OK if something like "g.tx()..." returns the same
> > > > > transaction. We could require users to open multiple
> > > > > DriverRemoteConnections/Clients to achieve this and that would be
> > > > > similar to other data connectivity APIs.
> > > > >
> > > > > Unfortunately I do not follow, could you elaborate more or give
> > > > > me example how do you see it ?
> > > > >
> > > > > On Thu, Dec 4, 2025 at 7:06 PM Ken Hu <ke...@gmail.com> wrote:
> > > > > > I might be thinking about this too much for just the remote
> > > > > > case and not considering the embedded case enough. In my mind,
> > > > > > there just needs to be an easy way for users of the language
> > > > > > variants (in particular JavaScript because of its single
> > > > > > threaded nature) to start multiple transactions from a single
> > > > > > thread. This could map to multiple threads on the provider
> > > > > > side. It's probably OK if something like "g.tx()..." returns
> > > > > > the same transaction. We could require users to open multiple
> > > > > > DriverRemoteConnections/Clients to achieve this and that would
> > > > > > be similar to other data connectivity APIs.
> > > > > >
> > > > > > On Thu, Dec 4, 2025 at 2:37 AM Andrii Lomakin via dev
> > > > > > <de...@tinkerpop.apache.org> wrote:
> > > > > > > The point about having multiple concurrent transactions
> > > > > > > within the same
> > > > > > > thread warrants a more detailed discussion.
> > > > > > >
> > > > > > > To compare this behavior, consider how major SQL databases
> > > > > > > handle attempts
> > > > > > > to start a second transaction when one is already active:
> > > > > > >
> > > > > > > 1. Error out: The database prevents the operation, signaling
> > > > > > > that a
> > > > > > > transaction is already active (e.g., "Transaction already
> > > > > > > active").
> > > > > > > 2. Create a nested transaction (savepoint): The database
> > > > > > > creates a
> > > > > > > sub-scope of the first transaction, not an independent
> > > > > > > transaction. If the
> > > > > > > parent transaction fails, the child scope also fails.
> > > > > > > 3. Implicitly commit the first: The database automatically
> > > > > > > commits the
> > > > > > > first transaction to open the second one (a common behavior
> > > > > > > in MySQL DDL).
> > > > > > >
> > > > > > > As far as I know, only Firebird has historically allowed
> > > > > > > several truly
> > > > > > > independent transactions concurrently on the same thread. I
> > > > > > > suspect support
> > > > > > > for simultaneous transactions may be even more limited among
> > > > > > > graph database
> > > > > > > vendors.
> > > > > > >
> > > > > > > On Thu, Dec 4, 2025 at 10:47 AM Andrii Lomakin
> > > > > > > <an...@jetbrains.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Good day.
> > > > > > > > As for questions from Ken
> > > > > > > > >What OGMs are you referring to here? Regular transactions
> > > > > > > are complicated
> > > > > > > > enough by itself, I would only want to introduce nested
> > > > > > > transactions if
> > > > > > > > there were well maintained OGMs that would have a
> > > > > > > demonstrable benefit from
> > > > > > > > this.
> > > > > > > >
> > > > > > > > We have such an OGM using TinkerPop that is well
> > > > > > > maintained, if you mean
> > > > > > > > by this years of support and thousands of instances running
> > > > > > > on OS version.
> > > > > > > > However, I have given it thought and believe that it is a
> > > > > > > relatively rare
> > > > > > > > case, and OGM is so complex that it can manage nested
> > > > > > > transactions itself
> > > > > > > > if needed.
> > > > > > > > I think we can support nested transactions on pause for
> > > > > > > now, till we
> > > > > > > > receive more feedback from users.
> > > > > > > >
> > > > > > > > There are many inconsistencies to discuss in the TX
> > > > > > > lifecycle, even
> > > > > > > > without nested transactions.
> > > > > > > >
> > > > > > > > >Sessions won't exist moving forward (4.x+) as they are
> > > > > > > tied to remote
> > > > > > > > Groovy execution. So let's continue this conversation as
> > > > > > > simply
> > > > > > > > transactions and not refer to sessions at all to prevent
> > > > > > > confusion, unless
> > > > > > > > you intend to have these changes in the 3.8.x line?
> > > > > > > > Yeah, we already removed Groovy support in our fork, so it
> > > > > > > is even now in
> > > > > > > > our distribution TX, and the session is the same.
> > > > > > > >
> > > > > > > >
> > > > > > > > >Could you expand a bit more on "Transaction scope is
> > > > > > > limited to the
> > > > > > > > current thread only".
> > > > > > > > I mean that TX visibility and manipulation are limited to a
> > > > > > > single thread
> > > > > > > > where it is created.
> > > > > > > > If some of the providers support TX visibility between
> > > > > > > several threads,
> > > > > > > > that will only add a bonus for them if such a need arises,
> > > > > > > of course,
> > > > > > > > without breaking the API.
> > > > > > > >
> > > > > > > > > I find it limiting that an attempt to open multiple
> > > > > > > transactions from
> > > > > > > > the same thread ends up returning the same Transaction
> > > > > > > instance
> > > > > > > > Do you propose to support several transactions in the same
> > > > > > > thread?
> > > > > > > > I am afraid not all current vendors can support this model.
> > > > > > > >
> > > > > > > > On Wed, Dec 3, 2025 at 1:23 PM Stephen Mallette
> > > > > > > <sp...@gmail.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Andrii, I'd like to just quickly clarify Ken's point
> > > > > > > stating " Sessions
> > > > > > > >> won't exist moving forward (4.x+) ". He's saying what you
> > > > > > > said: "
> > > > > > > >> 'session'
> > > > > > > >> and 'tx' are essentially the same concepts". We
> > > > > > > acknowledged that and
> > > > > > > >> decided to stop using "session" terminology and instead
> > > > > > > use "transaction"
> > > > > > > >> terminology only to avoid confusion.
> > > > > > > >>
> > > > > > > >> Much of what you wrote is in the spirit of what has been
> > > > > > > considered for
> > > > > > > >> 4.x, so that is nice alignment. I do think the goal though
> > > > > > > is to have a
> > > > > > > >> consistent transaction API/system that works for both
> > > > > > > embedded and remote
> > > > > > > >> cases. It will be interesting to see how that is achieved.
> > > > > > > >>
> > > > > > > >> On Tue, Dec 2, 2025 at 3:53 PM Ken Hu <ke...@gmail.com>
> > > > > > > wrote:
> > > > > > > >>
> > > > > > > >> > Hi Andrii,
> > > > > > > >> >
> > > > > > > >> > Good day to you as well. I have a couple questions about
> > > > > > > this that I
> > > > > > > >> would
> > > > > > > >> > like some more details on.
> > > > > > > >> >
> > > > > > > >> > 1. What OGMs are you referring to here? Regular
> > > > > > > transactions are
> > > > > > > >> > complicated enough by itself, I would only want to
> > > > > > > introduce nested
> > > > > > > >> > transactions if there were well maintained OGMs that
> > > > > > > would have a
> > > > > > > >> > demonstrable benefit from this.
> > > > > > > >> > 2. I like some of your ideas about removing Transaction
> > > > > > > as its own
> > > > > > > >> class,
> > > > > > > >> > but I'm not sure GraphTraversalSource is where it should
> > > > > > > be instead. Do
> > > > > > > >> you
> > > > > > > >> > have some more details about what you think the
> > > > > > > Transaction API should
> > > > > > > >> look
> > > > > > > >> > like?
> > > > > > > >> > 3. Could you expand a bit more on "Transaction scope is
> > > > > > > limited to the
> > > > > > > >> > current thread only". I always get confused by
> > > > > > > TinkerPop's threaded vs
> > > > > > > >> > multithreaded transaction. I find it limiting that an
> > > > > > > attempt to open
> > > > > > > >> > multiple transactions from the same thread ends up
> > > > > > > returning the same
> > > > > > > >> > Transaction instance. Maybe I didn't quite understand
> > > > > > > what you meant by
> > > > > > > >> > this.
> > > > > > > >> > 4. Sessions won't exist moving forward (4.x+) as they
> > > > > > > are tied to remote
> > > > > > > >> > Groovy execution. So let's continue this conversation as
> > > > > > > simply
> > > > > > > >> > transactions and not refer to sessions at all to prevent
> > > > > > > confusion,
> > > > > > > >> unless
> > > > > > > >> > you intend to have these changes in the 3.8.x line?
> > > > > > > >> >
> > > > > > > >> > Regards,
> > > > > > > >> > Ken
> > > > > > > >> >
> > > > > > > >> >
> > > > > > > >> > On Tue, Dec 2, 2025 at 3:33 AM Andrii Lomakin via dev <
> > > > > > > >> > dev@tinkerpop.apache.org> wrote:
> > > > > > > >> >
> > > > > > > >> > > Small clarification.
> > > > > > > >> > > Implementation notes are related to changes in the
> > > > > > > remote protocol.
> > > > > > > >> > >
> > > > > > > >> > > On Tue, Dec 2, 2025 at 8:55 AM Andrii Lomakin <
> > > > > > > >> > > andrey.lomakin@jetbrains.com>
> > > > > > > >> > > wrote:
> > > > > > > >> > >
> > > > > > > >> > > > To clarify: in this model, 'session' and 'tx' are
> > > > > > > essentially the
> > > > > > > >> same
> > > > > > > >> > > > concepts.
> > > > > > > >> > > >
> > > > > > > >> > > > I also think it is essential to have a common
> > > > > > > denominator that
> > > > > > > >> makes TP
> > > > > > > >> > > > applications portable across implementations.
> > > > > > > >> > > > That is why I also propose to restrict transaction
> > > > > > > scope to the
> > > > > > > >> thread
> > > > > > > >> > > > scope. It will be a common denominator that will not
> > > > > > > harm user
> > > > > > > >> > experience
> > > > > > > >> > > > but will ensure uniform and expected API behavior.
> > > > > > > >> > > > There should not be both `begin` and `open` methods
> > > > > > > because the
> > > > > > > >> > contract
> > > > > > > >> > > > of the beginning method is vague and unpredictable.
> > > > > > > >> > > >
> > > > > > > >> > > > On Tue, Dec 2, 2025 at 8:18 AM Andrii Lomakin <
> > > > > > > >> > > > andrey.lomakin@jetbrains.com> wrote:
> > > > > > > >> > > >
> > > > > > > >> > > >> Good day,
> > > > > > > >> > > >>
> > > > > > > >> > > >> I have a proposal to enhance the transaction
> > > > > > > handling logic in TP,
> > > > > > > >> > > >> specifically regarding nested transaction calls,
> > > > > > > which are common
> > > > > > > >> in
> > > > > > > >> > EE
> > > > > > > >> > > >> development, particularly when utilizing OGM.
> > > > > > > >> > > >>
> > > > > > > >> > > >> Currently, calling open multiple times causes an
> > > > > > > exception. This
> > > > > > > >> > > behavior
> > > > > > > >> > > >> is restrictive, and the term "open" can be
> > > > > > > confusing.
> > > > > > > >> > > >>
> > > > > > > >> > > >> I propose a design that improves the developer
> > > > > > > experience by using
> > > > > > > >> an
> > > > > > > >> > > >> internal counter for transaction calls:
> > > > > > > >> > > >>
> > > > > > > >> > > >> Proposed transaction counter logic
> > > > > > > >> > > >> 1. begin/commit counter:
> > > > > > > >> > > >> - begin would increment an internal counter.
> > > > > > > >> > > >> - commit would decrement the internal counter,
> > > > > > > causing the actual
> > > > > > > >> > > >> transaction commit only when the counter reaches
> > > > > > > zero.
> > > > > > > >> > > >> 2. rollback behavior:
> > > > > > > >> > > >> - rollback would immediately force a transaction
> > > > > > > rollback and
> > > > > > > >> > > decrements
> > > > > > > >> > > >> the internal counter
> > > > > > > >> > > >> - the subsequent calls to rollback on the same
> > > > > > > transaction object
> > > > > > > >> > > should
> > > > > > > >> > > >> be allowed until the stack trace reaches the
> > > > > > > initial begin method
> > > > > > > >> > call,
> > > > > > > >> > > >> after which further rollback calls would throw an
> > > > > > > exception.
> > > > > > > >> > > >> 3. Error handling:
> > > > > > > >> > > >> - A commit without a matching begin (e.g., counter
> > > > > > > is already zero)
> > > > > > > >> > > >> should throw an exception.
> > > > > > > >> > > >> - A rollback without a matching begin should throw
> > > > > > > an exception
> > > > > > > >> > > >>
> > > > > > > >> > > >> I would also consider hiding the Transaction object
> > > > > > > altogether and
> > > > > > > >> > > >> delegating the logic of transaction management to
> > > > > > > >> GraphTraversalSource
> > > > > > > >> > > it
> > > > > > > >> > > >> will simplify the API a lot, both for users and
> > > > > > > vendors.
> > > > > > > >> > > >>
> > > > > > > >> > > >> I also propose to restrict transaction scope to the
> > > > > > > thread scope,
> > > > > > > >> it
> > > > > > > >> > > will
> > > > > > > >> > > >> be a common denominator that will not harm user
> > > > > > > experience but will
> > > > > > > >> > > ensure
> > > > > > > >> > > >> uniform and expected API behavior across all
> > > > > > > implementations.
> > > > > > > >> > > >>
> > > > > > > >> > > >> Proposed API changes
> > > > > > > >> > > >> - I propose deprecating open and using begin with
> > > > > > > the semantics
> > > > > > > >> > > >> described above to avoid confusion.
> > > > > > > >> > > >> - Transaction object is deprecated, and TX control
> > > > > > > functionality is
> > > > > > > >> > > >> delegated to the GraphTraversal/Graph instances.
> > > > > > > >> > > >> - tx() method is deprecated.
> > > > > > > >> > > >> - Transaction scope is limited to the current
> > > > > > > thread only.
> > > > > > > >> > > >>
> > > > > > > >> > > >> Behavior for non-direct calls
> > > > > > > >> > > >> When begin/commit/rollback methods are not called
> > > > > > > directly, the
> > > > > > > >> > > >> transaction should be automatically committed by a
> > > > > > > terminal
> > > > > > > >> operation
> > > > > > > >> > > >> (e.g., when hasNext returns false) in both remote
> > > > > > > and embedded
> > > > > > > >> modes.
> > > > > > > >> > > >>
> > > > > > > >> > > >> Implementation notes:
> > > > > > > >> > > >> 1. The begin command should be added. Relying on
> > > > > > > the implicit
> > > > > > > >> start of
> > > > > > > >> > > TX
> > > > > > > >> > > >> by traversal can lead to non-controlled side
> > > > > > > effects.
> > > > > > > >> > > >> 2. For the sake of optimization of the remote
> > > > > > > protocol, while
> > > > > > > >> multiple
> > > > > > > >> > > >> calls of begin/commit/rollback are allowed in the
> > > > > > > remote protocol,
> > > > > > > >> > > >> practically, it will mean that we will track the
> > > > > > > counter locally on
> > > > > > > >> > the
> > > > > > > >> > > >> client and will send begin/commit/rollback only
> > > > > > > once for the TX
> > > > > > > >> > > lifecycle.
> > > > > > > >> > > >>
> > > > > > > >> > > >> This model should be uniform across all deployment
> > > > > > > types.
> > > > > > > >> > > >> What are your thoughts on this approach?
> > > > > > > >> > > >>
> > > > > > > >> > > >> -----
> > > > > > > >> > > >> Andrii Lomakin
> > > > > > > >> > > >> YouTrackDB development lead
> > > > > > > >> > > >>
> > > > > > > >> > > >
> > > > > > > >> > > >
> > > > > > > >> > > > --
> > > > > > > >> > > > Andrii Lomakin
> > > > > > > >> > > > YouTrackDB development lead
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > > --
> > > > > > > >> > > Andrii Lomakin
> > > > > > > >> > > YouTrackDB development lead
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Andrii Lomakin
> > > > > > > > YouTrackDB development lead
> > > > > > > >
> > > > > > >
> > > > > > >
> >
>
--
Andrii Lomakin
YouTrackDB development lead
Re: [DISCUSS] Handling of transactions - proposal
Posted by Ken Hu <ke...@gmail.com>.
Here are some of my current thoughts on the proposed changes and discussion.
-deprecate open() and replace with begin()
-I agree with this as having both is confusing and unncessary
-Transaction class is deprecated, and TX control functionality is delegated
to the GraphTraversal/Graph instances
-I mostly agree with this as well, but I think it's probably better if
it was delegated to GraphTraversalSource.
-tx() method is deprecated.
-I think this makes sense as I always found how tx() works with remote
to be unintuitive.
-Transaction scope is limited to the current thread only
-I had some questions about this earlier, but having thought about it
after seeing Andrii's architectural diagram, it's probably fine since most
developers (in all languages TinkerPop supports) will be used to this being
how it works as long as we have something that represents what a connection
is in other databases.
-I believe Non-TX mode is not appropriate for handling graph data
structures.
-Further, I'd say that transactions should be optional. Many persistent
db/graph systems are eventually consistent and do not support transactions,
so they should not be burdened with concepts foreign to them.
-I tend to agree with Pieter here. My reasoning is different though,
one of the use cases for graphs is analytics and transactions aren't really
used in that case.
-I think we DO need methods for explicit transaction management, but 90% of
the cases will be covered by implicit transaction management throughout the
GraphTraversal life cycle.
-What I propose is to use lambda-controlled TXs
-My idea is that the transaction api should be unopinionated. That means it
should follow closely to `UserTransaction`
-I think having implicit transactions is fine as long as it's just an
option. I'm not sure which should be the default. I'm not sure we can make
lambda-controlled transactions work though since lambdas wouldn't be
compatible with Gremlin scripts. At this point, we probably still want
parity between scripts and GraphTraversals.
I think we need some more proposals for what the explicit transaction
management API could look like because we probably can't move ahead with
lambda-controlled TX for now and the current tx() solution is clunky. I'll
try to propose one in the next couple of days.
On Wed, Jan 7, 2026 at 12:24 PM pieter <pi...@riseup.net> wrote:
> Hi,
>
> I had a brief discussion with Andrii in the discord chat regarding the
> transaction api.
>
> My idea is that the transaction api should be unopinionated.
> That means it should follow closely to
> `javax.transaction.UserTransaction` which in turn means we have a
> dedicated class modelling the transaction with `begin()`, `commit()`
> and `rollback()`.
>
> My reasoning is that gremlin is a low level graph language. There are
> many ways different developers / architects can go about integrating
> gremlin into their stack.
>
> Some like lambdas, some like IOT containers, some have domain entity
> frameworks, some use aspect orienated programming and on an on.
>
> It is common for IOT containers to manage transaction boundaries and
> lifecycles. Similarly it is common to use the web server's interceptors
> to manage transaction boundaries. And it is very common to do it all
> manually...
>
> Further, I'd say that transactions should be optional. Many persistent
> db/graph systems are eventually consistent and do not support
> transactions, so they should not be burdened with concepts foreign to
> them.
>
> Regarding binding the transaction to the thread:
> This is the only way I have ever worked so its entirely natural to me.
> However I can image that people want to defince their own transaction
> context, so perhaps we should not hardcode that part. Long ago Gavin
> King from Hibernate fame had the notion of `conversation` context.
> Perhaps such things are still around?
>
> Regards
> Pieter
>
> On Thu, 2025-12-11 at 09:02 +0100, Andrii Lomakin via dev wrote:
> > Unfortunately, all images have been removed, so I am sending a link
> > to the diagrams instead
> > https://miro.com/app/board/uXjVGdd8m44=/?share_link_id=686668490384
> > .
> >
> >
> >
> > On Thu, Dec 11, 2025 at 8:53 AM Andrii Lomakin
> > <an...@jetbrains.com> wrote:
> > > Good day.
> > >
> > > If I followed Ken's line of thought correctly, I would like to
> > > expand it to be part of the new proposal for the transaction API.
> > > First of all, about the model, I think we came to a consensus that
> > > it should be:
> > >
> > >
> > > image.png
> > > The point is that we do not have a notion of connection in
> > > TinkerPop, but we have a notion of GraphTraversal.
> > > So, from my perspective, this diagram can be changed to
> > > image.png
> > >
> > > There is a corner case for Non-TX databases that we need to
> > > discuss.
> > > I believe Non-TX mode is not appropriate for handling graph data
> > > structures. GQL, for example, requires serializable isolation to
> > > satisfy the standard requirements.
> > > Instead, we should require to have presence of a transaction, which
> > > is started implicitly when GraphTraversal is created and committed
> > > once it is closed.
> > >
> > > I think we DO need methods for explicit transaction management, but
> > > 90% of the cases will be covered by implicit transaction management
> > > throughout the GraphTraversal life cycle.
> > > 1. TX is started when GraphTraversal is created.
> > > 2. TX is committed when GraphTraversal is closed (which includes
> > > terminal operations like toList or iterate).
> > > 3. TX is rolled back in case of exceptional execution of
> > > GraphTraversal. Users can always roll back a transaction by using
> > > the fail() step.
> > >
> > > We also provide in the YTDB the following methods that I hope will
> > > become part of the framework: gSource.computeInTx(g ->{}),
> > > gSource.executeInTx(g - {}), gSource.autoExecuteInTx(g - {}: g)
> > > (this one executes g.iterate() on the returned GraphTraversal,
> > > which we find really handy).
> > >
> > > Now, regarding explicit TX management.
> > > I think that explicit TX management is still needed if you want to
> > > execute several queries and mix them with complex logic, but I
> > > think that methods that explicitly cause TX begin/start/commit are
> > > confusing and provide bad DevX.
> > > What I propose is to use lambda-controlled TXs (slightly different
> > > from the above). I think that an explicit rollback/commit of TX by
> > > the user is similar to the goto operator in old programming
> > > languages; it gives finer control over TXs, but overall only adds
> > > confusion and can be completely avoided in modern languages.
> > >
> > > Explicit control on TXs is proposed to be implemented using methods
> > > similar to above, with a different signature, namely:
> > > gSource.executeInTx(graphTraversalSupplier -> {
> > > var g = graphTraversalSupplier.get();
> > > var v = g.V().next();
> > > g.close()
> > >
> > > v.property("p", "val");
> > > int i = v.value("int");
> > >
> > > if(i > 0) {
> > > g = graphTraversalSupplier.get();
> > > g.V().has("i2", i).property("i2", i + 1).iterate();
> > > }
> > > })
> > >
> > > In this case, the transaction is detached from GraphTraversal, and
> > > its lifecycle is not controlled by GraphTraversal.
> > > Instead, TX is committed if an exception is not thrown from the
> > > lambda method.
> > >
> > > To inform the user if GraphTraversal controls TX or not, a simple
> > > isTxAttached method is proposed to be introduced
> > > gSource.<V>computeInTX(graphTraversalSupplier -> {}) works in
> > > similar way but returns value.
> > >
> > >
> > > So the final diagram could look like the following:
> > >
> > >
> > > image.png
> > >
> > > Please note that in this model, there is no need for an explicit
> > > transaction object.
> > >
> > > On Fri, Dec 5, 2025 at 10:18 AM Andrii Lomakin
> > > <an...@jetbrains.com> wrote:
> > > > Good day, Ken.
> > > >
> > > > >This could map to multiple threads on the provider side. It's
> > > > probably OK if something like "g.tx()..." returns the same
> > > > transaction. We could require users to open multiple
> > > > DriverRemoteConnections/Clients to achieve this and that would be
> > > > similar to other data connectivity APIs.
> > > >
> > > > Unfortunately I do not follow, could you elaborate more or give
> > > > me example how do you see it ?
> > > >
> > > > On Thu, Dec 4, 2025 at 7:06 PM Ken Hu <ke...@gmail.com> wrote:
> > > > > I might be thinking about this too much for just the remote
> > > > > case and not considering the embedded case enough. In my mind,
> > > > > there just needs to be an easy way for users of the language
> > > > > variants (in particular JavaScript because of its single
> > > > > threaded nature) to start multiple transactions from a single
> > > > > thread. This could map to multiple threads on the provider
> > > > > side. It's probably OK if something like "g.tx()..." returns
> > > > > the same transaction. We could require users to open multiple
> > > > > DriverRemoteConnections/Clients to achieve this and that would
> > > > > be similar to other data connectivity APIs.
> > > > >
> > > > > On Thu, Dec 4, 2025 at 2:37 AM Andrii Lomakin via dev
> > > > > <de...@tinkerpop.apache.org> wrote:
> > > > > > The point about having multiple concurrent transactions
> > > > > > within the same
> > > > > > thread warrants a more detailed discussion.
> > > > > >
> > > > > > To compare this behavior, consider how major SQL databases
> > > > > > handle attempts
> > > > > > to start a second transaction when one is already active:
> > > > > >
> > > > > > 1. Error out: The database prevents the operation, signaling
> > > > > > that a
> > > > > > transaction is already active (e.g., "Transaction already
> > > > > > active").
> > > > > > 2. Create a nested transaction (savepoint): The database
> > > > > > creates a
> > > > > > sub-scope of the first transaction, not an independent
> > > > > > transaction. If the
> > > > > > parent transaction fails, the child scope also fails.
> > > > > > 3. Implicitly commit the first: The database automatically
> > > > > > commits the
> > > > > > first transaction to open the second one (a common behavior
> > > > > > in MySQL DDL).
> > > > > >
> > > > > > As far as I know, only Firebird has historically allowed
> > > > > > several truly
> > > > > > independent transactions concurrently on the same thread. I
> > > > > > suspect support
> > > > > > for simultaneous transactions may be even more limited among
> > > > > > graph database
> > > > > > vendors.
> > > > > >
> > > > > > On Thu, Dec 4, 2025 at 10:47 AM Andrii Lomakin
> > > > > > <an...@jetbrains.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Good day.
> > > > > > > As for questions from Ken
> > > > > > > >What OGMs are you referring to here? Regular transactions
> > > > > > are complicated
> > > > > > > enough by itself, I would only want to introduce nested
> > > > > > transactions if
> > > > > > > there were well maintained OGMs that would have a
> > > > > > demonstrable benefit from
> > > > > > > this.
> > > > > > >
> > > > > > > We have such an OGM using TinkerPop that is well
> > > > > > maintained, if you mean
> > > > > > > by this years of support and thousands of instances running
> > > > > > on OS version.
> > > > > > > However, I have given it thought and believe that it is a
> > > > > > relatively rare
> > > > > > > case, and OGM is so complex that it can manage nested
> > > > > > transactions itself
> > > > > > > if needed.
> > > > > > > I think we can support nested transactions on pause for
> > > > > > now, till we
> > > > > > > receive more feedback from users.
> > > > > > >
> > > > > > > There are many inconsistencies to discuss in the TX
> > > > > > lifecycle, even
> > > > > > > without nested transactions.
> > > > > > >
> > > > > > > >Sessions won't exist moving forward (4.x+) as they are
> > > > > > tied to remote
> > > > > > > Groovy execution. So let's continue this conversation as
> > > > > > simply
> > > > > > > transactions and not refer to sessions at all to prevent
> > > > > > confusion, unless
> > > > > > > you intend to have these changes in the 3.8.x line?
> > > > > > > Yeah, we already removed Groovy support in our fork, so it
> > > > > > is even now in
> > > > > > > our distribution TX, and the session is the same.
> > > > > > >
> > > > > > >
> > > > > > > >Could you expand a bit more on "Transaction scope is
> > > > > > limited to the
> > > > > > > current thread only".
> > > > > > > I mean that TX visibility and manipulation are limited to a
> > > > > > single thread
> > > > > > > where it is created.
> > > > > > > If some of the providers support TX visibility between
> > > > > > several threads,
> > > > > > > that will only add a bonus for them if such a need arises,
> > > > > > of course,
> > > > > > > without breaking the API.
> > > > > > >
> > > > > > > > I find it limiting that an attempt to open multiple
> > > > > > transactions from
> > > > > > > the same thread ends up returning the same Transaction
> > > > > > instance
> > > > > > > Do you propose to support several transactions in the same
> > > > > > thread?
> > > > > > > I am afraid not all current vendors can support this model.
> > > > > > >
> > > > > > > On Wed, Dec 3, 2025 at 1:23 PM Stephen Mallette
> > > > > > <sp...@gmail.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Andrii, I'd like to just quickly clarify Ken's point
> > > > > > stating " Sessions
> > > > > > >> won't exist moving forward (4.x+) ". He's saying what you
> > > > > > said: "
> > > > > > >> 'session'
> > > > > > >> and 'tx' are essentially the same concepts". We
> > > > > > acknowledged that and
> > > > > > >> decided to stop using "session" terminology and instead
> > > > > > use "transaction"
> > > > > > >> terminology only to avoid confusion.
> > > > > > >>
> > > > > > >> Much of what you wrote is in the spirit of what has been
> > > > > > considered for
> > > > > > >> 4.x, so that is nice alignment. I do think the goal though
> > > > > > is to have a
> > > > > > >> consistent transaction API/system that works for both
> > > > > > embedded and remote
> > > > > > >> cases. It will be interesting to see how that is achieved.
> > > > > > >>
> > > > > > >> On Tue, Dec 2, 2025 at 3:53 PM Ken Hu <ke...@gmail.com>
> > > > > > wrote:
> > > > > > >>
> > > > > > >> > Hi Andrii,
> > > > > > >> >
> > > > > > >> > Good day to you as well. I have a couple questions about
> > > > > > this that I
> > > > > > >> would
> > > > > > >> > like some more details on.
> > > > > > >> >
> > > > > > >> > 1. What OGMs are you referring to here? Regular
> > > > > > transactions are
> > > > > > >> > complicated enough by itself, I would only want to
> > > > > > introduce nested
> > > > > > >> > transactions if there were well maintained OGMs that
> > > > > > would have a
> > > > > > >> > demonstrable benefit from this.
> > > > > > >> > 2. I like some of your ideas about removing Transaction
> > > > > > as its own
> > > > > > >> class,
> > > > > > >> > but I'm not sure GraphTraversalSource is where it should
> > > > > > be instead. Do
> > > > > > >> you
> > > > > > >> > have some more details about what you think the
> > > > > > Transaction API should
> > > > > > >> look
> > > > > > >> > like?
> > > > > > >> > 3. Could you expand a bit more on "Transaction scope is
> > > > > > limited to the
> > > > > > >> > current thread only". I always get confused by
> > > > > > TinkerPop's threaded vs
> > > > > > >> > multithreaded transaction. I find it limiting that an
> > > > > > attempt to open
> > > > > > >> > multiple transactions from the same thread ends up
> > > > > > returning the same
> > > > > > >> > Transaction instance. Maybe I didn't quite understand
> > > > > > what you meant by
> > > > > > >> > this.
> > > > > > >> > 4. Sessions won't exist moving forward (4.x+) as they
> > > > > > are tied to remote
> > > > > > >> > Groovy execution. So let's continue this conversation as
> > > > > > simply
> > > > > > >> > transactions and not refer to sessions at all to prevent
> > > > > > confusion,
> > > > > > >> unless
> > > > > > >> > you intend to have these changes in the 3.8.x line?
> > > > > > >> >
> > > > > > >> > Regards,
> > > > > > >> > Ken
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > On Tue, Dec 2, 2025 at 3:33 AM Andrii Lomakin via dev <
> > > > > > >> > dev@tinkerpop.apache.org> wrote:
> > > > > > >> >
> > > > > > >> > > Small clarification.
> > > > > > >> > > Implementation notes are related to changes in the
> > > > > > remote protocol.
> > > > > > >> > >
> > > > > > >> > > On Tue, Dec 2, 2025 at 8:55 AM Andrii Lomakin <
> > > > > > >> > > andrey.lomakin@jetbrains.com>
> > > > > > >> > > wrote:
> > > > > > >> > >
> > > > > > >> > > > To clarify: in this model, 'session' and 'tx' are
> > > > > > essentially the
> > > > > > >> same
> > > > > > >> > > > concepts.
> > > > > > >> > > >
> > > > > > >> > > > I also think it is essential to have a common
> > > > > > denominator that
> > > > > > >> makes TP
> > > > > > >> > > > applications portable across implementations.
> > > > > > >> > > > That is why I also propose to restrict transaction
> > > > > > scope to the
> > > > > > >> thread
> > > > > > >> > > > scope. It will be a common denominator that will not
> > > > > > harm user
> > > > > > >> > experience
> > > > > > >> > > > but will ensure uniform and expected API behavior.
> > > > > > >> > > > There should not be both `begin` and `open` methods
> > > > > > because the
> > > > > > >> > contract
> > > > > > >> > > > of the beginning method is vague and unpredictable.
> > > > > > >> > > >
> > > > > > >> > > > On Tue, Dec 2, 2025 at 8:18 AM Andrii Lomakin <
> > > > > > >> > > > andrey.lomakin@jetbrains.com> wrote:
> > > > > > >> > > >
> > > > > > >> > > >> Good day,
> > > > > > >> > > >>
> > > > > > >> > > >> I have a proposal to enhance the transaction
> > > > > > handling logic in TP,
> > > > > > >> > > >> specifically regarding nested transaction calls,
> > > > > > which are common
> > > > > > >> in
> > > > > > >> > EE
> > > > > > >> > > >> development, particularly when utilizing OGM.
> > > > > > >> > > >>
> > > > > > >> > > >> Currently, calling open multiple times causes an
> > > > > > exception. This
> > > > > > >> > > behavior
> > > > > > >> > > >> is restrictive, and the term "open" can be
> > > > > > confusing.
> > > > > > >> > > >>
> > > > > > >> > > >> I propose a design that improves the developer
> > > > > > experience by using
> > > > > > >> an
> > > > > > >> > > >> internal counter for transaction calls:
> > > > > > >> > > >>
> > > > > > >> > > >> Proposed transaction counter logic
> > > > > > >> > > >> 1. begin/commit counter:
> > > > > > >> > > >> - begin would increment an internal counter.
> > > > > > >> > > >> - commit would decrement the internal counter,
> > > > > > causing the actual
> > > > > > >> > > >> transaction commit only when the counter reaches
> > > > > > zero.
> > > > > > >> > > >> 2. rollback behavior:
> > > > > > >> > > >> - rollback would immediately force a transaction
> > > > > > rollback and
> > > > > > >> > > decrements
> > > > > > >> > > >> the internal counter
> > > > > > >> > > >> - the subsequent calls to rollback on the same
> > > > > > transaction object
> > > > > > >> > > should
> > > > > > >> > > >> be allowed until the stack trace reaches the
> > > > > > initial begin method
> > > > > > >> > call,
> > > > > > >> > > >> after which further rollback calls would throw an
> > > > > > exception.
> > > > > > >> > > >> 3. Error handling:
> > > > > > >> > > >> - A commit without a matching begin (e.g., counter
> > > > > > is already zero)
> > > > > > >> > > >> should throw an exception.
> > > > > > >> > > >> - A rollback without a matching begin should throw
> > > > > > an exception
> > > > > > >> > > >>
> > > > > > >> > > >> I would also consider hiding the Transaction object
> > > > > > altogether and
> > > > > > >> > > >> delegating the logic of transaction management to
> > > > > > >> GraphTraversalSource
> > > > > > >> > > it
> > > > > > >> > > >> will simplify the API a lot, both for users and
> > > > > > vendors.
> > > > > > >> > > >>
> > > > > > >> > > >> I also propose to restrict transaction scope to the
> > > > > > thread scope,
> > > > > > >> it
> > > > > > >> > > will
> > > > > > >> > > >> be a common denominator that will not harm user
> > > > > > experience but will
> > > > > > >> > > ensure
> > > > > > >> > > >> uniform and expected API behavior across all
> > > > > > implementations.
> > > > > > >> > > >>
> > > > > > >> > > >> Proposed API changes
> > > > > > >> > > >> - I propose deprecating open and using begin with
> > > > > > the semantics
> > > > > > >> > > >> described above to avoid confusion.
> > > > > > >> > > >> - Transaction object is deprecated, and TX control
> > > > > > functionality is
> > > > > > >> > > >> delegated to the GraphTraversal/Graph instances.
> > > > > > >> > > >> - tx() method is deprecated.
> > > > > > >> > > >> - Transaction scope is limited to the current
> > > > > > thread only.
> > > > > > >> > > >>
> > > > > > >> > > >> Behavior for non-direct calls
> > > > > > >> > > >> When begin/commit/rollback methods are not called
> > > > > > directly, the
> > > > > > >> > > >> transaction should be automatically committed by a
> > > > > > terminal
> > > > > > >> operation
> > > > > > >> > > >> (e.g., when hasNext returns false) in both remote
> > > > > > and embedded
> > > > > > >> modes.
> > > > > > >> > > >>
> > > > > > >> > > >> Implementation notes:
> > > > > > >> > > >> 1. The begin command should be added. Relying on
> > > > > > the implicit
> > > > > > >> start of
> > > > > > >> > > TX
> > > > > > >> > > >> by traversal can lead to non-controlled side
> > > > > > effects.
> > > > > > >> > > >> 2. For the sake of optimization of the remote
> > > > > > protocol, while
> > > > > > >> multiple
> > > > > > >> > > >> calls of begin/commit/rollback are allowed in the
> > > > > > remote protocol,
> > > > > > >> > > >> practically, it will mean that we will track the
> > > > > > counter locally on
> > > > > > >> > the
> > > > > > >> > > >> client and will send begin/commit/rollback only
> > > > > > once for the TX
> > > > > > >> > > lifecycle.
> > > > > > >> > > >>
> > > > > > >> > > >> This model should be uniform across all deployment
> > > > > > types.
> > > > > > >> > > >> What are your thoughts on this approach?
> > > > > > >> > > >>
> > > > > > >> > > >> -----
> > > > > > >> > > >> Andrii Lomakin
> > > > > > >> > > >> YouTrackDB development lead
> > > > > > >> > > >>
> > > > > > >> > > >
> > > > > > >> > > >
> > > > > > >> > > > --
> > > > > > >> > > > Andrii Lomakin
> > > > > > >> > > > YouTrackDB development lead
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > > --
> > > > > > >> > > Andrii Lomakin
> > > > > > >> > > YouTrackDB development lead
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Andrii Lomakin
> > > > > > > YouTrackDB development lead
> > > > > > >
> > > > > >
> > > > > >
>