You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tinkerpop.apache.org by "xiazcy (via GitHub)" <gi...@apache.org> on 2026/04/30 08:54:36 UTC
[PR] Support Streaming in .NET [tinkerpop]
xiazcy opened a new pull request, #3403:
URL: https://github.com/apache/tinkerpop/pull/3403
Updated .NET driver to support streaming HTTP responses. Changes include:
Non-breaking:
- `Connection.SubmitAsync` uses `ResponseHeadersRead` + background `Task` + `Channel<object>` to stream results incrementally
- `ResponseMessageSerializer` yields results via `IAsyncEnumerable<object>` instead of buffering into `List<object>`
- `StreamExtensions` reads use `ReadExactlyAsync` for EOF safety, reads and writes use `BinaryPrimitives`
- Traversal API (`Next()`, `ToList()`, `HasNext()`, etc.) unchanged, blocks internally via `WaitUnwrap()`
Breaking:
- `ResultSet<T>` rebuilt as `IAsyncEnumerable<T>` (was `IReadOnlyCollection<T>`). `Count` and sync `foreach` removed, use `ToListAsync()` or `await foreach`
- `IMessageSerializer.DeserializeMessageAsync` accepts `Stream` → `IAsyncEnumerable<object>` (was `byte[]` → `Task<ResponseMessage<List<object>>>`). Custom serializers must update
- `ResponseMessage<T>` deleted
- `ResponseException` thrown during result iteration, not during `SubmitAsync`, for direct `ResultSet<T>` consumers
- `ITraversal.Traversers` changed from `IEnumerable<Traverser>` to `IAsyncEnumerable<Traverser>`; `IEnumerator` inheritance removed (`Reset()` gone)
- Minimum runtime: .NET 8.0 (was `netstandard2.0;net6.0`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support Streaming in .NET [tinkerpop]
Posted by "kenhuuu (via GitHub)" <gi...@apache.org>.
kenhuuu commented on code in PR #3403:
URL: https://github.com/apache/tinkerpop/pull/3403#discussion_r3184733313
##########
gremlin-dotnet/src/Gremlin.Net/Driver/ResultSet.cs:
##########
@@ -21,49 +21,134 @@
#endregion
-using System.Collections;
+using System;
using System.Collections.Generic;
+using System.Threading;
+using System.Threading.Channels;
+using System.Threading.Tasks;
namespace Gremlin.Net.Driver
{
/// <summary>
- /// A ResultSet is returned from the submission of a Gremlin script to the server and represents the results
- /// provided by the server. ResultSet includes enumerable data and status attributes.
+ /// A streaming ResultSet returned from the submission of a Gremlin request.
+ /// Implements <see cref="IAsyncEnumerable{T}"/>; so consumers can iterate results as they
+ /// arrive from the server via <c>await foreach</c>.
+ /// Single-consumer only — calling GetAsyncEnumerator a second time throws
+ /// InvalidOperationException.
/// </summary>
/// <typeparam name="T">Type of the result elements</typeparam>
- public sealed class ResultSet<T> : IReadOnlyCollection<T>
+ public sealed class ResultSet<T> : IAsyncEnumerable<T>, IAsyncDisposable, IDisposable
{
- private readonly IReadOnlyCollection<T> _data;
+ private readonly ChannelReader<object> _channelReader;
+ private readonly CancellationTokenSource _disposeCts;
+ private readonly Task _backgroundTask;
+ private int _enumerated; // 0 = not yet, 1 = already enumerated
Review Comment:
Nit: not immediately obvious that these are ints not booleans due to the use of InterLocked and restrictions prior to .NET 9.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support Streaming in .NET [tinkerpop]
Posted by "Cole-Greer (via GitHub)" <gi...@apache.org>.
Cole-Greer commented on PR #3403:
URL: https://github.com/apache/tinkerpop/pull/3403#issuecomment-4392702233
VOTE +1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support Streaming in .NET [tinkerpop]
Posted by "xiazcy (via GitHub)" <gi...@apache.org>.
xiazcy merged PR #3403:
URL: https://github.com/apache/tinkerpop/pull/3403
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support Streaming in .NET [tinkerpop]
Posted by "kenhuuu (via GitHub)" <gi...@apache.org>.
kenhuuu commented on code in PR #3403:
URL: https://github.com/apache/tinkerpop/pull/3403#discussion_r3184829480
##########
gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/ResponseMessageSerializer.cs:
##########
@@ -23,46 +23,57 @@
using System.Collections.Generic;
using System.IO;
+using System.Runtime.CompilerServices;
using System.Threading;
-using System.Threading.Tasks;
using Gremlin.Net.Driver.Exceptions;
-using Gremlin.Net.Driver.Messages;
using Gremlin.Net.Process.Traversal;
namespace Gremlin.Net.Structure.IO.GraphBinary4
{
/// <summary>
- /// Deserializes a <see cref="ResponseMessage{T}"/> from the GraphBinary 4.0 streaming format.
+ /// Deserializes a streaming GraphBinary 4.0 response, yielding each result object
+ /// as it is deserialized from the stream. After the Marker, reads the status
+ /// footer and throws <see cref="ResponseException"/> if the status code is not 200.
/// </summary>
public class ResponseMessageSerializer
{
/// <summary>
- /// Reads a response message from the stream in the 4.0 format:
- /// version byte + bulked flag + result data + marker + status footer.
+ /// Reads a streaming GraphBinary 4.0 response, yielding each result object
+ /// as it is deserialized from the stream. After the Marker, reads the status
+ /// footer and throws <see cref="ResponseException"/> if the status code is not 200.
/// </summary>
- /// <param name="stream">The GraphBinary data to parse.</param>
+ /// <param name="stream">The stream to read the GraphBinary response from.</param>
/// <param name="reader">A <see cref="GraphBinaryReader"/> that can be used to read nested values.</param>
/// <param name="cancellationToken">The token to cancel the operation. The default value is None.</param>
- /// <returns>The read response message.</returns>
- public async Task<ResponseMessage<List<object>>> ReadValueAsync(MemoryStream stream,
- GraphBinaryReader reader, CancellationToken cancellationToken = default)
+ /// <returns>An async enumerable of deserialized result objects.</returns>
+ public async IAsyncEnumerable<object> ReadStreamingAsync(Stream stream,
Review Comment:
This class is the ResponseMessageSerializer, but it no longer returns a ResponseMessage. This probably require some restructuring. Is `ResponseMessage` needed at all anymore?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support Streaming in .NET [tinkerpop]
Posted by "kenhuuu (via GitHub)" <gi...@apache.org>.
kenhuuu commented on code in PR #3403:
URL: https://github.com/apache/tinkerpop/pull/3403#discussion_r3184873658
##########
gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/StreamExtensions.cs:
##########
@@ -52,10 +53,16 @@ public static async Task WriteByteAsync(this Stream stream, byte value,
/// <param name="stream">The <see cref="Stream"/> to read from.</param>
/// <param name="cancellationToken">The token to cancel the operation. The default value is None.</param>
/// <returns>The read <see cref="byte"/>.</returns>
- public static async Task<byte> ReadByteAsync(this Stream stream, CancellationToken cancellationToken = default)
+ public static async ValueTask<byte> ReadByteAsync(this Stream stream,
Review Comment:
I mean the change from Task to ValueTask is a valid change but was the line break necessary?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support Streaming in .NET [tinkerpop]
Posted by "kenhuuu (via GitHub)" <gi...@apache.org>.
kenhuuu commented on code in PR #3403:
URL: https://github.com/apache/tinkerpop/pull/3403#discussion_r3184870886
##########
gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/StreamExtensions.cs:
##########
@@ -52,10 +53,16 @@ public static async Task WriteByteAsync(this Stream stream, byte value,
/// <param name="stream">The <see cref="Stream"/> to read from.</param>
/// <param name="cancellationToken">The token to cancel the operation. The default value is None.</param>
/// <returns>The read <see cref="byte"/>.</returns>
- public static async Task<byte> ReadByteAsync(this Stream stream, CancellationToken cancellationToken = default)
+ public static async ValueTask<byte> ReadByteAsync(this Stream stream,
Review Comment:
Nit: These types of changes exist throughout the PR that are just spacing changes, but the make the diff hard to read. Can these be reverted?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support Streaming in .NET [tinkerpop]
Posted by "xiazcy (via GitHub)" <gi...@apache.org>.
xiazcy commented on code in PR #3403:
URL: https://github.com/apache/tinkerpop/pull/3403#discussion_r3185087285
##########
gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/ResponseMessageSerializer.cs:
##########
@@ -23,46 +23,57 @@
using System.Collections.Generic;
using System.IO;
+using System.Runtime.CompilerServices;
using System.Threading;
-using System.Threading.Tasks;
using Gremlin.Net.Driver.Exceptions;
-using Gremlin.Net.Driver.Messages;
using Gremlin.Net.Process.Traversal;
namespace Gremlin.Net.Structure.IO.GraphBinary4
{
/// <summary>
- /// Deserializes a <see cref="ResponseMessage{T}"/> from the GraphBinary 4.0 streaming format.
+ /// Deserializes a streaming GraphBinary 4.0 response, yielding each result object
+ /// as it is deserialized from the stream. After the Marker, reads the status
+ /// footer and throws <see cref="ResponseException"/> if the status code is not 200.
/// </summary>
public class ResponseMessageSerializer
{
/// <summary>
- /// Reads a response message from the stream in the 4.0 format:
- /// version byte + bulked flag + result data + marker + status footer.
+ /// Reads a streaming GraphBinary 4.0 response, yielding each result object
+ /// as it is deserialized from the stream. After the Marker, reads the status
+ /// footer and throws <see cref="ResponseException"/> if the status code is not 200.
/// </summary>
- /// <param name="stream">The GraphBinary data to parse.</param>
+ /// <param name="stream">The stream to read the GraphBinary response from.</param>
/// <param name="reader">A <see cref="GraphBinaryReader"/> that can be used to read nested values.</param>
/// <param name="cancellationToken">The token to cancel the operation. The default value is None.</param>
- /// <returns>The read response message.</returns>
- public async Task<ResponseMessage<List<object>>> ReadValueAsync(MemoryStream stream,
- GraphBinaryReader reader, CancellationToken cancellationToken = default)
+ /// <returns>An async enumerable of deserialized result objects.</returns>
+ public async IAsyncEnumerable<object> ReadStreamingAsync(Stream stream,
Review Comment:
No, `ResponseMessage` was deleted. I'll update naming & stuff for this one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support Streaming in .NET [tinkerpop]
Posted by "xiazcy (via GitHub)" <gi...@apache.org>.
xiazcy commented on code in PR #3403:
URL: https://github.com/apache/tinkerpop/pull/3403#discussion_r3197286668
##########
gremlin-dotnet/src/Gremlin.Net/Structure/IO/GraphBinary4/StreamExtensions.cs:
##########
@@ -52,10 +53,16 @@ public static async Task WriteByteAsync(this Stream stream, byte value,
/// <param name="stream">The <see cref="Stream"/> to read from.</param>
/// <param name="cancellationToken">The token to cancel the operation. The default value is None.</param>
/// <returns>The read <see cref="byte"/>.</returns>
- public static async Task<byte> ReadByteAsync(this Stream stream, CancellationToken cancellationToken = default)
+ public static async ValueTask<byte> ReadByteAsync(this Stream stream,
Review Comment:
It was a styling nit for line length that's auto applied.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support Streaming in .NET [tinkerpop]
Posted by "GumpacG (via GitHub)" <gi...@apache.org>.
GumpacG commented on PR #3403:
URL: https://github.com/apache/tinkerpop/pull/3403#issuecomment-4392135599
VOTE +1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Support Streaming in .NET [tinkerpop]
Posted by "kenhuuu (via GitHub)" <gi...@apache.org>.
kenhuuu commented on PR #3403:
URL: https://github.com/apache/tinkerpop/pull/3403#issuecomment-4392359128
VOTE +1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tinkerpop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org