You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Augusto Vivaldelli <au...@gmail.com> on 2025/09/15 20:53:45 UTC
[ANNOUNCE][3rd-Party] TypeScript Spark Connect client — ts-spark-connector
Hi all,
I’d like to share a third-party project: **ts-spark-connector**, a
TypeScript client for Spark Connect that enables you to build typed logical
plans and interact with a Spark Connect server from Node.js environments.
📎 GitHub: https://github.com/BaldrVivaldelli/ts-spark-connector
📦 npm: https://www.npmjs.com/package/ts-spark-connector
🧾 License: Apache 2.0
⚙️ Compatibility: Spark 3.5+, Node 18/20/22
### Key Features:
- Typed DataFrame API with transformations, joins, aggregates, window
functions, repartitioning, etc.
- Logical plan visualization to **JSON** or **Mermaid** (great for
debugging and CI validation).
- Full **TLS** support for secure Spark Connect communication.
- Structured Streaming: **`readStream(...)` implemented**, with watermark
support.
It’s under active development, and I’d love feedback from anyone using
Spark Connect, especially around interoperability or potential upstream
considerations. If there's interest, I’d be happy to open a PR to list it
under “Third-Party Projects” on the Spark website.
Thanks!
— Augusto Vivaldelli
Re: [ANNOUNCE][3rd-Party] TypeScript Spark Connect client — ts-spark-connector
Posted by Augusto Vivaldelli <au...@gmail.com>.
The PR is the following: https://github.com/apache/spark-website/pull/632
On Mon, 15 Sept 2025 at 17:53, Augusto Vivaldelli <
augusto.a.vivaldelli@gmail.com> wrote:
> Hi all,
>
> I’d like to share a third-party project: **ts-spark-connector**, a
> TypeScript client for Spark Connect that enables you to build typed logical
> plans and interact with a Spark Connect server from Node.js environments.
>
> 📎 GitHub: https://github.com/BaldrVivaldelli/ts-spark-connector
> 📦 npm: https://www.npmjs.com/package/ts-spark-connector
> 🧾 License: Apache 2.0
> ⚙️ Compatibility: Spark 3.5+, Node 18/20/22
>
> ### Key Features:
> - Typed DataFrame API with transformations, joins, aggregates, window
> functions, repartitioning, etc.
> - Logical plan visualization to **JSON** or **Mermaid** (great for
> debugging and CI validation).
> - Full **TLS** support for secure Spark Connect communication.
> - Structured Streaming: **`readStream(...)` implemented**, with watermark
> support.
>
> It’s under active development, and I’d love feedback from anyone using
> Spark Connect, especially around interoperability or potential upstream
> considerations. If there's interest, I’d be happy to open a PR to list it
> under “Third-Party Projects” on the Spark website.
>
> Thanks!
> — Augusto Vivaldelli
>