You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "ankitsultana (via GitHub)" <gi...@apache.org> on 2026/03/22 05:53:53 UTC
[I] [discuss] Native Graph Query Language Support in Pinot [pinot]
ankitsultana opened a new issue, #17935:
URL: https://github.com/apache/pinot/issues/17935
I wanted to check if there's any interest in the community about adding Native Graph Query Language Support to Pinot (openCypher or GQL).
My read of the Graph space is that companies often have a very limited set of use-cases which can benefit from Graphs, and having a separate Database like Neo4J might be quite an overkill.
Users can technically use solutions like Puppy Graph to avoid maintaining another datastore, but it's not open source.
Having Graph Query Language support in Pinot would have the advantage that users can use the same tables for low-latency SQL and Time Series Monitoring/Alerting use-cases. So even if their graph use-cases fizzle out, they can still use SQL or Time Series QL with their datasets.
However, I don't think we'll be able to do super high QPS or super low latencies. My guess is that 100ms or so latencies with QPS in the order of 100 might be doable, and this might be sufficient for most use-cases.
Technical details aside, **are there any companies that use Pinot that are interested in this?**
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [discuss] Native Graph Query Language Support in Pinot [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #17935:
URL: https://github.com/apache/pinot/issues/17935#issuecomment-5841469080
This issue has been closed because it has been inactive for 180 days. If you believe this issue is still relevant, please feel free to reopen it or create a new issue. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [discuss] Native Graph Query Language Support in Pinot [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #17935: [discuss] Native Graph Query Language Support in Pinot
URL: https://github.com/apache/pinot/issues/17935
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [discuss] Native Graph Query Language Support in Pinot [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #17935:
URL: https://github.com/apache/pinot/issues/17935#issuecomment-5418755167
This issue has been automatically marked as stale because it has not had any activity in the last 150 days. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [discuss] Native Graph Query Language Support in Pinot [pinot]
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 commented on issue #17935:
URL: https://github.com/apache/pinot/issues/17935#issuecomment-4108165599
## Design Doc: openCypher Graph Query Frontend for Pinot
### Overview
This design adds an experimental openCypher-subset frontend that **transpiles Cypher queries into SQL JOINs** and executes them via Pinot's existing multi-stage engine (MSE). No native graph runtime operators are introduced.
### Architecture
```
Cypher Query → CypherParser → CypherIR → CypherSemanticValidator → SqlGenerator → SQL
↓
MSE (existing) ← BrokerRequestHandler
↓
Tabular Results
```
**Key decision**: Cypher-to-SQL transpilation (not Cypher-to-RelNode). This keeps the graph layer as a pure preprocessor with zero impact on the existing query engine.
### Module Layout
```
pinot-graph/
├── pinot-graph-spi/ # GraphSchemaConfig, PinotGraphConfiguration
└── pinot-graph-planner/ # CypherParser, CypherIR, SqlGenerator, CypherToSqlTranslator
```
### Graph Schema Model
The graph schema maps vertex labels and edge labels to Pinot tables:
```json
{
"vertexLabels": {
"User": { "tableName": "users", "primaryKey": "id", "properties": {"name": "user_name"} }
},
"edgeLabels": {
"FOLLOWS": {
"tableName": "follows", "sourceVertexLabel": "User", "sourceKey": "follower_id",
"targetVertexLabel": "User", "targetKey": "followee_id"
}
}
}
```
For MVP, the schema is passed inline in the request JSON. Future: stored in ZK with controller CRUD APIs.
### Query Translation Example
**1-hop:**
```cypher
MATCH (a:User {id: 1})-[:FOLLOWS]->(b:User) WHERE b.age > 25 RETURN b.name ORDER BY b.name LIMIT 100
```
→
```sql
SELECT b.user_name FROM follows AS e
JOIN users AS a ON e.follower_id = a.id
JOIN users AS b ON e.followee_id = b.id
WHERE a.id = 1 AND b.age > 25
ORDER BY b.user_name ASC LIMIT 100
```
**2-hop (friends of friends):**
```cypher
MATCH (a:User {id: 1})-[:FOLLOWS]->(b:User)-[:FOLLOWS]->(c:User) RETURN DISTINCT c.id LIMIT 100
```
→
```sql
SELECT DISTINCT c.id FROM follows AS e1
JOIN users AS a ON e1.follower_id = a.id
JOIN users AS b ON e1.followee_id = b.id
JOIN follows AS e2 ON e2.follower_id = b.id
JOIN users AS c ON e2.followee_id = c.id
WHERE a.id = 1 LIMIT 100
```
### Supported Cypher Subset
| Feature | Syntax | SQL Lowering |
|---|---|---|
| 1-hop & 2-hop MATCH | `(a)-[:R]->(b)-[:R]->(c)` | Chained JOINs |
| Inline property filters | `{id: 1}` | WHERE clause |
| WHERE clause | `=, <>, <, >, <=, >=, AND, OR, NOT` | Direct mapping |
| String predicates | `STARTS WITH, ENDS WITH, CONTAINS` | LIKE patterns |
| IN lists | `WHERE a.id IN [1, 2, 3]` | SQL IN |
| RETURN with aliases | `RETURN b.id AS userId` | SELECT ... AS |
| DISTINCT | `RETURN DISTINCT b.id` | SELECT DISTINCT |
| Aggregations | `count, sum, avg, min, max` | SQL aggregates + GROUP BY |
| ORDER BY | `ORDER BY b.name DESC` | SQL ORDER BY |
| SKIP / LIMIT | `SKIP 20 LIMIT 100` | OFFSET / LIMIT |
| Direction | outgoing `->`, incoming `<-`, undirected `-` | Join key swap |
### Entry Point
`POST /query/graph` on the broker, gated behind `pinot.graph.enabled=true` (default: false).
### Distributed Execution
Since generated SQL uses standard JOINs, the MSE handles all distributed execution:
- Hash exchange for join key partitioning across servers
- Multi-stage plan fragmentation
- gRPC mailbox for inter-server data transfer
### Explicit Non-Goals (MVP)
- No native graph operators (BFS, shortest path)
- No variable-length paths `[*1..N]`
- No writes (CREATE, MERGE, DELETE, SET)
- No OPTIONAL MATCH, WITH, UNION, subqueries
- No graph-specific cost model or optimization
- No Bolt protocol (HTTP REST only)
- No ZK-stored graph catalog (inline schema only)
### Test Coverage
233 tests: 211 unit (parser/generator/lowering) + 12 handler + 10 E2E integration (2 servers, distributed data)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [discuss] Native Graph Query Language Support in Pinot [pinot]
Posted by "hpvd (via GitHub)" <gi...@apache.org>.
hpvd commented on issue #17935:
URL: https://github.com/apache/pinot/issues/17935#issuecomment-4148750131
would like to point to this somehow related graph issue, opened some time ago:
https://github.com/apache/pinot/issues/14568
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] [discuss] Native Graph Query Language Support in Pinot [pinot]
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 commented on issue #17935:
URL: https://github.com/apache/pinot/issues/17935#issuecomment-4105766837
I feel it could be a good feature addition if there are active contributors to maintain and use this feature.
Would love to understand your use case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org