You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@graphar.apache.org by "yangxk1 (via GitHub)" <gi...@apache.org> on 2025/11/05 03:54:29 UTC
[PR] feat(python): refactor python module(from cli module) [incubator-graphar]
yangxk1 opened a new pull request, #799:
URL: https://github.com/apache/incubator-graphar/pull/799
<!--
Thanks for contributing to GraphAr.
If this is your first pull request you can find detailed information on [CONTRIBUTING.md](https://github.com/apache/graphar/blob/main/CONTRIBUTING.md)
The Apache GraphAr (incubating) community has restrictions on the naming of pr title. You can find instructions in
[CONTRIBUTING.md](https://github.com/apache/graphar/blob/main/CONTRIBUTING.md#title) too.
-->
### Reason for this PR
<!--
Why are you proposing this change? If this is already tracked in an issue, please link to the issue here.
Explaining clearly why this change is beneficial is important for the reviewers to understand the context.
-->
This PR refactors and enhances the Python module of Apache GraphAr (incubating), transforming it from a CLI-only tool into a full-featured Python SDK with both programmatic APIs and a command-line interface.
The original Python implementation was tightly coupled as a CLI utility (graphar_cli). To support broader use cases—such as integration into data pipelines, Jupyter notebooks, and Python-based graph applications—this PR restructures the module into a proper SDK while preserving and improving CLI functionality.
### What changes are included in this PR?
<!--
There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR.
-->
close #798
**Module restructuring:** The `cli/` directory is renamed to `python/`, and internal code is reorganized into logical packages:
- `graphar/`: Core Python API (e.g., GraphInfo, VertexInfo, EdgeInfo)
- `client/`: CLI implementation (graphar command)
- `graphar/importer/`: Data import utilities
**New public APIs:** Exposes C++ GraphAr types (e.g., `Type`, `FileType`, `AdjListType`) and metadata classes via pybind11 bindings, enabling programmatic access to GraphAr metadata.
**Improved CLI:**
- Adds `--version` flag
- Updates CI workflows to test both CLI and SDK
**Testing:**
- Adds comprehensive unit tests for `GraphInfo`, `vertex`/`edge` metadata, and `propertyGroups`
- Includes CLI integration tests
- Enables pytest in CI for both Ubuntu and macOS
### Are these changes tested?
<!--
We typically require tests for all PRs in order to:
1. Prevent the code from being accidentally broken by subsequent changes
2. Serve as another way to document the expected behavior of the code
If tests are not included in your PR, please explain why (for example, are they covered by existing tests)?
-->
yes, add pytest methods
### Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be updated before approving the PR.
-->
YES
Users can now import graphar and use APIs to read GraphAr metadata
```
import graphar
graphar.graph_info.GraphInfo.load(graph_path)
```
<!--
If there are any breaking changes to public APIs, please uncomment the line below and explain which changes are breaking.
-->
<!-- **BREAKING CHANGE: <description>** -->
<!--
Please uncomment the line below (and provide explanation) if the changes fix either
(a) a security vulnerability,
(b) a bug that caused incorrect or invalid data to be produced, or
(c) a bug that causes a crash (even when the API contract is upheld).
We use this to highlight fixes to issues that may affect users without their knowledge. For this reason, fixing bugs that cause errors don't count, since those are usually obvious.
-->
<!-- **Critical Fix: <description>** -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For additional commands, e-mail: commits-help@graphar.apache.org
Re: [PR] feat(python): refactor python module(from cli module) [incubator-graphar]
Posted by "yangxk1 (via GitHub)" <gi...@apache.org>.
yangxk1 commented on PR #799:
URL: https://github.com/apache/incubator-graphar/pull/799#issuecomment-3494594250
Looks great, I'll consider using ctypes to optimize the code!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For additional commands, e-mail: commits-help@graphar.apache.org
Re: [PR] feat(python): refactor python module(from cli module) [incubator-graphar]
Posted by "yangxk1 (via GitHub)" <gi...@apache.org>.
yangxk1 commented on PR #799:
URL: https://github.com/apache/incubator-graphar/pull/799#issuecomment-3489618992
@SemyonSinchenko , This PR only involves **schema yaml**.
At the same time I am working on another PR to build the `high_level` api. These two PRs are not designed for arrow operations. With these two APIs, our Python SDK can provide core functions.
After completing these two tasks, I will work on c arrow and pyarrow, which will allow users to operate the data set more precisely.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For additional commands, e-mail: commits-help@graphar.apache.org
Re: [PR] feat(python): refactor python module(from cli module) [incubator-graphar]
Posted by "Thespica (via GitHub)" <gi...@apache.org>.
Thespica commented on PR #799:
URL: https://github.com/apache/incubator-graphar/pull/799#issuecomment-3489630859
I think it will be weird to call `cli` as `client`. Consider rename it back to `cli` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For additional commands, e-mail: commits-help@graphar.apache.org
Re: [PR] feat(python): refactor python module(from cli module) [incubator-graphar]
Posted by "yangxk1 (via GitHub)" <gi...@apache.org>.
yangxk1 merged PR #799:
URL: https://github.com/apache/incubator-graphar/pull/799
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For additional commands, e-mail: commits-help@graphar.apache.org
Re: [PR] feat(python): refactor python module(from cli module) [incubator-graphar]
Posted by "yangxk1 (via GitHub)" <gi...@apache.org>.
yangxk1 commented on PR #799:
URL: https://github.com/apache/incubator-graphar/pull/799#issuecomment-3490267231
> I think it will be weird to call `cli`(command-line-interface) as `client`. Consider rename it back to `cli` ?
renamed client->cli
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For additional commands, e-mail: commits-help@graphar.apache.org
Re: [PR] feat(python): refactor python module(from cli module) [incubator-graphar]
Posted by "yangxk1 (via GitHub)" <gi...@apache.org>.
yangxk1 commented on PR #799:
URL: https://github.com/apache/incubator-graphar/pull/799#issuecomment-3489637231
> I think it will be weird to call `cli` as `client`. Consider rename it back to `cli` ?
An error occurred when I used `cli` as the package name. Maybe it conflicts with python's own package? I haven't thought of a good plan yet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For additional commands, e-mail: commits-help@graphar.apache.org
Re: [PR] feat(python): refactor python module(from cli module) [incubator-graphar]
Posted by "adsharma (via GitHub)" <gi...@apache.org>.
adsharma commented on PR #799:
URL: https://github.com/apache/incubator-graphar/pull/799#issuecomment-3494573719
I'm sure there are implementation details such as C++ enums. Was thinking about it from end user perspective:
```
from ctypes import cint64_t as i64
- assert id_property_type.to_type_name() == "int64"
+ assert id_property_type == i64
```
There may be type checkers that understand integer overflow and other concepts to find bugs that'd not be possible when you use those types vs graphar specific C++ enums.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For additional commands, e-mail: commits-help@graphar.apache.org
Re: [PR] feat(python): refactor python module(from cli module) [incubator-graphar]
Posted by "yangxk1 (via GitHub)" <gi...@apache.org>.
yangxk1 commented on PR #799:
URL: https://github.com/apache/incubator-graphar/pull/799#issuecomment-3489489065
cc @adsharma
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For additional commands, e-mail: commits-help@graphar.apache.org
Re: [PR] feat(python): refactor python module(from cli module) [incubator-graphar]
Posted by "adsharma (via GitHub)" <gi...@apache.org>.
adsharma commented on PR #799:
URL: https://github.com/apache/incubator-graphar/pull/799#issuecomment-3492088465
Looks great! This is what I had in mind before I got busy with LadybugDB.
One high level thing to consider: instead of exposing `graphar` types via `types_binding.cc`, consider using python's built-in `ctypes`.
Example:
https://github.com/py2many/py2many/blob/main/pycpp/clike.py#L15-L29
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For additional commands, e-mail: commits-help@graphar.apache.org
Re: [PR] feat(python): refactor python module(from cli module) [incubator-graphar]
Posted by "SemyonSinchenko (via GitHub)" <gi...@apache.org>.
SemyonSinchenko commented on PR #799:
URL: https://github.com/apache/incubator-graphar/pull/799#issuecomment-3489601566
@yangxk1 Nice work! Does it contain arrow-pyarrow bindings to cpp module? It should be possible to do in a zer-copy way in my understanding... Convert to arrow-type would be nice to expose too...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For additional commands, e-mail: commits-help@graphar.apache.org
Re: [PR] feat(python): refactor python module(from cli module) [incubator-graphar]
Posted by "yangxk1 (via GitHub)" <gi...@apache.org>.
yangxk1 commented on PR #799:
URL: https://github.com/apache/incubator-graphar/pull/799#issuecomment-3494541474
> One high level thing to consider: instead of exposing graphar types via types_binding.cc, consider using python's built-in ctypes.
Thanks for your suggestion, I will study ctypes carefully.
However, `types_binding.cc` mainly deals with **enum** classes in c++. I think this cannot be replaced by ctypes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@graphar.apache.org
For additional commands, e-mail: commits-help@graphar.apache.org