You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hugegraph.apache.org by Galaxy V <dy...@gmail.com> on 2023/10/26 07:41:44 UTC
[Proposal] Introduce new modules PD and Store components into HugeGraph
Hello everyone, I am VGalaxies, a new contributor to Apache HugeGraph. I am
excited to share with you a new feature that I have been working on for the
past few months. Currently, the architecture of HugeGraph is a monolithic
one centered around hugegraph-server. In order to improve the scalability
and performance of HugeGraph, the community have designed and implemented a
distributed architecture with two additional modules: hugegraph-pd and
hugegraph-store, which support graph data partitioning and multiple
replicas, and separate storage from computation for flexible scaling. The
responsibilities of these two modules are as follows: - hugegraph-pd: pd
stands for placement driver, which is a meta server responsible for service
discovery, partition information management, and node scheduling. -
hugegraph-store: this is a new built-in storage backend, which uses RocksDB
as the distributed backend storage engine. This feature is inspired by the
internal version of HugeGraph that has been used in some scenarios. I have
merged the code of PD and Store into HugeGraph on the preview branch
'pd-store'[1], and made some adjustments to the project structure,
configurations, and related codes. You can find the details in this
issue[2]. I have also done some preliminary tests and validations, and the
results are promising. I hope you are interested in this feature and would
like to participate in the development and testing. You can also leave your
comments and suggestions in the issue[2] or reply to me in this thread.
Appreciate any suggestion/feedback & contribution. Thank you for your
attention and support. Best regards, VGalaxies Reference:
1. https://github.com/apache/incubator-hugegraph/issues/2265
(summary-issue)
2. https://github.com/apache/incubator-hugegraph/tree/pd-store
(preview-branch)
Re: [Proposal] Introduce new modules PD and Store components into HugeGraph
Posted by Shiming Zhang <mi...@apache.org>.
Great feature ! ! !
On 2023/10/26 08:34:41 Galaxy V wrote:
> Hello everyone,
>
> I am VGalaxies. I apologize for the previous email, which was sent in
> HTML mode by mistake. This caused some formatting issues and made the
> email hard to read. Please ignore that email and refer to this one
> instead. Thank you for your understanding.
>
> [The rest of the email is the same as before]
>
> Hello everyone,
>
> I am VGalaxies, a new contributor to Apache HugeGraph. I am excited to
> share with you a new feature that I have been working on for the past
> few months.
>
> Currently, the architecture of HugeGraph is a monolithic one centered
> around hugegraph-server. In order to improve the scalability and
> performance of HugeGraph, the community have designed and implemented
> a distributed architecture with two additional modules: hugegraph-pd
> and hugegraph-store, which support graph data partitioning and
> multiple replicas, and separate storage from computation for flexible
> scaling.
>
> The responsibilities of these two modules are as follows:
>
> - hugegraph-pd: pd stands for placement driver, which is a meta server
> responsible for service discovery, partition information management,
> and node scheduling.
> - hugegraph-store: this is a new built-in storage backend, which uses
> RocksDB as the distributed backend storage engine.
>
> This feature is inspired by the internal version of HugeGraph that has
> been used in some scenarios. I have merged the code of PD and Store
> into HugeGraph on the preview branch 'pd-store'[1], and made some
> adjustments to the project structure, configurations, and related
> codes. You can find the details in this issue[2]. I have also done
> some preliminary tests and validations, and the results are promising.
>
> I hope you are interested in this feature and would like to
> participate in the development and testing. You can also leave your
> comments and suggestions in the issue[2] or reply to me in this
> thread. Appreciate any suggestion/feedback & contribution.
>
> Thank you for your attention and support.
>
> Best regards,
>
> VGalaxies
>
> Reference:
>
> 1. https://github.com/apache/incubator-hugegraph/issues/2265 (summary-issue)
> 2. https://github.com/apache/incubator-hugegraph/tree/pd-store (preview-branch)
>
Re: [Proposal] Introduce new modules PD and Store components into HugeGraph
Posted by Jacky Yang <ya...@gmail.com>.
Nice Feature ！
Imba Jin <ji...@apache.org> 于2023年10月27日周五 16:07写道：
>
> A very important feature, looking forward to future progress. In addition, we should supplement the corresponding architecture design and usage documents in the future.
>
> Keep moving~
>
> On 2023/10/26 08:34:41 Galaxy V wrote:
> > Hello everyone,
> >
> > I am VGalaxies. I apologize for the previous email, which was sent in
> > HTML mode by mistake. This caused some formatting issues and made the
> > email hard to read. Please ignore that email and refer to this one
> > instead. Thank you for your understanding.
> >
> > [The rest of the email is the same as before]
> >
> > Hello everyone,
> >
> > I am VGalaxies, a new contributor to Apache HugeGraph. I am excited to
> > share with you a new feature that I have been working on for the past
> > few months.
> >
> > Currently, the architecture of HugeGraph is a monolithic one centered
> > around hugegraph-server. In order to improve the scalability and
> > performance of HugeGraph, the community have designed and implemented
> > a distributed architecture with two additional modules: hugegraph-pd
> > and hugegraph-store, which support graph data partitioning and
> > multiple replicas, and separate storage from computation for flexible
> > scaling.
> >
> > The responsibilities of these two modules are as follows:
> >
> > - hugegraph-pd: pd stands for placement driver, which is a meta server
> > responsible for service discovery, partition information management,
> > and node scheduling.
> > - hugegraph-store: this is a new built-in storage backend, which uses
> > RocksDB as the distributed backend storage engine.
> >
> > This feature is inspired by the internal version of HugeGraph that has
> > been used in some scenarios. I have merged the code of PD and Store
> > into HugeGraph on the preview branch 'pd-store'[1], and made some
> > adjustments to the project structure, configurations, and related
> > codes. You can find the details in this issue[2]. I have also done
> > some preliminary tests and validations, and the results are promising.
> >
> > I hope you are interested in this feature and would like to
> > participate in the development and testing. You can also leave your
> > comments and suggestions in the issue[2] or reply to me in this
> > thread. Appreciate any suggestion/feedback & contribution.
> >
> > Thank you for your attention and support.
> >
> > Best regards,
> >
> > VGalaxies
> >
> > Reference:
> >
> > 1. https://github.com/apache/incubator-hugegraph/issues/2265 (summary-issue)
> > 2. https://github.com/apache/incubator-hugegraph/tree/pd-store (preview-branch)
> >
Re: [Proposal] Introduce new modules PD and Store components into HugeGraph
Posted by Imba Jin <ji...@apache.org>.
A very important feature, looking forward to future progress. In addition, we should supplement the corresponding architecture design and usage documents in the future.
Keep moving~
On 2023/10/26 08:34:41 Galaxy V wrote:
> Hello everyone,
>
> I am VGalaxies. I apologize for the previous email, which was sent in
> HTML mode by mistake. This caused some formatting issues and made the
> email hard to read. Please ignore that email and refer to this one
> instead. Thank you for your understanding.
>
> [The rest of the email is the same as before]
>
> Hello everyone,
>
> I am VGalaxies, a new contributor to Apache HugeGraph. I am excited to
> share with you a new feature that I have been working on for the past
> few months.
>
> Currently, the architecture of HugeGraph is a monolithic one centered
> around hugegraph-server. In order to improve the scalability and
> performance of HugeGraph, the community have designed and implemented
> a distributed architecture with two additional modules: hugegraph-pd
> and hugegraph-store, which support graph data partitioning and
> multiple replicas, and separate storage from computation for flexible
> scaling.
>
> The responsibilities of these two modules are as follows:
>
> - hugegraph-pd: pd stands for placement driver, which is a meta server
> responsible for service discovery, partition information management,
> and node scheduling.
> - hugegraph-store: this is a new built-in storage backend, which uses
> RocksDB as the distributed backend storage engine.
>
> This feature is inspired by the internal version of HugeGraph that has
> been used in some scenarios. I have merged the code of PD and Store
> into HugeGraph on the preview branch 'pd-store'[1], and made some
> adjustments to the project structure, configurations, and related
> codes. You can find the details in this issue[2]. I have also done
> some preliminary tests and validations, and the results are promising.
>
> I hope you are interested in this feature and would like to
> participate in the development and testing. You can also leave your
> comments and suggestions in the issue[2] or reply to me in this
> thread. Appreciate any suggestion/feedback & contribution.
>
> Thank you for your attention and support.
>
> Best regards,
>
> VGalaxies
>
> Reference:
>
> 1. https://github.com/apache/incubator-hugegraph/issues/2265 (summary-issue)
> 2. https://github.com/apache/incubator-hugegraph/tree/pd-store (preview-branch)
>
Re: [Proposal] Introduce new modules PD and Store components into HugeGraph
Posted by Galaxy V <dy...@gmail.com>.
Hello everyone,
I am VGalaxies. I apologize for the previous email, which was sent in
HTML mode by mistake. This caused some formatting issues and made the
email hard to read. Please ignore that email and refer to this one
instead. Thank you for your understanding.
[The rest of the email is the same as before]
Hello everyone,
I am VGalaxies, a new contributor to Apache HugeGraph. I am excited to
share with you a new feature that I have been working on for the past
few months.
Currently, the architecture of HugeGraph is a monolithic one centered
around hugegraph-server. In order to improve the scalability and
performance of HugeGraph, the community have designed and implemented
a distributed architecture with two additional modules: hugegraph-pd
and hugegraph-store, which support graph data partitioning and
multiple replicas, and separate storage from computation for flexible
scaling.
The responsibilities of these two modules are as follows:
- hugegraph-pd: pd stands for placement driver, which is a meta server
responsible for service discovery, partition information management,
and node scheduling.
- hugegraph-store: this is a new built-in storage backend, which uses
RocksDB as the distributed backend storage engine.
This feature is inspired by the internal version of HugeGraph that has
been used in some scenarios. I have merged the code of PD and Store
into HugeGraph on the preview branch 'pd-store'[1], and made some
adjustments to the project structure, configurations, and related
codes. You can find the details in this issue[2]. I have also done
some preliminary tests and validations, and the results are promising.
I hope you are interested in this feature and would like to
participate in the development and testing. You can also leave your
comments and suggestions in the issue[2] or reply to me in this
thread. Appreciate any suggestion/feedback & contribution.
Thank you for your attention and support.
Best regards,
VGalaxies
Reference:
1. https://github.com/apache/incubator-hugegraph/issues/2265 (summary-issue)
2. https://github.com/apache/incubator-hugegraph/tree/pd-store (preview-branch)