Slow VertexIter using S3

[Muniraju Pulipalyam <mu...@hotmail.com>]

Hi All,

 I have a graph of 5000 vertices constructed and stored in S3. I have two issues:

  1.
 When I try to iterate over it using VertexCollection, VertexIter is very slow. Works just fine using local storage. It takes a long time in: readers_.emplace_back(vertex_info, pg, prefix); in graph_reader.h. Is this a known problem? I am running from a compute instance which is in the same region/network as the S3 instance.
  2.
Why does graphar::GraphInfo::Load require '?' In the path for S3 always? In PathToDirectory method, you are finding suffix of '?'. If it is not found, shouldn't it be just ignored?

Thanks,
Muni

  1.

Re: Slow VertexIter using S3

[Weibin Zeng <we...@apache.org>]

Hi, Muniraju, 

Thanks for the feedback.

1. We didn't test the VertexCollection iteration performance with S3 before, so it may has some problem. Can you open an issue for this problem and we can reproduce and fix that?

2. sorry, I'am a little confused for the problem, what's the `?` you mean in the path, if necessary, feel free to open an issue.

Best,
Weibin Zeng

On 2024/09/02 08:00:13 Muniraju Pulipalyam wrote:
> Hi All,
> 
>  I have a graph of 5000 vertices constructed and stored in S3. I have two issues:
> 
>   1.
>  When I try to iterate over it using VertexCollection, VertexIter is very slow. Works just fine using local storage. It takes a long time in: readers_.emplace_back(vertex_info, pg, prefix); in graph_reader.h. Is this a known problem? I am running from a compute instance which is in the same region/network as the S3 instance.
>   2.
> Why does graphar::GraphInfo::Load require '?' In the path for S3 always? In PathToDirectory method, you are finding suffix of '?'. If it is not found, shouldn't it be just ignored?
> 
> Thanks,
> Muni
> 
>   1.
> 
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@graphar.apache.org
For additional commands, e-mail: dev-help@graphar.apache.org