You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@giraph.apache.org by "Hyunsik Choi (JIRA)" <ji...@apache.org> on 2011/09/10 20:25:09 UTC
[jira] [Created] (GIRAPH-29) Implement TextVertexInputFormat for
text-format graph data
Implement TextVertexInputFormat for text-format graph data
----------------------------------------------------------
Key: GIRAPH-29
URL: https://issues.apache.org/jira/browse/GIRAPH-29
Project: Giraph
Issue Type: New Feature
Components: bsp
Reporter: Hyunsik Choi
Assignee: Hyunsik Choi
Priority: Minor
Fix For: 0.70.0
Supporting text-format graph data would be nice. It is helpful for developing graph algorithms and debugging because text-format graph data are human-readable and enable users to easily write sample data sets. Furthermore, text-format data are exchangeable regardless of operating systems or programming languages.
So, we need a basic InputFormat to help users develop user-defined InputFormat classes to deal text-represented graph data sets.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-29) Implement TextVertexInputFormat for
text-format graph data
Posted by "Avery Ching (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-29?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102142#comment-13102142 ]
Avery Ching commented on GIRAPH-29:
-----------------------------------
No problem, the real issue is that there is little documentation (my fault).
Contrary to MapReduce (map tasks = input splits), workers need not equal input split from VertexInputFormat. Workers in Giraph process InputSplits as fast as possible and may process 0 or more InputSplits.
> Implement TextVertexInputFormat for text-format graph data
> ----------------------------------------------------------
>
> Key: GIRAPH-29
> URL: https://issues.apache.org/jira/browse/GIRAPH-29
> Project: Giraph
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Priority: Minor
> Fix For: 0.70.0
>
>
> Supporting text-format graph data would be nice. It is helpful for developing graph algorithms and debugging because text-format graph data are human-readable and enable users to easily write sample data sets. Furthermore, text-format data are exchangeable regardless of operating systems or programming languages.
> So, we need a basic InputFormat to help users develop user-defined InputFormat classes to deal text-represented graph data sets.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (GIRAPH-29) Implement TextVertexInputFormat for
text-format graph data
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-29?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyunsik Choi resolved GIRAPH-29.
--------------------------------
Resolution: Won't Fix
> Implement TextVertexInputFormat for text-format graph data
> ----------------------------------------------------------
>
> Key: GIRAPH-29
> URL: https://issues.apache.org/jira/browse/GIRAPH-29
> Project: Giraph
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Priority: Minor
> Fix For: 0.70.0
>
>
> Supporting text-format graph data would be nice. It is helpful for developing graph algorithms and debugging because text-format graph data are human-readable and enable users to easily write sample data sets. Furthermore, text-format data are exchangeable regardless of operating systems or programming languages.
> So, we need a basic InputFormat to help users develop user-defined InputFormat classes to deal text-represented graph data sets.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-29) Implement TextVertexInputFormat for
text-format graph data
Posted by "Avery Ching (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-29?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102104#comment-13102104 ]
Avery Ching commented on GIRAPH-29:
-----------------------------------
I agree that text format graph data is nice.
We have a helper abstract classes based on TextInputFormat and TextOutputFormat to do this:
org.apache.giraph.lib.TextVertexInputFormat
org.apache.giraph.lib.TextVertexOutputFormat
An example implementation that uses those helper classes is
org.apache.giraph.lib.JsonBase64VertexInputFormat
org.apache.giraph.lib.JsonBase64VertexInputFormat
Does this satisfy your needs? Any suggestions for improvement?
> Implement TextVertexInputFormat for text-format graph data
> ----------------------------------------------------------
>
> Key: GIRAPH-29
> URL: https://issues.apache.org/jira/browse/GIRAPH-29
> Project: Giraph
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Priority: Minor
> Fix For: 0.70.0
>
>
> Supporting text-format graph data would be nice. It is helpful for developing graph algorithms and debugging because text-format graph data are human-readable and enable users to easily write sample data sets. Furthermore, text-format data are exchangeable regardless of operating systems or programming languages.
> So, we need a basic InputFormat to help users develop user-defined InputFormat classes to deal text-represented graph data sets.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-29) Implement TextVertexInputFormat for
text-format graph data
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-29?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102111#comment-13102111 ]
Hyunsik Choi commented on GIRAPH-29:
------------------------------------
I'm sorry for my big mistake. I overlookd org.apache.giraph.lib package.
I have a question. When a program use TextVertexInputFormat, the active workers are determined by the number of blocks? How does giraph work when the blocks are more than numWorkers? Should the numWorkers is set by user by considering both the length of input data and the number of numWorkers.
> Implement TextVertexInputFormat for text-format graph data
> ----------------------------------------------------------
>
> Key: GIRAPH-29
> URL: https://issues.apache.org/jira/browse/GIRAPH-29
> Project: Giraph
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Priority: Minor
> Fix For: 0.70.0
>
>
> Supporting text-format graph data would be nice. It is helpful for developing graph algorithms and debugging because text-format graph data are human-readable and enable users to easily write sample data sets. Furthermore, text-format data are exchangeable regardless of operating systems or programming languages.
> So, we need a basic InputFormat to help users develop user-defined InputFormat classes to deal text-represented graph data sets.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GIRAPH-29) Implement TextVertexInputFormat for
text-format graph data
Posted by "Hyunsik Choi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GIRAPH-29?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13102177#comment-13102177 ]
Hyunsik Choi commented on GIRAPH-29:
------------------------------------
Thank you for your kind reply.
> Implement TextVertexInputFormat for text-format graph data
> ----------------------------------------------------------
>
> Key: GIRAPH-29
> URL: https://issues.apache.org/jira/browse/GIRAPH-29
> Project: Giraph
> Issue Type: New Feature
> Components: bsp
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Priority: Minor
> Fix For: 0.70.0
>
>
> Supporting text-format graph data would be nice. It is helpful for developing graph algorithms and debugging because text-format graph data are human-readable and enable users to easily write sample data sets. Furthermore, text-format data are exchangeable regardless of operating systems or programming languages.
> So, we need a basic InputFormat to help users develop user-defined InputFormat classes to deal text-represented graph data sets.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira