You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hama.apache.org by "Samuel Guo (JIRA)" <ji...@apache.org> on 2008/10/21 04:38:44 UTC
[jira] Created: (HAMA-86) Implements generic load/save operations
in hama
Implements generic load/save operations in hama
-----------------------------------------------
Key: HAMA-86
URL: https://issues.apache.org/jira/browse/HAMA-86
Project: Hama
Issue Type: Improvement
Components: implementation
Affects Versions: 0.1.0
Reporter: Samuel Guo
Fix For: 0.1.0
we need a generic "pathname" class to identify the input & output.
"pathname" can contain two parts: one is scheme, such as HBASE or HDFS; one will be the actual name, such as table name in HBase, filename in HDFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-86) Implements generic load/save operations
in hama
Posted by "Samuel Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12641755#action_12641755 ]
Samuel Guo commented on HAMA-86:
--------------------------------
I think there are different features between a matrix in HBase & a matrix in FileSystem.
- A matrix in HBase has a fixed format. But matrices in FileSystem may have different format, parsing of the file as a matrix depends on the format.
- A matrix in HBase is a full-featured matrix. we can randomly access it by its row & column. But matrices in FileSystem don't. A matrix as block-indexing format may be useful for multiplication, but may be not efficient in other computation(just my guessing).
for example:
/**
* Save to a table or file
*
* @param path
* @return true if saved
* @throws IOException
*/
public boolean save(String path) throws IOException;
if we implements "save to file" operation in matrix, we may need an OutputFormat to identify the format of the file.
so I suggest that let HamaAdmin just keep all the aliases of the matrices in HBase.
And add a method like "boolean saveAsFile(Path path, OutputFormat format);" to deal with file I/O of matrices.
Any ideas?
> Implements generic load/save operations in hama
> -----------------------------------------------
>
> Key: HAMA-86
> URL: https://issues.apache.org/jira/browse/HAMA-86
> Project: Hama
> Issue Type: Improvement
> Components: implementation
> Affects Versions: 0.1.0
> Reporter: Samuel Guo
> Fix For: 0.1.0
>
>
> we need a generic "pathname" class to identify the input & output.
> "pathname" can contain two parts: one is scheme, such as HBASE or HDFS; one will be the actual name, such as table name in HBase, filename in HDFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-86) Implements generic load/save operations
in hama
Posted by "Samuel Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12641256#action_12641256 ]
Samuel Guo commented on HAMA-86:
--------------------------------
let's discuss the generic load/save implementation here.
> Implements generic load/save operations in hama
> -----------------------------------------------
>
> Key: HAMA-86
> URL: https://issues.apache.org/jira/browse/HAMA-86
> Project: Hama
> Issue Type: Improvement
> Components: implementation
> Affects Versions: 0.1.0
> Reporter: Samuel Guo
> Fix For: 0.1.0
>
>
> we need a generic "pathname" class to identify the input & output.
> "pathname" can contain two parts: one is scheme, such as HBASE or HDFS; one will be the actual name, such as table name in HBase, filename in HDFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HAMA-86) Implements generic load/save operations
in hama
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12641798#action_12641798 ]
Edward J. Yoon commented on HAMA-86:
------------------------------------
...
>> so I suggest that let HamaAdmin just keep all the aliases of the matrices in HBase.
Yes, I agree with you.
>> And add a method like "boolean saveAsFile(Path path, OutputFormat format);" to deal with file I/O of matrices.
That's good idea.
> Implements generic load/save operations in hama
> -----------------------------------------------
>
> Key: HAMA-86
> URL: https://issues.apache.org/jira/browse/HAMA-86
> Project: Hama
> Issue Type: Improvement
> Components: implementation
> Affects Versions: 0.1.0
> Reporter: Samuel Guo
> Fix For: 0.1.0
>
>
> we need a generic "pathname" class to identify the input & output.
> "pathname" can contain two parts: one is scheme, such as HBASE or HDFS; one will be the actual name, such as table name in HBase, filename in HDFS.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HAMA-86) Implements generic load/save operations
in hama
Posted by "Samuel Guo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-86?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Samuel Guo updated HAMA-86:
---------------------------
Description:
Let's discuss the generic load/save operation from/to other sources(such as, FileSystem).
matrices in FileSystem may have different format, parsing of the file as a matrix depends on the format.
we may need some methods like:
/**
* Save a matrix as file using specified OutputFormat.
*
* @param path
* the file path.
* @param format
* the format of the matrix.
* @return true if saved
* @throws IOException
*/
public boolean saveAsFile(Path filepath, OutputFormat format) throws IOException;
/**
* Load a matrix from file
*
* @param path
* the file path.
* @param format
* the format of the matrix.
* @return true if saved
* @throws IOException
*/
public static Matrix loadFromFile(Path filepath, OutputFormat format) throws IOException;
was:
we need a generic "pathname" class to identify the input & output.
"pathname" can contain two parts: one is scheme, such as HBASE or HDFS; one will be the actual name, such as table name in HBase, filename in HDFS.
I've changed the description.
> Implements generic load/save operations in hama
> -----------------------------------------------
>
> Key: HAMA-86
> URL: https://issues.apache.org/jira/browse/HAMA-86
> Project: Hama
> Issue Type: Improvement
> Components: implementation
> Affects Versions: 0.1.0
> Reporter: Samuel Guo
> Fix For: 0.1.0
>
>
> Let's discuss the generic load/save operation from/to other sources(such as, FileSystem).
> matrices in FileSystem may have different format, parsing of the file as a matrix depends on the format.
> we may need some methods like:
> /**
> * Save a matrix as file using specified OutputFormat.
> *
> * @param path
> * the file path.
> * @param format
> * the format of the matrix.
> * @return true if saved
> * @throws IOException
> */
> public boolean saveAsFile(Path filepath, OutputFormat format) throws IOException;
> /**
> * Load a matrix from file
> *
> * @param path
> * the file path.
> * @param format
> * the format of the matrix.
> * @return true if saved
> * @throws IOException
> */
> public static Matrix loadFromFile(Path filepath, OutputFormat format) throws IOException;
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HAMA-86) Implements generic load/save operations
in hama
Posted by "Edward J. Yoon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HAMA-86?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Edward J. Yoon resolved HAMA-86.
--------------------------------
Resolution: Later
It's a later issue.
> Implements generic load/save operations in hama
> -----------------------------------------------
>
> Key: HAMA-86
> URL: https://issues.apache.org/jira/browse/HAMA-86
> Project: Hama
> Issue Type: Improvement
> Components: matrix
> Affects Versions: 0.1.0
> Reporter: Samuel Guo
> Fix For: 0.1.0
>
>
> Let's discuss the generic load/save operation from/to other sources(such as, FileSystem).
> matrices in FileSystem may have different format, parsing of the file as a matrix depends on the format.
> we may need some methods like:
> /**
> * Save a matrix as file using specified OutputFormat.
> *
> * @param path
> * the file path.
> * @param format
> * the format of the matrix.
> * @return true if saved
> * @throws IOException
> */
> public boolean saveAsFile(Path filepath, OutputFormat format) throws IOException;
> /**
> * Load a matrix from file
> *
> * @param path
> * the file path.
> * @param format
> * the format of the matrix.
> * @return true if saved
> * @throws IOException
> */
> public static Matrix loadFromFile(Path filepath, OutputFormat format) throws IOException;
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.