You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@systemml.apache.org by de...@apache.org on 2017/02/13 20:46:56 UTC
incubator-systemml git commit: [SYSTEMML-1239] DML Language Reference
write description parameter
Repository: incubator-systemml
Updated Branches:
refs/heads/master 100075046 -> 5665e152e
[SYSTEMML-1239] DML Language Reference write description parameter
Give example of write description parameter in DML Language Reference.
Describe author and created parameters.
Update metadata examples in DML Language Reference and Beginner's Guide
to DML and PyDML.
Closes #385.
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-systemml/commit/5665e152
Tree: http://git-wip-us.apache.org/repos/asf/incubator-systemml/tree/5665e152
Diff: http://git-wip-us.apache.org/repos/asf/incubator-systemml/diff/5665e152
Branch: refs/heads/master
Commit: 5665e152ef2955c7a34970cc64ed2bfc9f3b4042
Parents: 1000750
Author: Deron Eriksson <de...@us.ibm.com>
Authored: Mon Feb 13 12:44:10 2017 -0800
Committer: Deron Eriksson <de...@us.ibm.com>
Committed: Mon Feb 13 12:44:10 2017 -0800
----------------------------------------------------------------------
docs/beginners-guide-to-dml-and-pydml.md | 59 ++++++++++++++-------------
docs/dml-language-reference.md | 56 +++++++++++++++++++------
2 files changed, 74 insertions(+), 41 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/5665e152/docs/beginners-guide-to-dml-and-pydml.md
----------------------------------------------------------------------
diff --git a/docs/beginners-guide-to-dml-and-pydml.md b/docs/beginners-guide-to-dml-and-pydml.md
index 67f1c96..8479ef7 100644
--- a/docs/beginners-guide-to-dml-and-pydml.md
+++ b/docs/beginners-guide-to-dml-and-pydml.md
@@ -283,14 +283,15 @@ is 0-based.*
</div>
<div data-lang="m.txt.mtd" markdown="1">
- {
- "data_type": "matrix"
- ,"value_type": "double"
- ,"rows": 4
- ,"cols": 3
- ,"nnz": 6
- ,"format": "text"
- ,"description": { "author": "SystemML" }
+ {
+ "data_type": "matrix",
+ "value_type": "double",
+ "rows": 4,
+ "cols": 3,
+ "nnz": 6,
+ "format": "text",
+ "author": "SystemML",
+ "created": "2017-01-01 00:00:01 PST"
}
</div>
@@ -313,16 +314,17 @@ is 0-based.*
</div>
<div data-lang="m.csv.mtd" markdown="1">
- {
- "data_type": "matrix"
- ,"value_type": "double"
- ,"rows": 4
- ,"cols": 3
- ,"nnz": 6
- ,"format": "csv"
- ,"header": false
- ,"sep": ","
- ,"description": { "author": "SystemML" }
+ {
+ "data_type": "matrix",
+ "value_type": "double",
+ "rows": 4,
+ "cols": 3,
+ "nnz": 6,
+ "format": "csv",
+ "header": false,
+ "sep": ",",
+ "author": "SystemML",
+ "created": "2017-01-01 00:00:01 PST"
}
</div>
@@ -331,16 +333,17 @@ is 0-based.*
</div>
<div data-lang="m.binary.mtd" markdown="1">
- {
- "data_type": "matrix"
- ,"value_type": "double"
- ,"rows": 4
- ,"cols": 3
- ,"rows_in_block": 1000
- ,"cols_in_block": 1000
- ,"nnz": 6
- ,"format": "binary"
- ,"description": { "author": "SystemML" }
+ {
+ "data_type": "matrix",
+ "value_type": "double",
+ "rows": 4,
+ "cols": 3,
+ "rows_in_block": 1000,
+ "cols_in_block": 1000,
+ "nnz": 6,
+ "format": "binary",
+ "author": "SystemML",
+ "created": "2017-01-01 00:00:01 PST"
}
</div>
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/5665e152/docs/dml-language-reference.md
----------------------------------------------------------------------
diff --git a/docs/dml-language-reference.md b/docs/dml-language-reference.md
index 05625fd..22ec0d9 100644
--- a/docs/dml-language-reference.md
+++ b/docs/dml-language-reference.md
@@ -933,7 +933,8 @@ Below, we have examples of this matrix in the CSV, Matrix Market, IJV, and Binar
"format": "csv",
"header": false,
"sep": ",",
- "description": { "author": "SystemML" }
+ "author": "SystemML",
+ "created": "2017-01-01 00:00:01 PST"
}
</div>
@@ -965,7 +966,8 @@ Below, we have examples of this matrix in the CSV, Matrix Market, IJV, and Binar
"cols": 3,
"nnz": 6,
"format": "text",
- "description": { "author": "SystemML" }
+ "author": "SystemML",
+ "created": "2017-01-01 00:00:01 PST"
}
</div>
@@ -983,7 +985,8 @@ Below, we have examples of this matrix in the CSV, Matrix Market, IJV, and Binar
"cols_in_block": 1000,
"nnz": 6,
"format": "binary",
- "description": { "author": "SystemML" }
+ "author": "SystemML",
+ "created": "2017-01-01 00:00:01 PST"
}
</div>
@@ -992,12 +995,13 @@ Below, we have examples of this matrix in the CSV, Matrix Market, IJV, and Binar
As another example, here we see the content of the MTD file `scalar.mtd` associated with a scalar data file `scalar`
that contains the scalar value 2.0.
- {
- "data_type": "scalar",
- "value_type": "double",
- "format": "text",
- "description": { "author": "SystemML" }
- }
+ {
+ "data_type": "scalar",
+ "value_type": "double",
+ "format": "text",
+ "author": "SystemML",
+ "created": "2017-01-01 00:00:01 PST"
+ }
Metadata is represented as an MTD file that contains a single JSON object with the attributes described below.
@@ -1015,6 +1019,8 @@ Parameter Name | Description | Optional | Permissible values | Data type valid f
`nnz` | Number of non-zero values | Yes | any integer > `0` | `matrix`
`format` | Data file format | Yes. Default value is `text` | `csv`, `mm`, `text`, `binary` | `matrix`, `scalar`. Formats `csv` and `mm` are applicable only to matrices
`description` | Description of the data | Yes | Any valid JSON string or object | `matrix`, `scalar`
+`author` | User that created the metadata file, defaults to `SystemML` | N/A | N/A | N/A
+`created` | Date/time when metadata file was written | N/A | N/A | N/A
In addition, when reading or writing CSV files, the metadata may contain one or more of the following five attributes.
@@ -1126,7 +1132,8 @@ Example content of `out/file.ijv.mtd`:
"cols": 8,
"nnz": 4,
"format": "text",
- "description": { "author": "SystemML" }
+ "author": "SystemML",
+ "created": "2017-01-01 00:00:01 PST"
}
Write `V` to `out/file` in `binary` format:
@@ -1144,7 +1151,8 @@ Example content of `out/file.mtd`:
"rows_in_block": 1000,
"cols_in_block": 1000,
"format": "binary",
- "description": { "author": "SystemML" }
+ "author": "SystemML",
+ "created": "2017-01-01 00:00:01 PST"
}
Write `V` to `n.csv` in `csv` format with column headers, `";"` as delimiter, and zero values are not written.
@@ -1162,7 +1170,8 @@ Example content of `n.csv.mtd`:
"format": "csv",
"header": true,
"sep": ";",
- "description": { "author": "SystemML" }
+ "author": "SystemML",
+ "created": "2017-01-01 00:00:01 PST"
}
Write `x` integer value to file `out/scalar_i`
@@ -1175,7 +1184,8 @@ Example content of `out/scalar_i.mtd`:
"data_type": "scalar",
"value_type": "int",
"format": "text",
- "description": { "author": "SystemML" }
+ "author": "SystemML",
+ "created": "2017-01-01 00:00:01 PST"
}
Unlike `read`, the `write` function does not need a constant string expression, so the following example will work:
@@ -1186,6 +1196,26 @@ Unlike `read`, the `write` function does not need a constant string expression,
file = "A" + i + ".mtx";
write(A, dir + file, format="csv");
+The `description` parameter can be used to attach a description to the metadata:
+
+ A = matrix("1 2 3 4", rows=2, cols=2)
+ write(A, "mymatrix.csv", format="csv", description="my matrix")
+
+This will generate the following `mymatrix.csv.mtd` metadata file:
+
+ {
+ "data_type": "matrix",
+ "value_type": "double",
+ "rows": 2,
+ "cols": 2,
+ "nnz": 4,
+ "format": "csv",
+ "header": false,
+ "sep": ",",
+ "description": "my matrix",
+ "author": "SystemML",
+ "created": "2017-01-01 00:00:01 PST"
+ }
### Data Pre-Processing Built-In Functions