You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by greghogan <gi...@git.apache.org> on 2017/07/15 00:47:19 UTC
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
GitHub user greghogan opened a pull request:
https://github.com/apache/flink/pull/4346
[FLINK-7199] [gelly] Graph simplification does not set parallelism
The Simplify parameter should accept and set the parallelism when calling the Simplify algorithms.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/greghogan/flink 7199_graph_simplification_does_not_set_parallelism
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4346.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4346
----
commit d1e9dd0711d5b0925928760eba77ebf4d627e288
Author: Greg Hogan <co...@greghogan.com>
Date: 2017-07-15T00:35:51Z
[FLINK-7199] [gelly] Graph simplification does not set parallelism
The Simplify parameter should accept and set the parallelism when
calling the Simplify algorithms.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r132452312
--- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/drivers/parameter/LongParameter.java ---
@@ -52,16 +52,6 @@ public LongParameter(ParameterizedBase owner, String name) {
public LongParameter setDefaultValue(long defaultValue) {
super.setDefaultValue(defaultValue);
- if (hasMinimumValue) {
- Util.checkParameter(defaultValue >= minimumValue,
- "Default value (" + defaultValue + ") must be greater than or equal to minimum (" + minimumValue + ")");
- }
-
- if (hasMaximumValue) {
- Util.checkParameter(defaultValue <= maximumValue,
- "Default value (" + defaultValue + ") must be less than or equal to maximum (" + maximumValue + ")");
- }
-
--- End diff --
ok, should we make this explicit in the `LongParamater` class' javadoc? For example:
```
/**
* A {@link Parameter} storing a {@link Long} within <tt>min</tt> and <tt>max</tt> bounds (inclusive).
*
* <p>Note that the default value may be outside of these bounds.
*/
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r132401190
--- Diff: flink-libraries/flink-gelly-examples/src/test/java/org/apache/flink/graph/drivers/EdgeListITCase.java ---
@@ -56,214 +56,304 @@ public void testLongDescription() throws Exception {
ProgramParametrizationException.class);
}
+ // CirculantGraph
+
+ private String[] getCirculantGraphParamters(String output) {
+ return parameters("CirculantGraph", output, "--vertex_count", "42", "--range0", "13:4");
+ }
+
@Test
public void testHashWithCirculantGraph() throws Exception {
- expectedChecksum(
- parameters("CirculantGraph", "hash", "--vertex_count", "42", "--range0", "13:4"),
- 168, 0x000000000001ae80);
+ expectedChecksum(getCirculantGraphParamters("hash"), 168, 0x000000000001ae80);
}
@Test
public void testPrintWithCirculantGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CirculantGraph", "print", "--vertex_count", "42", "--range0", "13:4"),
- new Checksum(168, 0x0000004bdcc52cbcL));
+ expectedOutputChecksum(getCirculantGraphParamters("print"), new Checksum(168, 0x0000004bdcc52cbcL));
+ }
+
+ @Test
+ public void testParallelismWithCirculantGraph() throws Exception {
+ TestUtils.verifyParallelism(getCirculantGraphParamters("print"));
+ }
+
+ // CompleteGraph
+
+ private String[] getCompleteGraphParamters(String output) {
+ return parameters("CompleteGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCompleteGraph() throws Exception {
- expectedChecksum(
- parameters("CompleteGraph", "hash", "--vertex_count", "42"),
- 1722, 0x0000000000113ca0L);
+ expectedChecksum(getCompleteGraphParamters("hash"), 1722, 0x0000000000113ca0L);
}
@Test
public void testPrintWithCompleteGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CompleteGraph", "print", "--vertex_count", "42"),
- new Checksum(1722, 0x0000031109a0c398L));
+ expectedOutputChecksum(getCompleteGraphParamters("print"), new Checksum(1722, 0x0000031109a0c398L));
+ }
+
+ @Test
+ public void testParallelismWithCompleteGraph() throws Exception {
+ TestUtils.verifyParallelism(getCompleteGraphParamters("print"));
+ }
+
+ // CycleGraph
+
+ private String[] getCycleGraphParamters(String output) {
+ return parameters("CycleGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCycleGraph() throws Exception {
- expectedChecksum(
- parameters("CycleGraph", "hash", "--vertex_count", "42"),
- 84, 0x000000000000d740L);
+ expectedChecksum(getCycleGraphParamters("hash"), 84, 0x000000000000d740L);
}
@Test
public void testPrintWithCycleGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CycleGraph", "print", "--vertex_count", "42"),
- new Checksum(84, 0x000000272a136fcaL));
+ expectedOutputChecksum(getCycleGraphParamters("print"), new Checksum(84, 0x000000272a136fcaL));
+ }
+
+ @Test
+ public void testParallelismWithCycleGraph() throws Exception {
+ TestUtils.verifyParallelism(getCycleGraphParamters("print"));
+ }
+
+ // EchoGraph
+
+ private String[] getEchoGraphParamters(String output) {
+ return parameters("EchoGraph", output, "--vertex_count", "42", "--vertex_degree", "13");
}
@Test
public void testHashWithEchoGraph() throws Exception {
- expectedChecksum(
- parameters("EchoGraph", "hash", "--vertex_count", "42", "--vertex_degree", "13"),
- 546, 0x0000000000057720L);
+ expectedChecksum(getEchoGraphParamters("hash"), 546, 0x0000000000057720L);
}
@Test
public void testPrintWithEchoGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("EchoGraph", "print", "--vertex_count", "42", "--vertex_degree", "13"),
- new Checksum(546, 0x000000f7190b8fcaL));
+ expectedOutputChecksum(getEchoGraphParamters("print"), new Checksum(546, 0x000000f7190b8fcaL));
+ }
+
+ @Test
+ public void testParallelismWithEchoGraph() throws Exception {
+ TestUtils.verifyParallelism(getEchoGraphParamters("print"));
+ }
+
+ // EmptyGraph
+
+ private String[] getEmptyGraphParamters(String output) {
+ return parameters("EmptyGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithEmptyGraph() throws Exception {
- expectedChecksum(
- parameters("EmptyGraph", "hash", "--vertex_count", "42"),
- 0, 0x0000000000000000L);
+ expectedChecksum(getEmptyGraphParamters("hash"), 0, 0x0000000000000000L);
+ }
+
+ @Test
+ public void testPrintWithEmptyGraph() throws Exception {
+ expectedOutputChecksum(getEmptyGraphParamters("print"), new Checksum(0, 0x0000000000000000L));
+ }
+
+ @Test
+ public void testParallelismWithEmptyGraph() throws Exception {
+ TestUtils.verifyParallelism(getEmptyGraphParamters("print"));
+ }
+
+ // GridGraph
+
+ private String[] getGridGraphParamters(String output) {
+ return parameters("GridGraph", output, "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true");
}
@Test
public void testHashWithGridGraph() throws Exception {
- expectedChecksum(
- parameters("GridGraph", "hash", "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true"),
- 130, 0x000000000000eba0L);
+ expectedChecksum(getGridGraphParamters("hash"), 130, 0x000000000000eba0L);
}
@Test
public void testPrintWithGridGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("GridGraph", "print", "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true"),
- new Checksum(130, 0x00000033237d24eeL));
+ expectedOutputChecksum(getGridGraphParamters("print"), new Checksum(130, 0x00000033237d24eeL));
+ }
+
+ @Test
+ public void testParallelismWithGridGraph() throws Exception {
+ TestUtils.verifyParallelism(getGridGraphParamters("print"));
+ }
+
+ // HypercubeGraph
+
+ private String[] getHypercubeGraphParamters(String output) {
+ return parameters("HypercubeGraph", output, "--dimensions", "7");
}
@Test
public void testHashWithHypercubeGraph() throws Exception {
- expectedChecksum(
- parameters("HypercubeGraph", "hash", "--dimensions", "7"),
- 896, 0x00000000001bc800L);
+ expectedChecksum(getHypercubeGraphParamters("hash"), 896, 0x00000000001bc800L);
}
@Test
public void testPrintWithHypercubeGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("HypercubeGraph", "print", "--dimensions", "7"),
- new Checksum(896, 0x000001f243ee33b2L));
+ expectedOutputChecksum(getHypercubeGraphParamters("print"), new Checksum(896, 0x000001f243ee33b2L));
+ }
+
+ @Test
+ public void testParallelismWithHypercubeGraph() throws Exception {
+ TestUtils.verifyParallelism(getHypercubeGraphParamters("print"));
+ }
+
+ // PathGraph
+
+ private String[] getPathGraphParamters(String output) {
--- End diff --
`getPathGraphParameters`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4346: [FLINK-7199] [gelly] Graph simplification does not set pa...
Posted by greghogan <gi...@git.apache.org>.
Github user greghogan commented on the issue:
https://github.com/apache/flink/pull/4346
@NicoK I have fixed the noted issues and will merge.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by greghogan <gi...@git.apache.org>.
Github user greghogan commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r129282059
--- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/drivers/input/InputBase.java ---
@@ -32,6 +35,9 @@
extends ParameterizedBase
implements Input<K, VV, EV> {
+ protected LongParameter parallelism = new LongParameter(this, "__parallelism")
+ .setDefaultValue(PARALLELISM_DEFAULT);
--- End diff --
We didn't write an `IntParameter`. These are only used on the client when parsing command-line parameters.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r129230954
--- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/drivers/input/InputBase.java ---
@@ -32,6 +35,9 @@
extends ParameterizedBase
implements Input<K, VV, EV> {
+ protected LongParameter parallelism = new LongParameter(this, "__parallelism")
+ .setDefaultValue(PARALLELISM_DEFAULT);
--- End diff --
I know, it's been that way before, but what a `LongParameter` although only the `intValue()` of it is actually used?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r132401371
--- Diff: flink-libraries/flink-gelly-examples/src/test/java/org/apache/flink/graph/drivers/EdgeListITCase.java ---
@@ -56,214 +56,304 @@ public void testLongDescription() throws Exception {
ProgramParametrizationException.class);
}
+ // CirculantGraph
+
+ private String[] getCirculantGraphParamters(String output) {
+ return parameters("CirculantGraph", output, "--vertex_count", "42", "--range0", "13:4");
+ }
+
@Test
public void testHashWithCirculantGraph() throws Exception {
- expectedChecksum(
- parameters("CirculantGraph", "hash", "--vertex_count", "42", "--range0", "13:4"),
- 168, 0x000000000001ae80);
+ expectedChecksum(getCirculantGraphParamters("hash"), 168, 0x000000000001ae80);
}
@Test
public void testPrintWithCirculantGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CirculantGraph", "print", "--vertex_count", "42", "--range0", "13:4"),
- new Checksum(168, 0x0000004bdcc52cbcL));
+ expectedOutputChecksum(getCirculantGraphParamters("print"), new Checksum(168, 0x0000004bdcc52cbcL));
+ }
+
+ @Test
+ public void testParallelismWithCirculantGraph() throws Exception {
+ TestUtils.verifyParallelism(getCirculantGraphParamters("print"));
+ }
+
+ // CompleteGraph
+
+ private String[] getCompleteGraphParamters(String output) {
+ return parameters("CompleteGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCompleteGraph() throws Exception {
- expectedChecksum(
- parameters("CompleteGraph", "hash", "--vertex_count", "42"),
- 1722, 0x0000000000113ca0L);
+ expectedChecksum(getCompleteGraphParamters("hash"), 1722, 0x0000000000113ca0L);
}
@Test
public void testPrintWithCompleteGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CompleteGraph", "print", "--vertex_count", "42"),
- new Checksum(1722, 0x0000031109a0c398L));
+ expectedOutputChecksum(getCompleteGraphParamters("print"), new Checksum(1722, 0x0000031109a0c398L));
+ }
+
+ @Test
+ public void testParallelismWithCompleteGraph() throws Exception {
+ TestUtils.verifyParallelism(getCompleteGraphParamters("print"));
+ }
+
+ // CycleGraph
+
+ private String[] getCycleGraphParamters(String output) {
+ return parameters("CycleGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCycleGraph() throws Exception {
- expectedChecksum(
- parameters("CycleGraph", "hash", "--vertex_count", "42"),
- 84, 0x000000000000d740L);
+ expectedChecksum(getCycleGraphParamters("hash"), 84, 0x000000000000d740L);
}
@Test
public void testPrintWithCycleGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CycleGraph", "print", "--vertex_count", "42"),
- new Checksum(84, 0x000000272a136fcaL));
+ expectedOutputChecksum(getCycleGraphParamters("print"), new Checksum(84, 0x000000272a136fcaL));
+ }
+
+ @Test
+ public void testParallelismWithCycleGraph() throws Exception {
+ TestUtils.verifyParallelism(getCycleGraphParamters("print"));
+ }
+
+ // EchoGraph
+
+ private String[] getEchoGraphParamters(String output) {
+ return parameters("EchoGraph", output, "--vertex_count", "42", "--vertex_degree", "13");
}
@Test
public void testHashWithEchoGraph() throws Exception {
- expectedChecksum(
- parameters("EchoGraph", "hash", "--vertex_count", "42", "--vertex_degree", "13"),
- 546, 0x0000000000057720L);
+ expectedChecksum(getEchoGraphParamters("hash"), 546, 0x0000000000057720L);
}
@Test
public void testPrintWithEchoGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("EchoGraph", "print", "--vertex_count", "42", "--vertex_degree", "13"),
- new Checksum(546, 0x000000f7190b8fcaL));
+ expectedOutputChecksum(getEchoGraphParamters("print"), new Checksum(546, 0x000000f7190b8fcaL));
+ }
+
+ @Test
+ public void testParallelismWithEchoGraph() throws Exception {
+ TestUtils.verifyParallelism(getEchoGraphParamters("print"));
+ }
+
+ // EmptyGraph
+
+ private String[] getEmptyGraphParamters(String output) {
+ return parameters("EmptyGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithEmptyGraph() throws Exception {
- expectedChecksum(
- parameters("EmptyGraph", "hash", "--vertex_count", "42"),
- 0, 0x0000000000000000L);
+ expectedChecksum(getEmptyGraphParamters("hash"), 0, 0x0000000000000000L);
+ }
+
+ @Test
+ public void testPrintWithEmptyGraph() throws Exception {
+ expectedOutputChecksum(getEmptyGraphParamters("print"), new Checksum(0, 0x0000000000000000L));
+ }
+
+ @Test
+ public void testParallelismWithEmptyGraph() throws Exception {
+ TestUtils.verifyParallelism(getEmptyGraphParamters("print"));
+ }
+
+ // GridGraph
+
+ private String[] getGridGraphParamters(String output) {
+ return parameters("GridGraph", output, "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true");
}
@Test
public void testHashWithGridGraph() throws Exception {
- expectedChecksum(
- parameters("GridGraph", "hash", "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true"),
- 130, 0x000000000000eba0L);
+ expectedChecksum(getGridGraphParamters("hash"), 130, 0x000000000000eba0L);
}
@Test
public void testPrintWithGridGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("GridGraph", "print", "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true"),
- new Checksum(130, 0x00000033237d24eeL));
+ expectedOutputChecksum(getGridGraphParamters("print"), new Checksum(130, 0x00000033237d24eeL));
+ }
+
+ @Test
+ public void testParallelismWithGridGraph() throws Exception {
+ TestUtils.verifyParallelism(getGridGraphParamters("print"));
+ }
+
+ // HypercubeGraph
+
+ private String[] getHypercubeGraphParamters(String output) {
+ return parameters("HypercubeGraph", output, "--dimensions", "7");
}
@Test
public void testHashWithHypercubeGraph() throws Exception {
- expectedChecksum(
- parameters("HypercubeGraph", "hash", "--dimensions", "7"),
- 896, 0x00000000001bc800L);
+ expectedChecksum(getHypercubeGraphParamters("hash"), 896, 0x00000000001bc800L);
}
@Test
public void testPrintWithHypercubeGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("HypercubeGraph", "print", "--dimensions", "7"),
- new Checksum(896, 0x000001f243ee33b2L));
+ expectedOutputChecksum(getHypercubeGraphParamters("print"), new Checksum(896, 0x000001f243ee33b2L));
+ }
+
+ @Test
+ public void testParallelismWithHypercubeGraph() throws Exception {
+ TestUtils.verifyParallelism(getHypercubeGraphParamters("print"));
+ }
+
+ // PathGraph
+
+ private String[] getPathGraphParamters(String output) {
+ return parameters("PathGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithPathGraph() throws Exception {
- expectedChecksum(
- parameters("PathGraph", "hash", "--vertex_count", "42"),
- 82, 0x000000000000d220L);
+ expectedChecksum(getPathGraphParamters("hash"), 82, 0x000000000000d220L);
}
@Test
public void testPrintWithPathGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("PathGraph", "print", "--vertex_count", "42"),
- new Checksum(82, 0x000000269be2d4c2L));
+ expectedOutputChecksum(getPathGraphParamters("print"), new Checksum(82, 0x000000269be2d4c2L));
+ }
+
+ @Test
+ public void testParallelismWithPathGraph() throws Exception {
+ TestUtils.verifyParallelism(getPathGraphParamters("print"));
+ }
+
+ // RMatGraph
+
+ private String[] getRMatGraphParamters(String output, String simplify) {
+ if (simplify == null) {
+ return parameters("RMatGraph", output, "--scale", "7");
+ } else {
+ return parameters("RMatGraph", output, "--scale", "7","--simplify", simplify);
+ }
}
@Test
public void testHashWithRMatGraph() throws Exception {
- expectedChecksum(
- parameters("RMatGraph", "hash", "--scale", "7"),
- 2048, 0x00000000001ee529);
+ expectedChecksum(getRMatGraphParamters("hash", null), 2048, 0x00000000001ee529);
}
@Test
public void testPrintWithRMatGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("RMatGraph", "print", "--scale", "7"),
- new Checksum(2048, 0x000002f737939f05L));
+ expectedOutputChecksum(getRMatGraphParamters("print", null), new Checksum(2048, 0x000002f737939f05L));
+ }
+
+ @Test
+ public void testParallelismWithRMatGraph() throws Exception {
+ TestUtils.verifyParallelism(getRMatGraphParamters("print", null));
}
@Test
public void testHashWithDirectedRMatGraph() throws Exception {
- expectedChecksum(
- parameters("RMatGraph", "hash", "--scale", "7", "--simplify", "directed"),
- 1168, 0x00000000001579bdL);
+ expectedChecksum(getRMatGraphParamters("hash", "directed"), 1168, 0x00000000001579bdL);
}
@Test
public void testPrintWithDirectedRMatGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("RMatGraph", "print", "--scale", "7", "--simplify", "directed"),
- new Checksum(1168, 0x0000020e35b0f35dL));
+ expectedOutputChecksum(getRMatGraphParamters("print", "directed"), new Checksum(1168, 0x0000020e35b0f35dL));
+ }
+
+ @Test
+ public void testParallelismWithDirectedRMatGraph() throws Exception {
+ TestUtils.verifyParallelism(getRMatGraphParamters("print", "directed"));
}
@Test
public void testHashWithUndirectedRMatGraph() throws Exception {
- expectedChecksum(
- parameters("RMatGraph", "hash", "--scale", "7", "--simplify", "undirected"),
- 1854, 0x0000000000242920L);
+ expectedChecksum(getRMatGraphParamters("hash", "undirected"), 1854, 0x0000000000242920L);
}
@Test
public void testPrintWithUndirectedRMatGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("RMatGraph", "print", "--scale", "7", "--simplify", "undirected"),
- new Checksum(1854, 0x0000036fe5802162L));
+ expectedOutputChecksum(getRMatGraphParamters("print", "undirected"), new Checksum(1854, 0x0000036fe5802162L));
+ }
+
+ @Test
+ public void testParallelismWithUndirectedRMatGraph() throws Exception {
+ TestUtils.verifyParallelism(getRMatGraphParamters("print", "undirected"));
+ }
+
+ // SingletonEdgeGraph
+
+ private String[] getSingletonEdgeGraphParamters(String output) {
--- End diff --
`getSingletonEdgeGraphParameters`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r129231234
--- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/drivers/parameter/Simplify.java ---
@@ -109,21 +109,23 @@ public Ordering getValue() {
* @return output graph
* @throws Exception on error
*/
- public <T extends Comparable<T>> Graph<T, NullValue, NullValue> simplify(Graph<T, NullValue, NullValue> graph)
+ public <T extends Comparable<T>> Graph<T, NullValue, NullValue> simplify(Graph<T, NullValue, NullValue> graph, int parallelism)
throws Exception {
-
switch (value) {
case DIRECTED:
graph = graph
- .run(new org.apache.flink.graph.asm.simple.directed.Simplify<T, NullValue, NullValue>());
+ .run(new org.apache.flink.graph.asm.simple.directed.Simplify<T, NullValue, NullValue>()
+ .setParallelism(parallelism));
--- End diff --
the indention of the second call should be the same as the first one
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r132401256
--- Diff: flink-libraries/flink-gelly-examples/src/test/java/org/apache/flink/graph/drivers/EdgeListITCase.java ---
@@ -56,214 +56,304 @@ public void testLongDescription() throws Exception {
ProgramParametrizationException.class);
}
+ // CirculantGraph
+
+ private String[] getCirculantGraphParamters(String output) {
+ return parameters("CirculantGraph", output, "--vertex_count", "42", "--range0", "13:4");
+ }
+
@Test
public void testHashWithCirculantGraph() throws Exception {
- expectedChecksum(
- parameters("CirculantGraph", "hash", "--vertex_count", "42", "--range0", "13:4"),
- 168, 0x000000000001ae80);
+ expectedChecksum(getCirculantGraphParamters("hash"), 168, 0x000000000001ae80);
}
@Test
public void testPrintWithCirculantGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CirculantGraph", "print", "--vertex_count", "42", "--range0", "13:4"),
- new Checksum(168, 0x0000004bdcc52cbcL));
+ expectedOutputChecksum(getCirculantGraphParamters("print"), new Checksum(168, 0x0000004bdcc52cbcL));
+ }
+
+ @Test
+ public void testParallelismWithCirculantGraph() throws Exception {
+ TestUtils.verifyParallelism(getCirculantGraphParamters("print"));
+ }
+
+ // CompleteGraph
+
+ private String[] getCompleteGraphParamters(String output) {
+ return parameters("CompleteGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCompleteGraph() throws Exception {
- expectedChecksum(
- parameters("CompleteGraph", "hash", "--vertex_count", "42"),
- 1722, 0x0000000000113ca0L);
+ expectedChecksum(getCompleteGraphParamters("hash"), 1722, 0x0000000000113ca0L);
}
@Test
public void testPrintWithCompleteGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CompleteGraph", "print", "--vertex_count", "42"),
- new Checksum(1722, 0x0000031109a0c398L));
+ expectedOutputChecksum(getCompleteGraphParamters("print"), new Checksum(1722, 0x0000031109a0c398L));
+ }
+
+ @Test
+ public void testParallelismWithCompleteGraph() throws Exception {
+ TestUtils.verifyParallelism(getCompleteGraphParamters("print"));
+ }
+
+ // CycleGraph
+
+ private String[] getCycleGraphParamters(String output) {
+ return parameters("CycleGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCycleGraph() throws Exception {
- expectedChecksum(
- parameters("CycleGraph", "hash", "--vertex_count", "42"),
- 84, 0x000000000000d740L);
+ expectedChecksum(getCycleGraphParamters("hash"), 84, 0x000000000000d740L);
}
@Test
public void testPrintWithCycleGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CycleGraph", "print", "--vertex_count", "42"),
- new Checksum(84, 0x000000272a136fcaL));
+ expectedOutputChecksum(getCycleGraphParamters("print"), new Checksum(84, 0x000000272a136fcaL));
+ }
+
+ @Test
+ public void testParallelismWithCycleGraph() throws Exception {
+ TestUtils.verifyParallelism(getCycleGraphParamters("print"));
+ }
+
+ // EchoGraph
+
+ private String[] getEchoGraphParamters(String output) {
+ return parameters("EchoGraph", output, "--vertex_count", "42", "--vertex_degree", "13");
}
@Test
public void testHashWithEchoGraph() throws Exception {
- expectedChecksum(
- parameters("EchoGraph", "hash", "--vertex_count", "42", "--vertex_degree", "13"),
- 546, 0x0000000000057720L);
+ expectedChecksum(getEchoGraphParamters("hash"), 546, 0x0000000000057720L);
}
@Test
public void testPrintWithEchoGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("EchoGraph", "print", "--vertex_count", "42", "--vertex_degree", "13"),
- new Checksum(546, 0x000000f7190b8fcaL));
+ expectedOutputChecksum(getEchoGraphParamters("print"), new Checksum(546, 0x000000f7190b8fcaL));
+ }
+
+ @Test
+ public void testParallelismWithEchoGraph() throws Exception {
+ TestUtils.verifyParallelism(getEchoGraphParamters("print"));
+ }
+
+ // EmptyGraph
+
+ private String[] getEmptyGraphParamters(String output) {
+ return parameters("EmptyGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithEmptyGraph() throws Exception {
- expectedChecksum(
- parameters("EmptyGraph", "hash", "--vertex_count", "42"),
- 0, 0x0000000000000000L);
+ expectedChecksum(getEmptyGraphParamters("hash"), 0, 0x0000000000000000L);
+ }
+
+ @Test
+ public void testPrintWithEmptyGraph() throws Exception {
+ expectedOutputChecksum(getEmptyGraphParamters("print"), new Checksum(0, 0x0000000000000000L));
+ }
+
+ @Test
+ public void testParallelismWithEmptyGraph() throws Exception {
+ TestUtils.verifyParallelism(getEmptyGraphParamters("print"));
+ }
+
+ // GridGraph
+
+ private String[] getGridGraphParamters(String output) {
+ return parameters("GridGraph", output, "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true");
}
@Test
public void testHashWithGridGraph() throws Exception {
- expectedChecksum(
- parameters("GridGraph", "hash", "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true"),
- 130, 0x000000000000eba0L);
+ expectedChecksum(getGridGraphParamters("hash"), 130, 0x000000000000eba0L);
}
@Test
public void testPrintWithGridGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("GridGraph", "print", "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true"),
- new Checksum(130, 0x00000033237d24eeL));
+ expectedOutputChecksum(getGridGraphParamters("print"), new Checksum(130, 0x00000033237d24eeL));
+ }
+
+ @Test
+ public void testParallelismWithGridGraph() throws Exception {
+ TestUtils.verifyParallelism(getGridGraphParamters("print"));
+ }
+
+ // HypercubeGraph
+
+ private String[] getHypercubeGraphParamters(String output) {
+ return parameters("HypercubeGraph", output, "--dimensions", "7");
}
@Test
public void testHashWithHypercubeGraph() throws Exception {
- expectedChecksum(
- parameters("HypercubeGraph", "hash", "--dimensions", "7"),
- 896, 0x00000000001bc800L);
+ expectedChecksum(getHypercubeGraphParamters("hash"), 896, 0x00000000001bc800L);
}
@Test
public void testPrintWithHypercubeGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("HypercubeGraph", "print", "--dimensions", "7"),
- new Checksum(896, 0x000001f243ee33b2L));
+ expectedOutputChecksum(getHypercubeGraphParamters("print"), new Checksum(896, 0x000001f243ee33b2L));
+ }
+
+ @Test
+ public void testParallelismWithHypercubeGraph() throws Exception {
+ TestUtils.verifyParallelism(getHypercubeGraphParamters("print"));
+ }
+
+ // PathGraph
+
+ private String[] getPathGraphParamters(String output) {
+ return parameters("PathGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithPathGraph() throws Exception {
- expectedChecksum(
- parameters("PathGraph", "hash", "--vertex_count", "42"),
- 82, 0x000000000000d220L);
+ expectedChecksum(getPathGraphParamters("hash"), 82, 0x000000000000d220L);
}
@Test
public void testPrintWithPathGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("PathGraph", "print", "--vertex_count", "42"),
- new Checksum(82, 0x000000269be2d4c2L));
+ expectedOutputChecksum(getPathGraphParamters("print"), new Checksum(82, 0x000000269be2d4c2L));
+ }
+
+ @Test
+ public void testParallelismWithPathGraph() throws Exception {
+ TestUtils.verifyParallelism(getPathGraphParamters("print"));
+ }
+
+ // RMatGraph
+
+ private String[] getRMatGraphParamters(String output, String simplify) {
--- End diff --
`getRMatGraphParameters`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4346: [FLINK-7199] [gelly] Graph simplification does not set pa...
Posted by greghogan <gi...@git.apache.org>.
Github user greghogan commented on the issue:
https://github.com/apache/flink/pull/4346
@NicoK, parallelism tests have been added. There is a small refactor for `Runner` which needs more extensive rework elsewhere.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by greghogan <gi...@git.apache.org>.
Github user greghogan commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r132427211
--- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/drivers/parameter/LongParameter.java ---
@@ -52,16 +52,6 @@ public LongParameter(ParameterizedBase owner, String name) {
public LongParameter setDefaultValue(long defaultValue) {
super.setDefaultValue(defaultValue);
- if (hasMinimumValue) {
- Util.checkParameter(defaultValue >= minimumValue,
- "Default value (" + defaultValue + ") must be greater than or equal to minimum (" + minimumValue + ")");
- }
-
- if (hasMaximumValue) {
- Util.checkParameter(defaultValue <= maximumValue,
- "Default value (" + defaultValue + ") must be less than or equal to maximum (" + maximumValue + ")");
- }
-
--- End diff --
Right, `ExecutionConfig.DEFAULT_PARALLELISM` is currently `-1`. We're still checking the user configuration within the min and max bounds, it's only when no value is given that the default is used and now permitted to lie outside of the bounds.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r132401061
--- Diff: flink-libraries/flink-gelly-examples/src/test/java/org/apache/flink/graph/drivers/EdgeListITCase.java ---
@@ -56,214 +56,304 @@ public void testLongDescription() throws Exception {
ProgramParametrizationException.class);
}
+ // CirculantGraph
+
+ private String[] getCirculantGraphParamters(String output) {
+ return parameters("CirculantGraph", output, "--vertex_count", "42", "--range0", "13:4");
+ }
+
@Test
public void testHashWithCirculantGraph() throws Exception {
- expectedChecksum(
- parameters("CirculantGraph", "hash", "--vertex_count", "42", "--range0", "13:4"),
- 168, 0x000000000001ae80);
+ expectedChecksum(getCirculantGraphParamters("hash"), 168, 0x000000000001ae80);
}
@Test
public void testPrintWithCirculantGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CirculantGraph", "print", "--vertex_count", "42", "--range0", "13:4"),
- new Checksum(168, 0x0000004bdcc52cbcL));
+ expectedOutputChecksum(getCirculantGraphParamters("print"), new Checksum(168, 0x0000004bdcc52cbcL));
+ }
+
+ @Test
+ public void testParallelismWithCirculantGraph() throws Exception {
+ TestUtils.verifyParallelism(getCirculantGraphParamters("print"));
+ }
+
+ // CompleteGraph
+
+ private String[] getCompleteGraphParamters(String output) {
+ return parameters("CompleteGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCompleteGraph() throws Exception {
- expectedChecksum(
- parameters("CompleteGraph", "hash", "--vertex_count", "42"),
- 1722, 0x0000000000113ca0L);
+ expectedChecksum(getCompleteGraphParamters("hash"), 1722, 0x0000000000113ca0L);
}
@Test
public void testPrintWithCompleteGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CompleteGraph", "print", "--vertex_count", "42"),
- new Checksum(1722, 0x0000031109a0c398L));
+ expectedOutputChecksum(getCompleteGraphParamters("print"), new Checksum(1722, 0x0000031109a0c398L));
+ }
+
+ @Test
+ public void testParallelismWithCompleteGraph() throws Exception {
+ TestUtils.verifyParallelism(getCompleteGraphParamters("print"));
+ }
+
+ // CycleGraph
+
+ private String[] getCycleGraphParamters(String output) {
+ return parameters("CycleGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCycleGraph() throws Exception {
- expectedChecksum(
- parameters("CycleGraph", "hash", "--vertex_count", "42"),
- 84, 0x000000000000d740L);
+ expectedChecksum(getCycleGraphParamters("hash"), 84, 0x000000000000d740L);
}
@Test
public void testPrintWithCycleGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CycleGraph", "print", "--vertex_count", "42"),
- new Checksum(84, 0x000000272a136fcaL));
+ expectedOutputChecksum(getCycleGraphParamters("print"), new Checksum(84, 0x000000272a136fcaL));
+ }
+
+ @Test
+ public void testParallelismWithCycleGraph() throws Exception {
+ TestUtils.verifyParallelism(getCycleGraphParamters("print"));
+ }
+
+ // EchoGraph
+
+ private String[] getEchoGraphParamters(String output) {
+ return parameters("EchoGraph", output, "--vertex_count", "42", "--vertex_degree", "13");
}
@Test
public void testHashWithEchoGraph() throws Exception {
- expectedChecksum(
- parameters("EchoGraph", "hash", "--vertex_count", "42", "--vertex_degree", "13"),
- 546, 0x0000000000057720L);
+ expectedChecksum(getEchoGraphParamters("hash"), 546, 0x0000000000057720L);
}
@Test
public void testPrintWithEchoGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("EchoGraph", "print", "--vertex_count", "42", "--vertex_degree", "13"),
- new Checksum(546, 0x000000f7190b8fcaL));
+ expectedOutputChecksum(getEchoGraphParamters("print"), new Checksum(546, 0x000000f7190b8fcaL));
+ }
+
+ @Test
+ public void testParallelismWithEchoGraph() throws Exception {
+ TestUtils.verifyParallelism(getEchoGraphParamters("print"));
+ }
+
+ // EmptyGraph
+
+ private String[] getEmptyGraphParamters(String output) {
+ return parameters("EmptyGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithEmptyGraph() throws Exception {
- expectedChecksum(
- parameters("EmptyGraph", "hash", "--vertex_count", "42"),
- 0, 0x0000000000000000L);
+ expectedChecksum(getEmptyGraphParamters("hash"), 0, 0x0000000000000000L);
+ }
+
+ @Test
+ public void testPrintWithEmptyGraph() throws Exception {
+ expectedOutputChecksum(getEmptyGraphParamters("print"), new Checksum(0, 0x0000000000000000L));
+ }
+
+ @Test
+ public void testParallelismWithEmptyGraph() throws Exception {
+ TestUtils.verifyParallelism(getEmptyGraphParamters("print"));
+ }
+
+ // GridGraph
+
+ private String[] getGridGraphParamters(String output) {
--- End diff --
one more: `getGridGraphParameters`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/4346
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r132402963
--- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/drivers/parameter/LongParameter.java ---
@@ -52,16 +52,6 @@ public LongParameter(ParameterizedBase owner, String name) {
public LongParameter setDefaultValue(long defaultValue) {
super.setDefaultValue(defaultValue);
- if (hasMinimumValue) {
- Util.checkParameter(defaultValue >= minimumValue,
- "Default value (" + defaultValue + ") must be greater than or equal to minimum (" + minimumValue + ")");
- }
-
- if (hasMaximumValue) {
- Util.checkParameter(defaultValue <= maximumValue,
- "Default value (" + defaultValue + ") must be less than or equal to maximum (" + maximumValue + ")");
- }
-
--- End diff --
I'm no expert here, but can you elaborate on why you removed these checks for the default value being in line with the min and max?
I can only guess that this must be that way in case where the default parallelism is not within the bounds of the configured one. I'm wondering whether the `LongParameter` is used somewhere else were these bounds are expected to hold though.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r132400879
--- Diff: flink-libraries/flink-gelly-examples/src/test/java/org/apache/flink/graph/drivers/EdgeListITCase.java ---
@@ -56,214 +56,304 @@ public void testLongDescription() throws Exception {
ProgramParametrizationException.class);
}
+ // CirculantGraph
+
+ private String[] getCirculantGraphParamters(String output) {
+ return parameters("CirculantGraph", output, "--vertex_count", "42", "--range0", "13:4");
+ }
+
@Test
public void testHashWithCirculantGraph() throws Exception {
- expectedChecksum(
- parameters("CirculantGraph", "hash", "--vertex_count", "42", "--range0", "13:4"),
- 168, 0x000000000001ae80);
+ expectedChecksum(getCirculantGraphParamters("hash"), 168, 0x000000000001ae80);
}
@Test
public void testPrintWithCirculantGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CirculantGraph", "print", "--vertex_count", "42", "--range0", "13:4"),
- new Checksum(168, 0x0000004bdcc52cbcL));
+ expectedOutputChecksum(getCirculantGraphParamters("print"), new Checksum(168, 0x0000004bdcc52cbcL));
+ }
+
+ @Test
+ public void testParallelismWithCirculantGraph() throws Exception {
+ TestUtils.verifyParallelism(getCirculantGraphParamters("print"));
+ }
+
+ // CompleteGraph
+
+ private String[] getCompleteGraphParamters(String output) {
+ return parameters("CompleteGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCompleteGraph() throws Exception {
- expectedChecksum(
- parameters("CompleteGraph", "hash", "--vertex_count", "42"),
- 1722, 0x0000000000113ca0L);
+ expectedChecksum(getCompleteGraphParamters("hash"), 1722, 0x0000000000113ca0L);
}
@Test
public void testPrintWithCompleteGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CompleteGraph", "print", "--vertex_count", "42"),
- new Checksum(1722, 0x0000031109a0c398L));
+ expectedOutputChecksum(getCompleteGraphParamters("print"), new Checksum(1722, 0x0000031109a0c398L));
+ }
+
+ @Test
+ public void testParallelismWithCompleteGraph() throws Exception {
+ TestUtils.verifyParallelism(getCompleteGraphParamters("print"));
+ }
+
+ // CycleGraph
+
+ private String[] getCycleGraphParamters(String output) {
+ return parameters("CycleGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCycleGraph() throws Exception {
- expectedChecksum(
- parameters("CycleGraph", "hash", "--vertex_count", "42"),
- 84, 0x000000000000d740L);
+ expectedChecksum(getCycleGraphParamters("hash"), 84, 0x000000000000d740L);
}
@Test
public void testPrintWithCycleGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CycleGraph", "print", "--vertex_count", "42"),
- new Checksum(84, 0x000000272a136fcaL));
+ expectedOutputChecksum(getCycleGraphParamters("print"), new Checksum(84, 0x000000272a136fcaL));
+ }
+
+ @Test
+ public void testParallelismWithCycleGraph() throws Exception {
+ TestUtils.verifyParallelism(getCycleGraphParamters("print"));
+ }
+
+ // EchoGraph
+
+ private String[] getEchoGraphParamters(String output) {
--- End diff --
same here: `getEchoGraphParameters`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by greghogan <gi...@git.apache.org>.
Github user greghogan commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r129286498
--- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/drivers/parameter/Simplify.java ---
@@ -109,21 +109,23 @@ public Ordering getValue() {
* @return output graph
* @throws Exception on error
*/
- public <T extends Comparable<T>> Graph<T, NullValue, NullValue> simplify(Graph<T, NullValue, NullValue> graph)
+ public <T extends Comparable<T>> Graph<T, NullValue, NullValue> simplify(Graph<T, NullValue, NullValue> graph, int parallelism)
throws Exception {
-
switch (value) {
case DIRECTED:
graph = graph
- .run(new org.apache.flink.graph.asm.simple.directed.Simplify<T, NullValue, NullValue>());
+ .run(new org.apache.flink.graph.asm.simple.directed.Simplify<T, NullValue, NullValue>()
+ .setParallelism(parallelism));
--- End diff --
This is consistent throughout Gelly as we often chain operators.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r129231293
--- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/drivers/parameter/Simplify.java ---
@@ -109,21 +109,23 @@ public Ordering getValue() {
* @return output graph
* @throws Exception on error
*/
- public <T extends Comparable<T>> Graph<T, NullValue, NullValue> simplify(Graph<T, NullValue, NullValue> graph)
+ public <T extends Comparable<T>> Graph<T, NullValue, NullValue> simplify(Graph<T, NullValue, NullValue> graph, int parallelism)
throws Exception {
-
switch (value) {
case DIRECTED:
graph = graph
- .run(new org.apache.flink.graph.asm.simple.directed.Simplify<T, NullValue, NullValue>());
+ .run(new org.apache.flink.graph.asm.simple.directed.Simplify<T, NullValue, NullValue>()
+ .setParallelism(parallelism));
break;
case UNDIRECTED:
graph = graph
- .run(new org.apache.flink.graph.asm.simple.undirected.Simplify<T, NullValue, NullValue>(false));
+ .run(new org.apache.flink.graph.asm.simple.undirected.Simplify<T, NullValue, NullValue>(false)
+ .setParallelism(parallelism));
break;
case UNDIRECTED_CLIP_AND_FLIP:
graph = graph
- .run(new org.apache.flink.graph.asm.simple.undirected.Simplify<T, NullValue, NullValue>(true));
+ .run(new org.apache.flink.graph.asm.simple.undirected.Simplify<T, NullValue, NullValue>(true)
+ .setParallelism(parallelism));
--- End diff --
...and here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by greghogan <gi...@git.apache.org>.
Github user greghogan commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r129283124
--- Diff: flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/library/linkanalysis/HITS.java ---
@@ -168,7 +168,6 @@ protected void mergeConfiguration(GraphAlgorithmWrappingBase other) {
.setParallelism(parallelism)
.name("Square")
.reduce(new Sum())
- .setCombineHint(CombineHint.HASH)
--- End diff --
I had updated the commit message which was not reflected in the pull request message. I've updated the latter.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r132401474
--- Diff: flink-libraries/flink-gelly-examples/src/test/java/org/apache/flink/graph/drivers/EdgeListITCase.java ---
@@ -56,214 +56,304 @@ public void testLongDescription() throws Exception {
ProgramParametrizationException.class);
}
+ // CirculantGraph
+
+ private String[] getCirculantGraphParamters(String output) {
+ return parameters("CirculantGraph", output, "--vertex_count", "42", "--range0", "13:4");
+ }
+
@Test
public void testHashWithCirculantGraph() throws Exception {
- expectedChecksum(
- parameters("CirculantGraph", "hash", "--vertex_count", "42", "--range0", "13:4"),
- 168, 0x000000000001ae80);
+ expectedChecksum(getCirculantGraphParamters("hash"), 168, 0x000000000001ae80);
}
@Test
public void testPrintWithCirculantGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CirculantGraph", "print", "--vertex_count", "42", "--range0", "13:4"),
- new Checksum(168, 0x0000004bdcc52cbcL));
+ expectedOutputChecksum(getCirculantGraphParamters("print"), new Checksum(168, 0x0000004bdcc52cbcL));
+ }
+
+ @Test
+ public void testParallelismWithCirculantGraph() throws Exception {
+ TestUtils.verifyParallelism(getCirculantGraphParamters("print"));
+ }
+
+ // CompleteGraph
+
+ private String[] getCompleteGraphParamters(String output) {
+ return parameters("CompleteGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCompleteGraph() throws Exception {
- expectedChecksum(
- parameters("CompleteGraph", "hash", "--vertex_count", "42"),
- 1722, 0x0000000000113ca0L);
+ expectedChecksum(getCompleteGraphParamters("hash"), 1722, 0x0000000000113ca0L);
}
@Test
public void testPrintWithCompleteGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CompleteGraph", "print", "--vertex_count", "42"),
- new Checksum(1722, 0x0000031109a0c398L));
+ expectedOutputChecksum(getCompleteGraphParamters("print"), new Checksum(1722, 0x0000031109a0c398L));
+ }
+
+ @Test
+ public void testParallelismWithCompleteGraph() throws Exception {
+ TestUtils.verifyParallelism(getCompleteGraphParamters("print"));
+ }
+
+ // CycleGraph
+
+ private String[] getCycleGraphParamters(String output) {
+ return parameters("CycleGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCycleGraph() throws Exception {
- expectedChecksum(
- parameters("CycleGraph", "hash", "--vertex_count", "42"),
- 84, 0x000000000000d740L);
+ expectedChecksum(getCycleGraphParamters("hash"), 84, 0x000000000000d740L);
}
@Test
public void testPrintWithCycleGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CycleGraph", "print", "--vertex_count", "42"),
- new Checksum(84, 0x000000272a136fcaL));
+ expectedOutputChecksum(getCycleGraphParamters("print"), new Checksum(84, 0x000000272a136fcaL));
+ }
+
+ @Test
+ public void testParallelismWithCycleGraph() throws Exception {
+ TestUtils.verifyParallelism(getCycleGraphParamters("print"));
+ }
+
+ // EchoGraph
+
+ private String[] getEchoGraphParamters(String output) {
+ return parameters("EchoGraph", output, "--vertex_count", "42", "--vertex_degree", "13");
}
@Test
public void testHashWithEchoGraph() throws Exception {
- expectedChecksum(
- parameters("EchoGraph", "hash", "--vertex_count", "42", "--vertex_degree", "13"),
- 546, 0x0000000000057720L);
+ expectedChecksum(getEchoGraphParamters("hash"), 546, 0x0000000000057720L);
}
@Test
public void testPrintWithEchoGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("EchoGraph", "print", "--vertex_count", "42", "--vertex_degree", "13"),
- new Checksum(546, 0x000000f7190b8fcaL));
+ expectedOutputChecksum(getEchoGraphParamters("print"), new Checksum(546, 0x000000f7190b8fcaL));
+ }
+
+ @Test
+ public void testParallelismWithEchoGraph() throws Exception {
+ TestUtils.verifyParallelism(getEchoGraphParamters("print"));
+ }
+
+ // EmptyGraph
+
+ private String[] getEmptyGraphParamters(String output) {
+ return parameters("EmptyGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithEmptyGraph() throws Exception {
- expectedChecksum(
- parameters("EmptyGraph", "hash", "--vertex_count", "42"),
- 0, 0x0000000000000000L);
+ expectedChecksum(getEmptyGraphParamters("hash"), 0, 0x0000000000000000L);
+ }
+
+ @Test
+ public void testPrintWithEmptyGraph() throws Exception {
+ expectedOutputChecksum(getEmptyGraphParamters("print"), new Checksum(0, 0x0000000000000000L));
+ }
+
+ @Test
+ public void testParallelismWithEmptyGraph() throws Exception {
+ TestUtils.verifyParallelism(getEmptyGraphParamters("print"));
+ }
+
+ // GridGraph
+
+ private String[] getGridGraphParamters(String output) {
+ return parameters("GridGraph", output, "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true");
}
@Test
public void testHashWithGridGraph() throws Exception {
- expectedChecksum(
- parameters("GridGraph", "hash", "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true"),
- 130, 0x000000000000eba0L);
+ expectedChecksum(getGridGraphParamters("hash"), 130, 0x000000000000eba0L);
}
@Test
public void testPrintWithGridGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("GridGraph", "print", "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true"),
- new Checksum(130, 0x00000033237d24eeL));
+ expectedOutputChecksum(getGridGraphParamters("print"), new Checksum(130, 0x00000033237d24eeL));
+ }
+
+ @Test
+ public void testParallelismWithGridGraph() throws Exception {
+ TestUtils.verifyParallelism(getGridGraphParamters("print"));
+ }
+
+ // HypercubeGraph
+
+ private String[] getHypercubeGraphParamters(String output) {
+ return parameters("HypercubeGraph", output, "--dimensions", "7");
}
@Test
public void testHashWithHypercubeGraph() throws Exception {
- expectedChecksum(
- parameters("HypercubeGraph", "hash", "--dimensions", "7"),
- 896, 0x00000000001bc800L);
+ expectedChecksum(getHypercubeGraphParamters("hash"), 896, 0x00000000001bc800L);
}
@Test
public void testPrintWithHypercubeGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("HypercubeGraph", "print", "--dimensions", "7"),
- new Checksum(896, 0x000001f243ee33b2L));
+ expectedOutputChecksum(getHypercubeGraphParamters("print"), new Checksum(896, 0x000001f243ee33b2L));
+ }
+
+ @Test
+ public void testParallelismWithHypercubeGraph() throws Exception {
+ TestUtils.verifyParallelism(getHypercubeGraphParamters("print"));
+ }
+
+ // PathGraph
+
+ private String[] getPathGraphParamters(String output) {
+ return parameters("PathGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithPathGraph() throws Exception {
- expectedChecksum(
- parameters("PathGraph", "hash", "--vertex_count", "42"),
- 82, 0x000000000000d220L);
+ expectedChecksum(getPathGraphParamters("hash"), 82, 0x000000000000d220L);
}
@Test
public void testPrintWithPathGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("PathGraph", "print", "--vertex_count", "42"),
- new Checksum(82, 0x000000269be2d4c2L));
+ expectedOutputChecksum(getPathGraphParamters("print"), new Checksum(82, 0x000000269be2d4c2L));
+ }
+
+ @Test
+ public void testParallelismWithPathGraph() throws Exception {
+ TestUtils.verifyParallelism(getPathGraphParamters("print"));
+ }
+
+ // RMatGraph
+
+ private String[] getRMatGraphParamters(String output, String simplify) {
+ if (simplify == null) {
+ return parameters("RMatGraph", output, "--scale", "7");
+ } else {
+ return parameters("RMatGraph", output, "--scale", "7","--simplify", simplify);
+ }
}
@Test
public void testHashWithRMatGraph() throws Exception {
- expectedChecksum(
- parameters("RMatGraph", "hash", "--scale", "7"),
- 2048, 0x00000000001ee529);
+ expectedChecksum(getRMatGraphParamters("hash", null), 2048, 0x00000000001ee529);
}
@Test
public void testPrintWithRMatGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("RMatGraph", "print", "--scale", "7"),
- new Checksum(2048, 0x000002f737939f05L));
+ expectedOutputChecksum(getRMatGraphParamters("print", null), new Checksum(2048, 0x000002f737939f05L));
+ }
+
+ @Test
+ public void testParallelismWithRMatGraph() throws Exception {
+ TestUtils.verifyParallelism(getRMatGraphParamters("print", null));
}
@Test
public void testHashWithDirectedRMatGraph() throws Exception {
- expectedChecksum(
- parameters("RMatGraph", "hash", "--scale", "7", "--simplify", "directed"),
- 1168, 0x00000000001579bdL);
+ expectedChecksum(getRMatGraphParamters("hash", "directed"), 1168, 0x00000000001579bdL);
}
@Test
public void testPrintWithDirectedRMatGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("RMatGraph", "print", "--scale", "7", "--simplify", "directed"),
- new Checksum(1168, 0x0000020e35b0f35dL));
+ expectedOutputChecksum(getRMatGraphParamters("print", "directed"), new Checksum(1168, 0x0000020e35b0f35dL));
+ }
+
+ @Test
+ public void testParallelismWithDirectedRMatGraph() throws Exception {
+ TestUtils.verifyParallelism(getRMatGraphParamters("print", "directed"));
}
@Test
public void testHashWithUndirectedRMatGraph() throws Exception {
- expectedChecksum(
- parameters("RMatGraph", "hash", "--scale", "7", "--simplify", "undirected"),
- 1854, 0x0000000000242920L);
+ expectedChecksum(getRMatGraphParamters("hash", "undirected"), 1854, 0x0000000000242920L);
}
@Test
public void testPrintWithUndirectedRMatGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("RMatGraph", "print", "--scale", "7", "--simplify", "undirected"),
- new Checksum(1854, 0x0000036fe5802162L));
+ expectedOutputChecksum(getRMatGraphParamters("print", "undirected"), new Checksum(1854, 0x0000036fe5802162L));
+ }
+
+ @Test
+ public void testParallelismWithUndirectedRMatGraph() throws Exception {
+ TestUtils.verifyParallelism(getRMatGraphParamters("print", "undirected"));
+ }
+
+ // SingletonEdgeGraph
+
+ private String[] getSingletonEdgeGraphParamters(String output) {
+ return parameters("SingletonEdgeGraph", output, "--vertex_pair_count", "42");
}
@Test
public void testHashWithSingletonEdgeGraph() throws Exception {
- expectedChecksum(
- parameters("SingletonEdgeGraph", "hash", "--vertex_pair_count", "42"),
- 84, 0x000000000001b3c0L);
+ expectedChecksum(getSingletonEdgeGraphParamters("hash"), 84, 0x000000000001b3c0L);
}
@Test
public void testPrintWithSingletonEdgeGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("SingletonEdgeGraph", "print", "--vertex_pair_count", "42"),
- new Checksum(84, 0x0000002e59e10d9aL));
+ expectedOutputChecksum(getSingletonEdgeGraphParamters("print"), new Checksum(84, 0x0000002e59e10d9aL));
+ }
+
+ @Test
+ public void testParallelismWithSingletonEdgeGraph() throws Exception {
+ TestUtils.verifyParallelism(getSingletonEdgeGraphParamters("print"));
+ }
+
+ // StarGraph
+
+ private String[] getStarGraphParamters(String output) {
--- End diff --
`getStarGraphParameters`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r129295984
--- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/drivers/input/InputBase.java ---
@@ -32,6 +35,9 @@
extends ParameterizedBase
implements Input<K, VV, EV> {
+ protected LongParameter parallelism = new LongParameter(this, "__parallelism")
+ .setDefaultValue(PARALLELISM_DEFAULT);
--- End diff --
thanks for the explanation
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r132400996
--- Diff: flink-libraries/flink-gelly-examples/src/test/java/org/apache/flink/graph/drivers/EdgeListITCase.java ---
@@ -56,214 +56,304 @@ public void testLongDescription() throws Exception {
ProgramParametrizationException.class);
}
+ // CirculantGraph
+
+ private String[] getCirculantGraphParamters(String output) {
+ return parameters("CirculantGraph", output, "--vertex_count", "42", "--range0", "13:4");
+ }
+
@Test
public void testHashWithCirculantGraph() throws Exception {
- expectedChecksum(
- parameters("CirculantGraph", "hash", "--vertex_count", "42", "--range0", "13:4"),
- 168, 0x000000000001ae80);
+ expectedChecksum(getCirculantGraphParamters("hash"), 168, 0x000000000001ae80);
}
@Test
public void testPrintWithCirculantGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CirculantGraph", "print", "--vertex_count", "42", "--range0", "13:4"),
- new Checksum(168, 0x0000004bdcc52cbcL));
+ expectedOutputChecksum(getCirculantGraphParamters("print"), new Checksum(168, 0x0000004bdcc52cbcL));
+ }
+
+ @Test
+ public void testParallelismWithCirculantGraph() throws Exception {
+ TestUtils.verifyParallelism(getCirculantGraphParamters("print"));
+ }
+
+ // CompleteGraph
+
+ private String[] getCompleteGraphParamters(String output) {
+ return parameters("CompleteGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCompleteGraph() throws Exception {
- expectedChecksum(
- parameters("CompleteGraph", "hash", "--vertex_count", "42"),
- 1722, 0x0000000000113ca0L);
+ expectedChecksum(getCompleteGraphParamters("hash"), 1722, 0x0000000000113ca0L);
}
@Test
public void testPrintWithCompleteGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CompleteGraph", "print", "--vertex_count", "42"),
- new Checksum(1722, 0x0000031109a0c398L));
+ expectedOutputChecksum(getCompleteGraphParamters("print"), new Checksum(1722, 0x0000031109a0c398L));
+ }
+
+ @Test
+ public void testParallelismWithCompleteGraph() throws Exception {
+ TestUtils.verifyParallelism(getCompleteGraphParamters("print"));
+ }
+
+ // CycleGraph
+
+ private String[] getCycleGraphParamters(String output) {
+ return parameters("CycleGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCycleGraph() throws Exception {
- expectedChecksum(
- parameters("CycleGraph", "hash", "--vertex_count", "42"),
- 84, 0x000000000000d740L);
+ expectedChecksum(getCycleGraphParamters("hash"), 84, 0x000000000000d740L);
}
@Test
public void testPrintWithCycleGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CycleGraph", "print", "--vertex_count", "42"),
- new Checksum(84, 0x000000272a136fcaL));
+ expectedOutputChecksum(getCycleGraphParamters("print"), new Checksum(84, 0x000000272a136fcaL));
+ }
+
+ @Test
+ public void testParallelismWithCycleGraph() throws Exception {
+ TestUtils.verifyParallelism(getCycleGraphParamters("print"));
+ }
+
+ // EchoGraph
+
+ private String[] getEchoGraphParamters(String output) {
+ return parameters("EchoGraph", output, "--vertex_count", "42", "--vertex_degree", "13");
}
@Test
public void testHashWithEchoGraph() throws Exception {
- expectedChecksum(
- parameters("EchoGraph", "hash", "--vertex_count", "42", "--vertex_degree", "13"),
- 546, 0x0000000000057720L);
+ expectedChecksum(getEchoGraphParamters("hash"), 546, 0x0000000000057720L);
}
@Test
public void testPrintWithEchoGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("EchoGraph", "print", "--vertex_count", "42", "--vertex_degree", "13"),
- new Checksum(546, 0x000000f7190b8fcaL));
+ expectedOutputChecksum(getEchoGraphParamters("print"), new Checksum(546, 0x000000f7190b8fcaL));
+ }
+
+ @Test
+ public void testParallelismWithEchoGraph() throws Exception {
+ TestUtils.verifyParallelism(getEchoGraphParamters("print"));
+ }
+
+ // EmptyGraph
+
+ private String[] getEmptyGraphParamters(String output) {
--- End diff --
and here: `getEmptyGraphParameters`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r132401144
--- Diff: flink-libraries/flink-gelly-examples/src/test/java/org/apache/flink/graph/drivers/EdgeListITCase.java ---
@@ -56,214 +56,304 @@ public void testLongDescription() throws Exception {
ProgramParametrizationException.class);
}
+ // CirculantGraph
+
+ private String[] getCirculantGraphParamters(String output) {
+ return parameters("CirculantGraph", output, "--vertex_count", "42", "--range0", "13:4");
+ }
+
@Test
public void testHashWithCirculantGraph() throws Exception {
- expectedChecksum(
- parameters("CirculantGraph", "hash", "--vertex_count", "42", "--range0", "13:4"),
- 168, 0x000000000001ae80);
+ expectedChecksum(getCirculantGraphParamters("hash"), 168, 0x000000000001ae80);
}
@Test
public void testPrintWithCirculantGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CirculantGraph", "print", "--vertex_count", "42", "--range0", "13:4"),
- new Checksum(168, 0x0000004bdcc52cbcL));
+ expectedOutputChecksum(getCirculantGraphParamters("print"), new Checksum(168, 0x0000004bdcc52cbcL));
+ }
+
+ @Test
+ public void testParallelismWithCirculantGraph() throws Exception {
+ TestUtils.verifyParallelism(getCirculantGraphParamters("print"));
+ }
+
+ // CompleteGraph
+
+ private String[] getCompleteGraphParamters(String output) {
+ return parameters("CompleteGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCompleteGraph() throws Exception {
- expectedChecksum(
- parameters("CompleteGraph", "hash", "--vertex_count", "42"),
- 1722, 0x0000000000113ca0L);
+ expectedChecksum(getCompleteGraphParamters("hash"), 1722, 0x0000000000113ca0L);
}
@Test
public void testPrintWithCompleteGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CompleteGraph", "print", "--vertex_count", "42"),
- new Checksum(1722, 0x0000031109a0c398L));
+ expectedOutputChecksum(getCompleteGraphParamters("print"), new Checksum(1722, 0x0000031109a0c398L));
+ }
+
+ @Test
+ public void testParallelismWithCompleteGraph() throws Exception {
+ TestUtils.verifyParallelism(getCompleteGraphParamters("print"));
+ }
+
+ // CycleGraph
+
+ private String[] getCycleGraphParamters(String output) {
+ return parameters("CycleGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithCycleGraph() throws Exception {
- expectedChecksum(
- parameters("CycleGraph", "hash", "--vertex_count", "42"),
- 84, 0x000000000000d740L);
+ expectedChecksum(getCycleGraphParamters("hash"), 84, 0x000000000000d740L);
}
@Test
public void testPrintWithCycleGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CycleGraph", "print", "--vertex_count", "42"),
- new Checksum(84, 0x000000272a136fcaL));
+ expectedOutputChecksum(getCycleGraphParamters("print"), new Checksum(84, 0x000000272a136fcaL));
+ }
+
+ @Test
+ public void testParallelismWithCycleGraph() throws Exception {
+ TestUtils.verifyParallelism(getCycleGraphParamters("print"));
+ }
+
+ // EchoGraph
+
+ private String[] getEchoGraphParamters(String output) {
+ return parameters("EchoGraph", output, "--vertex_count", "42", "--vertex_degree", "13");
}
@Test
public void testHashWithEchoGraph() throws Exception {
- expectedChecksum(
- parameters("EchoGraph", "hash", "--vertex_count", "42", "--vertex_degree", "13"),
- 546, 0x0000000000057720L);
+ expectedChecksum(getEchoGraphParamters("hash"), 546, 0x0000000000057720L);
}
@Test
public void testPrintWithEchoGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("EchoGraph", "print", "--vertex_count", "42", "--vertex_degree", "13"),
- new Checksum(546, 0x000000f7190b8fcaL));
+ expectedOutputChecksum(getEchoGraphParamters("print"), new Checksum(546, 0x000000f7190b8fcaL));
+ }
+
+ @Test
+ public void testParallelismWithEchoGraph() throws Exception {
+ TestUtils.verifyParallelism(getEchoGraphParamters("print"));
+ }
+
+ // EmptyGraph
+
+ private String[] getEmptyGraphParamters(String output) {
+ return parameters("EmptyGraph", output, "--vertex_count", "42");
}
@Test
public void testHashWithEmptyGraph() throws Exception {
- expectedChecksum(
- parameters("EmptyGraph", "hash", "--vertex_count", "42"),
- 0, 0x0000000000000000L);
+ expectedChecksum(getEmptyGraphParamters("hash"), 0, 0x0000000000000000L);
+ }
+
+ @Test
+ public void testPrintWithEmptyGraph() throws Exception {
+ expectedOutputChecksum(getEmptyGraphParamters("print"), new Checksum(0, 0x0000000000000000L));
+ }
+
+ @Test
+ public void testParallelismWithEmptyGraph() throws Exception {
+ TestUtils.verifyParallelism(getEmptyGraphParamters("print"));
+ }
+
+ // GridGraph
+
+ private String[] getGridGraphParamters(String output) {
+ return parameters("GridGraph", output, "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true");
}
@Test
public void testHashWithGridGraph() throws Exception {
- expectedChecksum(
- parameters("GridGraph", "hash", "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true"),
- 130, 0x000000000000eba0L);
+ expectedChecksum(getGridGraphParamters("hash"), 130, 0x000000000000eba0L);
}
@Test
public void testPrintWithGridGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("GridGraph", "print", "--dim0", "2:true", "--dim1", "3:false", "--dim2", "5:true"),
- new Checksum(130, 0x00000033237d24eeL));
+ expectedOutputChecksum(getGridGraphParamters("print"), new Checksum(130, 0x00000033237d24eeL));
+ }
+
+ @Test
+ public void testParallelismWithGridGraph() throws Exception {
+ TestUtils.verifyParallelism(getGridGraphParamters("print"));
+ }
+
+ // HypercubeGraph
+
+ private String[] getHypercubeGraphParamters(String output) {
--- End diff --
that typo seems consistent... `getHypercubeGraphParameters`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r129231080
--- Diff: flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/library/linkanalysis/HITS.java ---
@@ -168,7 +168,6 @@ protected void mergeConfiguration(GraphAlgorithmWrappingBase other) {
.setParallelism(parallelism)
.name("Square")
.reduce(new Sum())
- .setCombineHint(CombineHint.HASH)
--- End diff --
this changes doesn't look related to the intention of the commit, or did I miss something?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r132400768
--- Diff: flink-libraries/flink-gelly-examples/src/test/java/org/apache/flink/graph/drivers/EdgeListITCase.java ---
@@ -56,214 +56,304 @@ public void testLongDescription() throws Exception {
ProgramParametrizationException.class);
}
+ // CirculantGraph
+
+ private String[] getCirculantGraphParamters(String output) {
+ return parameters("CirculantGraph", output, "--vertex_count", "42", "--range0", "13:4");
+ }
+
@Test
public void testHashWithCirculantGraph() throws Exception {
- expectedChecksum(
- parameters("CirculantGraph", "hash", "--vertex_count", "42", "--range0", "13:4"),
- 168, 0x000000000001ae80);
+ expectedChecksum(getCirculantGraphParamters("hash"), 168, 0x000000000001ae80);
}
@Test
public void testPrintWithCirculantGraph() throws Exception {
// skip 'char' since it is not printed as a number
Assume.assumeFalse(idType.equals("char") || idType.equals("nativeChar"));
- expectedOutputChecksum(
- parameters("CirculantGraph", "print", "--vertex_count", "42", "--range0", "13:4"),
- new Checksum(168, 0x0000004bdcc52cbcL));
+ expectedOutputChecksum(getCirculantGraphParamters("print"), new Checksum(168, 0x0000004bdcc52cbcL));
+ }
+
+ @Test
+ public void testParallelismWithCirculantGraph() throws Exception {
+ TestUtils.verifyParallelism(getCirculantGraphParamters("print"));
+ }
+
+ // CompleteGraph
+
+ private String[] getCompleteGraphParamters(String output) {
--- End diff --
typo, should be: `getCompleteGraphParameters`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4346: [FLINK-7199] [gelly] Graph simplification does not set pa...
Posted by greghogan <gi...@git.apache.org>.
Github user greghogan commented on the issue:
https://github.com/apache/flink/pull/4346
@NicoK thanks for the review! I'll work on refactoring the `Runner` and adding checks for checking the parallelism. I think we're okay if we whitelist the "large" operators by name.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r132400522
--- Diff: flink-libraries/flink-gelly-examples/src/test/java/org/apache/flink/graph/drivers/EdgeListITCase.java ---
@@ -56,214 +56,304 @@ public void testLongDescription() throws Exception {
ProgramParametrizationException.class);
}
+ // CirculantGraph
+
+ private String[] getCirculantGraphParamters(String output) {
--- End diff --
typo, should be `getCirculantGraphParameters`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r129231099
--- Diff: flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/library/linkanalysis/HITS.java ---
@@ -192,7 +191,6 @@ protected void mergeConfiguration(GraphAlgorithmWrappingBase other) {
.setParallelism(parallelism)
.name("Square")
.reduce(new Sum())
- .setCombineHint(CombineHint.HASH)
--- End diff --
same here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by greghogan <gi...@git.apache.org>.
Github user greghogan commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r129296601
--- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/drivers/input/InputBase.java ---
@@ -32,6 +35,9 @@
extends ParameterizedBase
implements Input<K, VV, EV> {
+ protected LongParameter parallelism = new LongParameter(this, "__parallelism")
+ .setDefaultValue(PARALLELISM_DEFAULT);
--- End diff --
This should have the `min` and `max` values set.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r129295166
--- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/drivers/parameter/Simplify.java ---
@@ -109,21 +109,23 @@ public Ordering getValue() {
* @return output graph
* @throws Exception on error
*/
- public <T extends Comparable<T>> Graph<T, NullValue, NullValue> simplify(Graph<T, NullValue, NullValue> graph)
+ public <T extends Comparable<T>> Graph<T, NullValue, NullValue> simplify(Graph<T, NullValue, NullValue> graph, int parallelism)
throws Exception {
-
switch (value) {
case DIRECTED:
graph = graph
- .run(new org.apache.flink.graph.asm.simple.directed.Simplify<T, NullValue, NullValue>());
+ .run(new org.apache.flink.graph.asm.simple.directed.Simplify<T, NullValue, NullValue>()
+ .setParallelism(parallelism));
--- End diff --
it's different in the changes to `LocalClusteringCoefficient.java` though - was just wondering
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r129231269
--- Diff: flink-libraries/flink-gelly-examples/src/main/java/org/apache/flink/graph/drivers/parameter/Simplify.java ---
@@ -109,21 +109,23 @@ public Ordering getValue() {
* @return output graph
* @throws Exception on error
*/
- public <T extends Comparable<T>> Graph<T, NullValue, NullValue> simplify(Graph<T, NullValue, NullValue> graph)
+ public <T extends Comparable<T>> Graph<T, NullValue, NullValue> simplify(Graph<T, NullValue, NullValue> graph, int parallelism)
throws Exception {
-
switch (value) {
case DIRECTED:
graph = graph
- .run(new org.apache.flink.graph.asm.simple.directed.Simplify<T, NullValue, NullValue>());
+ .run(new org.apache.flink.graph.asm.simple.directed.Simplify<T, NullValue, NullValue>()
+ .setParallelism(parallelism));
break;
case UNDIRECTED:
graph = graph
- .run(new org.apache.flink.graph.asm.simple.undirected.Simplify<T, NullValue, NullValue>(false));
+ .run(new org.apache.flink.graph.asm.simple.undirected.Simplify<T, NullValue, NullValue>(false)
+ .setParallelism(parallelism));
--- End diff --
same here...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #4346: [FLINK-7199] [gelly] Graph simplification does not...
Posted by NicoK <gi...@git.apache.org>.
Github user NicoK commented on a diff in the pull request:
https://github.com/apache/flink/pull/4346#discussion_r129296867
--- Diff: flink-libraries/flink-gelly/src/main/java/org/apache/flink/graph/library/linkanalysis/HITS.java ---
@@ -168,7 +168,6 @@ protected void mergeConfiguration(GraphAlgorithmWrappingBase other) {
.setParallelism(parallelism)
.name("Square")
.reduce(new Sum())
- .setCombineHint(CombineHint.HASH)
--- End diff --
would have expected a separate commit but ok - I can't say much about the change though since I don't know flink's graph API too much...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---