You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by GitBox <gi...@apache.org> on 2020/09/22 16:16:29 UTC
[GitHub] [systemds] Baunsgaard commented on a change in pull request #1060: [SYSTEMDS-2668] Fine-Grained Privacy Constraint Propagation for Matrix Multiplications
Baunsgaard commented on a change in pull request #1060:
URL: https://github.com/apache/systemds/pull/1060#discussion_r492856792
##########
File path: src/main/java/org/apache/sysds/runtime/privacy/finegrained/FineGrainedPrivacyList.java
##########
@@ -35,11 +35,61 @@
private ArrayList<Map.Entry<DataRange, PrivacyLevel>> constraintCollection = new ArrayList<>();
Review comment:
why is it an Array List of Map Entries.
You can use Immutable Pairs, if you want to stick to Lists.
or maybe use a Hash Map instead of array list?
Have you measured how it performs, since i assume that what you want it to search through ranges and get the privacy associated with that specific data?
##########
File path: src/main/java/org/apache/sysds/runtime/privacy/finegrained/FineGrainedPrivacyList.java
##########
@@ -35,11 +35,61 @@
private ArrayList<Map.Entry<DataRange, PrivacyLevel>> constraintCollection = new ArrayList<>();
+ @Override
+ public PrivacyLevel[] getRowPrivacy(int numRows, int numCols) {
+ PrivacyLevel[] privacyLevels = new PrivacyLevel[numRows];
+ for (int i = 0; i < numRows; i++) {
+ long[] beginRange1 = new long[] {i, 0};
+ long[] endRange1 = new long[] {i, numCols};
+ privacyLevels[i] = getStrictestPrivacyLevel(new DataRange(beginRange1, endRange1));
+ }
+ return privacyLevels;
+ }
+
+ @Override
+ public PrivacyLevel[] getColPrivacy(int numRows, int numCols) {
+ PrivacyLevel[] privacyLevels = new PrivacyLevel[numCols];
+ for (int i = 0; i < numCols; i++) {
+ long[] beginRange = new long[] {0, i};
+ long[] endRange = new long[] {numRows, i};
+ privacyLevels[i] = getStrictestPrivacyLevel(new DataRange(beginRange, endRange));
+ }
+ return privacyLevels;
+ }
+
+ private PrivacyLevel getStrictestPrivacyLevel(DataRange searchRange){
+ PrivacyLevel strictestLevel = PrivacyLevel.None;
+ for ( Map.Entry<DataRange, PrivacyLevel> constraint : constraintCollection ) {
Review comment:
because you have a for loop in the previous and loop through each entry here. it is going to be slow.
I think a future project would be an index structure for the privacy. maybe a tree structure would be good?
one for cols and one for rows?
##########
File path: src/main/java/org/apache/sysds/runtime/privacy/finegrained/FineGrainedPrivacyMap.java
##########
@@ -40,6 +40,18 @@ public void put(DataRange dataRange, PrivacyLevel privacyLevel) {
constraintCollection.put(dataRange, privacyLevel);
}
+ @Override public void putRow(int rowIndex, int rowLength, PrivacyLevel privacyLevel) {
Review comment:
misformatted, and does nothing inside the method.
##########
File path: src/main/java/org/apache/sysds/runtime/privacy/propagation/PrivacyPropagator.java
##########
@@ -64,7 +64,61 @@ public static Data parseAndSetPrivacyConstraint(Data cd, JSONObject mtd)
}
return cd;
}
-
+
+ private static boolean anyInputHasLevel(PrivacyLevel[] inputLevels, PrivacyLevel targetLevel){
+ return Arrays.stream(inputLevels).anyMatch(i -> i == targetLevel);
+ }
+
+ /**
+ * Returns the output privacy level based on the given input privacy levels and operator type.
+ * It represents the basic logic of privacy propagation:
+ *
+ * Unary input:
+ * Input | NonAggregate | Aggregate
+ * -----------------------------------
+ * priv | priv | priv
+ * privAgg | privAgg | none
+ * none | none | none
+ *
+ * Binary input:
+ * Input | NonAggregate | Aggregate
+ * -----------------------------------
Review comment:
minor alignment
##########
File path: src/main/java/org/apache/sysds/runtime/privacy/propagation/OperatorType.java
##########
@@ -0,0 +1,25 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.sysds.runtime.privacy.propagation;
+
+public enum OperatorType {
Review comment:
this could be considered to be moved into common/Types.java
or leverage already existing Operator Types?
##########
File path: src/main/java/org/apache/sysds/runtime/privacy/propagation/MatrixMultiplicationPropagatorNaive.java
##########
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.sysds.runtime.privacy.propagation;
+
+import org.apache.sysds.runtime.matrix.data.MatrixBlock;
+import org.apache.sysds.runtime.privacy.PrivacyConstraint;
+import org.apache.sysds.runtime.privacy.PrivacyConstraint.PrivacyLevel;
+import org.apache.sysds.runtime.privacy.finegrained.FineGrainedPrivacy;
+
+/**
+ * MatrixMultiplicationPropagator that overrides generateFineGrainedConstraints
+ * with a naive propagation of the fine-grained constraints.
+ * The output is correct, but is likely less efficient than other implementations.
+ */
+public class MatrixMultiplicationPropagatorNaive extends MatrixMultiplicationPropagator{
+
+ public MatrixMultiplicationPropagatorNaive(){
+ super();
+ }
+
+ public MatrixMultiplicationPropagatorNaive(MatrixBlock input1, PrivacyConstraint privacyConstraint1,
+ MatrixBlock input2, PrivacyConstraint privacyConstraint2) {
+ super(input1, privacyConstraint1, input2, privacyConstraint2);
+ }
+
+ @Override
Review comment:
put annotations after jdoc
##########
File path: src/main/java/org/apache/sysds/runtime/privacy/propagation/PrivacyPropagator.java
##########
@@ -64,7 +64,61 @@ public static Data parseAndSetPrivacyConstraint(Data cd, JSONObject mtd)
}
return cd;
}
-
+
+ private static boolean anyInputHasLevel(PrivacyLevel[] inputLevels, PrivacyLevel targetLevel){
+ return Arrays.stream(inputLevels).anyMatch(i -> i == targetLevel);
+ }
+
+ /**
+ * Returns the output privacy level based on the given input privacy levels and operator type.
+ * It represents the basic logic of privacy propagation:
+ *
+ * Unary input:
+ * Input | NonAggregate | Aggregate
+ * -----------------------------------
+ * priv | priv | priv
+ * privAgg | privAgg | none
+ * none | none | none
+ *
+ * Binary input:
+ * Input | NonAggregate | Aggregate
+ * -----------------------------------
+ * priv-priv | priv | priv
+ * priv-privAgg | priv | priv
+ * priv-none | priv | priv
+ * privAgg-priv | priv | priv
+ * none-priv | priv | priv
+ * privAgg-privAgg | privAgg | none
+ * none-none | none | none
+ * privAgg-none | privAgg | none
+ * none-privAgg | privAgg | none
+ *
+ * @param inputLevels privacy levels of the input
+ * @param operatorType type of the operator which is either an aggregation (Aggregate) or not an aggregation (NonAggregate)
+ * @return output privacy level
Review comment:
:+1: nice jdoc
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org