You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pirk.apache.org by jacobwil <gi...@git.apache.org> on 2016/09/10 18:47:56 UTC
[GitHub] incubator-pirk pull request #88: [wip] Jnagmp for modular multiply, gcd, and...
GitHub user jacobwil opened a pull request:
https://github.com/apache/incubator-pirk/pull/88
[wip] Jnagmp for modular multiply, gcd, and exactDivide.
Today I submitted [a pull request to Square's JNA-GMP](https://github.com/square/jna-gmp/pull/17) which adds modular multiplication, gcd, and exactDivide to JNA-GMP. This library lets us take advantage of the GNU Multiple Precision Arithmetic Library's high performance integer math functions.
Pending acceptance of my PR into JNA-GMP (hopefully) and a release, I have modified the pom files to pull an updated JNA-GMP jar out of my personal github repo. I don't intend for the pom files to be accepted into master in this state but wanted to make the wip available for everyone to try if they want.
Benchmark results:
Benchmark Mode Cnt Score Error Units
ExactDivideBenchmark.testWithGMP thrpt 200 110961.159 � 1071.711 ops/s
ExactDivideBenchmark.testWithoutGMP thrpt 200 53641.770 � 267.455 ops/s
GCDBenchmark.testWithGMP thrpt 200 24121.188 � 144.105 ops/s
GCDBenchmark.testWithoutGMP thrpt 200 1243.952 � 7.829 ops/s
ModBenchmark.testWithGMP thrpt 200 93148.587 � 762.959 ops/s
ModBenchmark.testWithoutGMP thrpt 200 2300046.853 � 267769.213 ops/s
ModularInverseBenchmark.testWithGMP thrpt 200 9891.903 � 38.648 ops/s
ModularInverseBenchmark.testWithoutGMP thrpt 200 390.049 � 1.966 ops/s
ModularMultiplyBenchmark.testWithGMP thrpt 200 36154.112 � 219.346 ops/s
ModularMultiplyBenchmark.testWithoutGMP thrpt 200 15777.195 � 100.081 ops/s
MultiplyBenchmark.testWithGMP thrpt 200 102739.040 � 522.167 ops/s
MultiplyBenchmark.testWithoutGMP thrpt 200 229137.717 � 658.034 ops/s
PaillierBenchmark.testWithGMP thrpt 200 26.943 � 0.122 ops/s
PaillierBenchmark.testWithGMPConstantTime thrpt 200 19.695 � 0.078 ops/s
PaillierBenchmark.testWithoutGMP thrpt 200 4.573 � 0.012 ops/s
Since modularMultiply is a major operation in the wideskies algorithm the 1.3x speedup we see is a huge win.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jacobwil/incubator-pirk JNAGMPForModularMultiply
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-pirk/pull/88.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #88
----
commit ab4363990705a9346caf927831cda10ee4763a84
Author: Jacob <ja...@apache.org>
Date: 2016-08-05T15:51:44Z
Renamed ModPowAbstraction to IntegerMathAbstraction, added new functionality for using JNA-GMP to execute modular multiplication.
commit a1360d49afc83e1b686b51774558be77359f4859
Author: Jacob <ja...@apache.org>
Date: 2016-08-05T15:53:25Z
Add benchmarks for the new functionality of IntegerMathAbstraction
commit ea0a7b978eae9d9d365d21a7449d492f8964bd90
Author: Jacob <ja...@apache.org>
Date: 2016-08-05T15:56:31Z
Update uses of ModPowAbstraction to new name and convert uses of BigInteger modular multiply and modular inverse to use IntegerMathAbstraction
commit bc588763dd7351beed686360eb8fdce31ddbec08
Author: Jacob <ja...@apache.org>
Date: 2016-08-05T16:30:11Z
Added documentation for new classes
commit 8cb3a2b827442331d85f5a3b146c5e371044b87a
Author: Jacob <ja...@apache.org>
Date: 2016-08-05T16:41:52Z
Add properties paillier.useGMPForModularMultiply and paillier.useGMPForModularInverse. Update formatting
commit b7daed04ab4f952449141869c63a0433965a4b4f
Author: Jacob <ja...@apache.org>
Date: 2016-08-05T16:42:41Z
Update the test suite to use IntegerMathAbstraction
commit 2fdcee4859915a4139daea14cc7d7a7b404d39d3
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-06T16:04:45Z
Updated pom files to pull the custom jnagmp 1.1.1-SNAPSHOT jar from jacobwil's github
commit 73eacc21258a3228d9c0fdfa2c028dcf6cdba43b
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-06T16:23:31Z
Merge with master. Mostly documentation changes (which are now out of date) but also absorb changes to DataSchemaLoader and QuerySchemaLoader.
commit fdd7f7533c9049417c9e4c9021f8d17668d860df
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-14T18:21:06Z
Fixed junit error in dependencies for pom-with-benchmarks.xml
commit 2c4dd8cf40c25096925f0202a3557bc150660d09
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-15T20:32:04Z
Merge changes from upstream master
commit da8544184b66aa390c96c0d4c281b7da4659fcd1
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-16T14:38:38Z
Add gcd and exactDivide to IntegerMathAbstraction. Add configuration properties to pirk.properties. Make use of new methods in Paillier.java. Update PaillierBenchmark to set all GMP options instead of just GMP modPow options.
commit e983e3dd5635b60e0845978640807bf13a00b4df
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-18T16:16:52Z
Merge with upstream master
commit c1e5870193a2a5e1c61b3c430a5c5f384fc58beb
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-18T17:24:11Z
Moved the two unmerged benchmarks to the new benchmarks location.
commit f99ca4475fa537bb375c3aaf2a31d4490aa99dbb
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-18T18:08:39Z
Added ExactDivide and GCD benchmarks
commit 71fc2826bb54fa1de02198720537c9276d7323ce
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-18T18:37:13Z
Tried to update the benchmarks pom to Java 8 but nothing works now. Not sure exactly when the break happened.
commit 7b9d2f5dc7682bfdc542c8e3f1404ca4d4b88bfa
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-18T19:59:48Z
Moved JMH benchmarks back into the main source tree since they do not work within the test source tree
commit 2bf4855da667318529a055920b13d349e2d7a53f
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-18T20:03:54Z
Merge changes from upstream master.
commit 6e687c0265609655a883aaf23a8129723014ef8f
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-18T21:09:26Z
Added Mod and Multiply to IntegerMathAbstraction, wrote benchmarks. Converted usages of IntegerMathAbstraction in Paillier to import statics.
commit 4b425bb1bcbed707f2741fbddf6d0431ba3ff7fb
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-20T02:51:31Z
More benchmarks to detail modular multiply performance using different strategies (all worse than the original strategy but now with a way to prove it on an individual system). Improved documentation.
commit 6543e5d839f8f8b6503b6cf4fee07f2257360a06
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-20T02:59:25Z
Merge with upstream master
commit 2a9a826e976db121e141a412f3db10f730a5710f
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-24T02:26:54Z
Merging with upstream master
commit dd182e78a80b7a3e5079dc7ba0ece86b2af00aae
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-25T15:38:42Z
Merge branch 'master' of github.com:apache/incubator-pirk into JNAGMPForModularMultiply
commit 51de33efc9a105ded42532e1320e92941334f527
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-08-28T04:40:55Z
Merge branch 'master' of github.com:apache/incubator-pirk into JNAGMPForModularMultiply
commit e5c140b6da3dc43ad524365fa5575c09cdb2b1b1
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-09-10T01:33:22Z
Fix pom-with-benchmarks, update JMH version
commit 53ae4254fdd61d960c07af7039a7ea385b16ac0b
Author: Jacob Wilder <ja...@apache.org>
Date: 2016-09-10T01:55:05Z
Merge branch 'master' of github.com:apache/incubator-pirk into JNAGMPForModularMultiply
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk pull request #88: [wip] Jnagmp for modular multiply, gcd, and...
Posted by ellisonanne <gi...@git.apache.org>.
Github user ellisonanne commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/88#discussion_r78297525
--- Diff: pom-with-benchmarks.xml ---
@@ -67,7 +67,7 @@
<id>mvn-public</id>
--- End diff --
I am assuming that most of the changes are due to a merge from an older pom file on your side...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk pull request #88: [wip] Jnagmp for modular multiply, gcd, and...
Posted by ellisonanne <gi...@git.apache.org>.
Github user ellisonanne commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/88#discussion_r78297559
--- Diff: pom.xml ---
@@ -95,6 +95,18 @@
<pirk.forkCount>1C</pirk.forkCount>
<pirk.reuseForks>true</pirk.reuseForks>
</properties>
--- End diff --
I'm fine with this for now in order to leverage the performance gain - however, we will need another long term solution if your PR to JNA-GMP is not rolled into a release in a timely manner.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk issue #88: [wip] Jnagmp for modular multiply, gcd, and exactD...
Posted by ellisonanne <gi...@git.apache.org>.
Github user ellisonanne commented on the issue:
https://github.com/apache/incubator-pirk/pull/88
Looks good - I made a few minor comment inline with the code above - when do you think that it will be ready to go from a [WIP] PR to ready-to-merge?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk pull request #88: [wip] Jnagmp for modular multiply, gcd, and...
Posted by ellisonanne <gi...@git.apache.org>.
Github user ellisonanne commented on a diff in the pull request:
https://github.com/apache/incubator-pirk/pull/88#discussion_r78297606
--- Diff: src/main/java/org/apache/pirk/encryption/IntegerMathAbstraction.java ---
@@ -0,0 +1,290 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pirk.encryption;
+
+import java.math.BigInteger;
+
+import org.apache.pirk.utils.SystemConfiguration;
+
+import com.squareup.jnagmp.Gmp;
+
+/**
+ * This class is designed to offer a one-stop-shop for invoking the desired version of
+ * modPow, modularMultiply, and modInverse
+ */
+public final class IntegerMathAbstraction
+{
+
+ private static boolean useGMPmodPow, useGMPConstantTimeMethods, useGMPmodularMultiply,
+ useGMPmodularInverse, useGMPgcd, useGMPexactDivide, useGMPmultiply, useGMPmod;
+
+ static
+ {
+ // Load the configuration
+ reloadConfiguration();
+ }
+
+ /**
+ * Reloads the configuration options for this class. They control which implementations are used for integer
+ * math: GMP or BigInteger.
+ */
+ public static void reloadConfiguration()
--- End diff --
These can be replaced with SystemConfiguration.getBooleanProperty
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk pull request #88: [wip] Jnagmp for modular multiply, gcd, and...
Posted by jacobwil <gi...@git.apache.org>.
Github user jacobwil closed the pull request at:
https://github.com/apache/incubator-pirk/pull/88
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk issue #88: [wip] Jnagmp for modular multiply, gcd, and exactD...
Posted by jacobwil <gi...@git.apache.org>.
Github user jacobwil commented on the issue:
https://github.com/apache/incubator-pirk/pull/88
I am unable to complete this pull request because JNA-GMP has not done a release with the new features and decided that composed functions like modular multiply were out of their scope.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-pirk issue #88: [wip] Jnagmp for modular multiply, gcd, and exactD...
Posted by jacobwil <gi...@git.apache.org>.
Github user jacobwil commented on the issue:
https://github.com/apache/incubator-pirk/pull/88
Thanks for the feedback Ellison Anne. I've been on a merge-with-master conflict resolution treadmill for a bit. Before I suggest this PR is ready for final merging I'll grab the latest poms and surgically make the version changes.
When the PR will be ready to merge depends on JNA-GMP's response to my PR. As things happen I'll post more comments here.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---