You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemds.apache.org by GitBox <gi...@apache.org> on 2020/12/12 15:08:50 UTC
[GitHub] [systemds] Baunsgaard opened a new pull request #1124: [SYSTEMDS-2757] PCA Predict and Inverse
Baunsgaard opened a new pull request #1124:
URL: https://github.com/apache/systemds/pull/1124
Adds a predict function for PCA and an inverse function.
The predict is for unseen data, that the PCA was not trained for just like our other predict functions for other algorithms
The Inverse is to apply on an PCA reduced dataset, to reconstruct an approximation of the original data.
The second is added because PCA can be considered a classic Lossy compression algorithm, that gives an smaller, approximated matrix of the original data. It also allows us to calculate how well the given PCA fits to the dateset.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] Baunsgaard commented on a change in pull request #1124: [SYSTEMDS-2757] PCA Predict and Inverse
Posted by GitBox <gi...@apache.org>.
Baunsgaard commented on a change in pull request #1124:
URL: https://github.com/apache/systemds/pull/1124#discussion_r543246881
##########
File path: scripts/builtin/pcaPredict.dml
##########
@@ -0,0 +1,49 @@
+#-------------------------------------------------------------
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+#-------------------------------------------------------------
+
+# Principal Component Analysis (PCA) for dimensionality reduction prediciton
+#
+# This method is used to predict on data, which the PCA was not trained on. To validate how good
+# The PCA is, and to apply in production.
+#
+# ---------------------------------------------------------------------------------------------
+# NAME TYPE DEFAULT MEANING
+# ---------------------------------------------------------------------------------------------
+# X Matrix --- Input feature matrix
+# Centering Matrix empty matrix The column means of the PCA model, subtracted to construct the PCA
+# ScaleFactor Matrix empty matrix The scaling of each dimension in the PCA model
+# ---------------------------------------------------------------------------------------------
+# Y Matrix --- Output feature matrix dimensionally reduced by PCA
+# ---------------------------------------------------------------------------------------------
+
+m_pcaPredict = function(Matrix[Double] X, Matrix[Double] Clusters, Matrix[Double] Centering, Matrix[Double] ScaleFactor)
Review comment:
but i would refrain from the underscore.
and we have to agree on that general syntax in our interface
are we using camel case "java syntax" or "google syntax" underscore?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] Shafaq-Siddiqi commented on a change in pull request #1124: [SYSTEMDS-2757] PCA Predict and Inverse
Posted by GitBox <gi...@apache.org>.
Shafaq-Siddiqi commented on a change in pull request #1124:
URL: https://github.com/apache/systemds/pull/1124#discussion_r543249340
##########
File path: scripts/builtin/pcaPredict.dml
##########
@@ -0,0 +1,49 @@
+#-------------------------------------------------------------
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+#-------------------------------------------------------------
+
+# Principal Component Analysis (PCA) for dimensionality reduction prediciton
+#
+# This method is used to predict on data, which the PCA was not trained on. To validate how good
+# The PCA is, and to apply in production.
+#
+# ---------------------------------------------------------------------------------------------
+# NAME TYPE DEFAULT MEANING
+# ---------------------------------------------------------------------------------------------
+# X Matrix --- Input feature matrix
+# Centering Matrix empty matrix The column means of the PCA model, subtracted to construct the PCA
+# ScaleFactor Matrix empty matrix The scaling of each dimension in the PCA model
+# ---------------------------------------------------------------------------------------------
+# Y Matrix --- Output feature matrix dimensionally reduced by PCA
+# ---------------------------------------------------------------------------------------------
+
+m_pcaPredict = function(Matrix[Double] X, Matrix[Double] Clusters, Matrix[Double] Centering, Matrix[Double] ScaleFactor)
Review comment:
I prefer the camel case. For now, I see both styles in our built-in package but the camel case is dominant. So go for this.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] Baunsgaard commented on a change in pull request #1124: [SYSTEMDS-2757] PCA Predict and Inverse
Posted by GitBox <gi...@apache.org>.
Baunsgaard commented on a change in pull request #1124:
URL: https://github.com/apache/systemds/pull/1124#discussion_r541622750
##########
File path: scripts/builtin/scale.dml
##########
@@ -19,29 +19,48 @@
#
#-------------------------------------------------------------
-# Scale and center individual features in the input matrix
-# (column-wise) using z-score to scale the values.
-# -----------------------------------------------------------------------------
-# NAME TYPE DEFAULT MEANING
-# -----------------------------------------------------------------------------
-# X Matrix --- Input feature matrix
-# center Boolean TRUE Indicates whether or not to center the feature matrix
-# scale Boolean TRUE Indicates whether or not to scale the feature matrix
-# -----------------------------------------------------------------------------
-# Y Matrix --- Output feature matrix with K columns
-# -----------------------------------------------------------------------------
+# Scale and center individual features in the input matrix (column wise.) using z-score to scale the values.
+# ---------------------------------------------------------------------------------------------
+# NAME TYPE DEFAULT MEANING
+# ---------------------------------------------------------------------------------------------
+# X Matrix --- Input feature matrix
+# Center Boolean TRUE Indicates whether or not to center the feature matrix
+# Scale Boolean TRUE Indicates whether or not to scale the feature matrix
+# ---------------------------------------------------------------------------------------------
+# Y Matrix --- Output feature matrix with K columns
+# ColMean Matrix --- The column means of the input, subtracted if Center was TRUE
+# ScaleFactor Matrix --- The Scaling of the values, to make each dimension have similar value ranges
+# ---------------------------------------------------------------------------------------------
-m_scale = function(Matrix[Double] X, Boolean center, Boolean scale) return (Matrix[Double] Y) {
- if( center )
- X = X - colMeans(X);
+m_scale = function(Matrix[Double] X, Boolean center, Boolean scale)
+ return (Matrix[Double] Y, Matrix[Double] ColMean, Matrix[Double] ScaleFactor)
+{
+ if(center){
+ ColMean = colMeans(X)
+ X = X - ColMean
+ }
+ else {
+ ColMean = matrix(0,rows=0,cols=0)
+ }
if (scale) {
- cvars = colSums(X^2)/(nrow(X)-1);
+ N = nrow(X);
+
+ ScaleFactor = sqrt(colSums(X^2)/(N-1))
- #scale by std-dev and replace NaNs with 0's
- X = replace(target=X/sqrt(cvars),
- pattern=NaN, replacement=0);
+ # Replace entries in the scale factor that are 0 with an small eps
+ # To avoid division by 0, and thereby introducing NaN to the ouput.
+ ScaleFactor = replace(target=ScaleFactor,
+ pattern=NaN, replacement=1e-16);
Review comment:
Alternatively we could replace the values with 1, that would maybe be better since that would remove the scaling of that dimension.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] Shafaq-Siddiqi commented on a change in pull request #1124: [SYSTEMDS-2757] PCA Predict and Inverse
Posted by GitBox <gi...@apache.org>.
Shafaq-Siddiqi commented on a change in pull request #1124:
URL: https://github.com/apache/systemds/pull/1124#discussion_r543225442
##########
File path: scripts/builtin/pcaPredict.dml
##########
@@ -0,0 +1,49 @@
+#-------------------------------------------------------------
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+#-------------------------------------------------------------
+
+# Principal Component Analysis (PCA) for dimensionality reduction prediciton
+#
+# This method is used to predict on data, which the PCA was not trained on. To validate how good
+# The PCA is, and to apply in production.
+#
+# ---------------------------------------------------------------------------------------------
+# NAME TYPE DEFAULT MEANING
+# ---------------------------------------------------------------------------------------------
+# X Matrix --- Input feature matrix
+# Centering Matrix empty matrix The column means of the PCA model, subtracted to construct the PCA
+# ScaleFactor Matrix empty matrix The scaling of each dimension in the PCA model
+# ---------------------------------------------------------------------------------------------
+# Y Matrix --- Output feature matrix dimensionally reduced by PCA
+# ---------------------------------------------------------------------------------------------
+
+m_pcaPredict = function(Matrix[Double] X, Matrix[Double] Clusters, Matrix[Double] Centering, Matrix[Double] ScaleFactor)
Review comment:
Can we reconsider the name of this built-in? As from the word predict we usually refer to some class or value prediction but here you are somehow transforming the data using pre-computed dominant vectors. I would suggest pca_tranform() it would be consistent with PCA implementation in Scikit-learn.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] Baunsgaard merged pull request #1124: [SYSTEMDS-2757] PCA Predict and Inverse
Posted by GitBox <gi...@apache.org>.
Baunsgaard merged pull request #1124:
URL: https://github.com/apache/systemds/pull/1124
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] Shafaq-Siddiqi commented on a change in pull request #1124: [SYSTEMDS-2757] PCA Predict and Inverse
Posted by GitBox <gi...@apache.org>.
Shafaq-Siddiqi commented on a change in pull request #1124:
URL: https://github.com/apache/systemds/pull/1124#discussion_r543249340
##########
File path: scripts/builtin/pcaPredict.dml
##########
@@ -0,0 +1,49 @@
+#-------------------------------------------------------------
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+#-------------------------------------------------------------
+
+# Principal Component Analysis (PCA) for dimensionality reduction prediciton
+#
+# This method is used to predict on data, which the PCA was not trained on. To validate how good
+# The PCA is, and to apply in production.
+#
+# ---------------------------------------------------------------------------------------------
+# NAME TYPE DEFAULT MEANING
+# ---------------------------------------------------------------------------------------------
+# X Matrix --- Input feature matrix
+# Centering Matrix empty matrix The column means of the PCA model, subtracted to construct the PCA
+# ScaleFactor Matrix empty matrix The scaling of each dimension in the PCA model
+# ---------------------------------------------------------------------------------------------
+# Y Matrix --- Output feature matrix dimensionally reduced by PCA
+# ---------------------------------------------------------------------------------------------
+
+m_pcaPredict = function(Matrix[Double] X, Matrix[Double] Clusters, Matrix[Double] Centering, Matrix[Double] ScaleFactor)
Review comment:
I prefer the camel case. For now, I see both styles in our built-in package but the camel case is dominant. So go for this. Though R and Python both use Google syntax.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [systemds] Baunsgaard commented on a change in pull request #1124: [SYSTEMDS-2757] PCA Predict and Inverse
Posted by GitBox <gi...@apache.org>.
Baunsgaard commented on a change in pull request #1124:
URL: https://github.com/apache/systemds/pull/1124#discussion_r543245864
##########
File path: scripts/builtin/pcaPredict.dml
##########
@@ -0,0 +1,49 @@
+#-------------------------------------------------------------
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+#-------------------------------------------------------------
+
+# Principal Component Analysis (PCA) for dimensionality reduction prediciton
+#
+# This method is used to predict on data, which the PCA was not trained on. To validate how good
+# The PCA is, and to apply in production.
+#
+# ---------------------------------------------------------------------------------------------
+# NAME TYPE DEFAULT MEANING
+# ---------------------------------------------------------------------------------------------
+# X Matrix --- Input feature matrix
+# Centering Matrix empty matrix The column means of the PCA model, subtracted to construct the PCA
+# ScaleFactor Matrix empty matrix The scaling of each dimension in the PCA model
+# ---------------------------------------------------------------------------------------------
+# Y Matrix --- Output feature matrix dimensionally reduced by PCA
+# ---------------------------------------------------------------------------------------------
+
+m_pcaPredict = function(Matrix[Double] X, Matrix[Double] Clusters, Matrix[Double] Centering, Matrix[Double] ScaleFactor)
Review comment:
sure transform works as well! :+1:
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org