You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Patrick Baier (JIRA)" <ji...@apache.org> on 2015/07/09 15:28:05 UTC
[jira] [Comment Edited] (SPARK-7751) Add @since to stable and
experimental methods in MLlib
[ https://issues.apache.org/jira/browse/SPARK-7751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14620486#comment-14620486 ]
Patrick Baier edited comment on SPARK-7751 at 7/9/15 1:27 PM:
--------------------------------------------------------------
I built this short batch script here to search for the version of methods:
{code:borderStyle=solid}
#$1=sourceFile to search
#$2=string to search for
versions=$(git tag)
for v in $versions
do
echo "Checking version "$v
versionedFile=$(git show $v:$1)
matches=$(echo $versionedFile | grep -c "$2")
if [ $matches -gt 0 ]
then
echo "Introduced in version "$v
exit 0
fi
done
echo "search string $2 not found!"
{code}
Note: You must be in the spark home directory to run it.
Example usage:
$1=mllib/src/main/scala/org/apache/spark/mllib/classification/LogisticRegression.scala
$2="override protected def createModel"
was (Author: pbaier):
If built this short batch script here to search for the version of methods:
{code:borderStyle=solid}
#$1=sourceFile to search
#$2=string to search for
versions=$(git tag)
for v in $versions
do
echo "Checking version "$v
versionedFile=$(git show $v:$1)
matches=$(echo $versionedFile | grep -c "$2")
if [ $matches -gt 0 ]
then
echo "Introduced in version "$v
exit 0
fi
done
echo "search string $2 not found!"
{code}
Note: You must be in the spark home directory to run it.
Example usage:
$1=mllib/src/main/scala/org/apache/spark/mllib/classification/LogisticRegression.scala
$2="override protected def createModel"
> Add @since to stable and experimental methods in MLlib
> ------------------------------------------------------
>
> Key: SPARK-7751
> URL: https://issues.apache.org/jira/browse/SPARK-7751
> Project: Spark
> Issue Type: Umbrella
> Components: Documentation, MLlib
> Affects Versions: 1.4.0
> Reporter: Xiangrui Meng
> Assignee: Xiangrui Meng
> Priority: Minor
> Labels: starter
>
> This is useful to check whether a feature exists in some version of Spark. This is an umbrella JIRA to track the progress. We want to have @since tag for both stable (those without any Experimental/DeveloperApi/AlphaComponent annotations) and experimental methods in MLlib:
> * an example PR for Scala: https://github.com/apache/spark/pull/6101
> * an example PR for Python: https://github.com/apache/spark/pull/6295
> We need to dig the history of git commit to figure out what was the Spark version when a method was first introduced. Take `NaiveBayes.setModelType` as an example. We can grep `def setModelType` at different version git tags.
> {code}
> meng@xm:~/src/spark
> $ git show v1.3.0:mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala | grep "def setModelType"
> meng@xm:~/src/spark
> $ git show v1.4.0:mllib/src/main/scala/org/apache/spark/mllib/classification/NaiveBayes.scala | grep "def setModelType"
> def setModelType(modelType: String): NaiveBayes = {
> {code}
> If there are better ways, please let us know.
> We cannot add all @since tags in a single PR, which is hard to review. So we made some subtasks for each package, for example `org.apache.spark.classification`. Feel free to add more sub-tasks for Python and the `spark.ml` package.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org