You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@toree.apache.org by lb...@apache.org on 2016/04/01 15:47:45 UTC

incubator-toree git commit: [TOREE 277]: Added tutorial notebook for magics

Repository: incubator-toree
Updated Branches:
  refs/heads/master 222a25c30 -> bf6fc5056


[TOREE 277]: Added tutorial notebook for magics


Project: http://git-wip-us.apache.org/repos/asf/incubator-toree/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-toree/commit/bf6fc505
Tree: http://git-wip-us.apache.org/repos/asf/incubator-toree/tree/bf6fc505
Diff: http://git-wip-us.apache.org/repos/asf/incubator-toree/diff/bf6fc505

Branch: refs/heads/master
Commit: bf6fc5056119378436902ef5715a40f1124b376f
Parents: 222a25c
Author: Corey A. Stubbs <cs...@us.ibm.com>
Authored: Wed Mar 30 16:03:16 2016 -0500
Committer: Corey A. Stubbs <cs...@us.ibm.com>
Committed: Thu Mar 31 14:10:03 2016 -0500

----------------------------------------------------------------------
 etc/examples/notebooks/magic-tutorial.ipynb | 795 +++++++++++++++++++++++
 1 file changed, 795 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-toree/blob/bf6fc505/etc/examples/notebooks/magic-tutorial.ipynb
----------------------------------------------------------------------
diff --git a/etc/examples/notebooks/magic-tutorial.ipynb b/etc/examples/notebooks/magic-tutorial.ipynb
new file mode 100644
index 0000000..ba33bc3
--- /dev/null
+++ b/etc/examples/notebooks/magic-tutorial.ipynb
@@ -0,0 +1,795 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Toree Magics<a name=\"top\"></a>\n",
+    "Magics are special \"functions\" which enable features or execute some special code. Magics can receive input arguments when they are invoked. There are two types of magics: `cell` magics and `line` magics. Magics invocations are not case sensitive.\n",
+    "\n",
+    "**Table of Contents**\n",
+    "\n",
+    "1. [Line Magics](#line-magics)\n",
+    "    1. [LsMagic](#lsmagic)\n",
+    "    1. [Truncation](#truncation)\n",
+    "    1. [ShowTypes](#showtypes)\n",
+    "    1. [AddJar](#addjar)\n",
+    "    1. [AddDeps](#adddeps)\n",
+    "1. [Cell Magics](#cell-magics)\n",
+    "    1. [Html](#html)\n",
+    "    1. [JavaScript](#javascript)\n",
+    "    1. [PySpark](#pyspark)\n",
+    "    1. [SparkR](#sparkr)\n",
+    "    1. [SparkSQL](#sparksql)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Line Magics<a name=\"line-magics\"></a><span style=\"float: right; font-size: 0.5em\"><a href=\"#top\">Top</a></span>\n",
+    "Line magics are run on a single line and can have other code and line magics within the same cell. Line magics use the following syntax: \n",
+    "\n",
+    "```\n",
+    "%magicname [args]\n",
+    "```\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### %LsMagic<a name=\"lsmagic\"></a><span style=\"float: right; font-size: 0.5em\"><a href=\"#top\">Top</a></span>\n",
+    "The `LsMagic` is a magic to list all the available magics."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Available line magics:\n",
+      "%addjar %lsmagic %showtypes %adddeps %truncation\n",
+      "\n",
+      "Available cell magics:\n",
+      "%%sql %%html %%javascript %%rdd %%scala %%sparkr %%pyspark\n",
+      "\n",
+      "Type %<magic_name> for usage info.\n",
+      "         \n"
+     ]
+    }
+   ],
+   "source": [
+    "%LsMagic"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### %Truncation<a name=\"truncation\"></a><span style=\"float: right; font-size: 0.5em\"><a href=\"#top\">Top</a></span>\n",
+    "Toree will, by default, truncate results from statements. This can be managed through the `%Truncation` magic. To see the current state of the truncation setting you can invoke the magic."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Truncation is currently on \n"
+     ]
+    }
+   ],
+   "source": [
+    "// invoke the truncation magic to see if truncation is on or off\n",
+    "%Truncation"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "Range(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170..."
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "// return a value to see the truncation\n",
+    "(1 to 200)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Output will NOT be truncated\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "Range(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200)"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "%Truncation off\n",
+    "(1 to 200)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Output WILL be truncated.\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "Range(1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170..."
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "%Truncation on\n",
+    "(1 to 200)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### %ShowTypes<a name=\"showtypes\"></a><span style=\"float: right; font-size: 0.5em\"><a href=\"#top\">Top</a></span>\n",
+    "The type information for a result is hidden by default. This behavior can be changed by using the `%ShowTypes` magic. You can view the current state of `%ShowTypes` by invoking it with no arguments."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "ShowTypes is currently off \n"
+     ]
+    }
+   ],
+   "source": [
+    "%ShowTypes"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "Hello types!"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "\"Hello types!\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Types will be printed.\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "String = Hello types!"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "%ShowTypes on\n",
+    "\"Hello types!\""
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "(Int, String) = (1,Hello types!)"
+      ]
+     },
+     "execution_count": 9,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "(1, \"Hello types!\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Types will not be printed\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "Hello types!"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "%ShowTypes off\n",
+    "\"Hello types!\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### %AddJar<a name=\"addjar\"></a><span style=\"float: right; font-size: 0.5em\"><a href=\"#top\">Top</a></span>\n",
+    "`AddJar` is a magic that allows the addition of jars to Torree's environment. You can see the arguments for `AddJar` by invoking it with no arguments."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 11,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Usage: %AddJar <jar_url>\n",
+      "\n",
+      "Option   Description                        \n",
+      "------   -----------                        \n",
+      "-f       forces re-download of specified jar\n",
+      "--magic  loads jar as a magic extension     \n"
+     ]
+    }
+   ],
+   "source": [
+    "%AddJar"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Starting download from https://repo1.maven.org/maven2/org/lwjgl/lwjgl/3.0.0b/lwjgl-3.0.0b.jar\n",
+      "Finished download of lwjgl-3.0.0b.jar\n"
+     ]
+    }
+   ],
+   "source": [
+    "%AddJar https://repo1.maven.org/maven2/org/lwjgl/lwjgl/3.0.0b/lwjgl-3.0.0b.jar"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "3.0.0b SNAPSHOT"
+      ]
+     },
+     "execution_count": 13,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "org.lwjgl.Version.getVersion()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## %AddDeps<a name=\"adddeps\"></a><span style=\"float: right; font-size: 0.5em\"><a href=\"#top\">Top</a></span>\n",
+    "`AddDeps` is a magic to add dependencies from a maven repository. You can see the arguments for `AddDeps` by invoking it with no arguments."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 14,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Usage: %AddDeps my.company artifact-id version\n",
+      "\n",
+      "Option                        Description                          \n",
+      "------                        -----------                          \n",
+      "--abort-on-resolution-errors  Abort (no downloads) when resolution \n",
+      "                                fails                              \n",
+      "--repository                  Adds an additional repository to     \n",
+      "                                available list                     \n",
+      "--trace                       Prints out trace of download progress\n",
+      "--transitive                  Retrieve dependencies recursively    \n",
+      "--verbose                     Prints out additional information    \n"
+     ]
+    }
+   ],
+   "source": [
+    "%AddDeps"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Note, that by default the `AddDeps` magic will only retrieve the specified dependency. If you want the transitive dependencies provide the `--transitive` flag."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 15,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Marking org.joda:joda-money:0.11 for download\n",
+      "Preparing to fetch from:\n",
+      "-> file:/tmp/.ivy2/\n",
+      "-> https://repo1.maven.org/maven2\n",
+      "=> 1 (): Downloading https://repo1.maven.org/maven2/org/joda/joda-money/0.11/joda-money-0.11.pom.sha1\n",
+      "=> 1 (): Downloading https://repo1.maven.org/maven2/org/joda/joda-money/0.11/joda-money-0.11.pom\n",
+      "===> 1 (joda-money-0.11.pom.sha1): Is 40 total bytes\n",
+      "===> 1 (joda-money-0.11.pom.sha1): Downloaded 40 bytes (100.00%)\n",
+      "=> 1 (joda-money-0.11.pom.sha1): Finished downloading\n",
+      "===> 1 (joda-money-0.11.pom): Is 22792 total bytes\n",
+      "===> 1 (joda-money-0.11.pom): Downloaded 15727 bytes (69.00%)\n",
+      "===> 1 (joda-money-0.11.pom): Downloaded 22792 bytes (100.00%)\n",
+      "=> 1 (joda-money-0.11.pom): Finished downloading\n",
+      "=> 1 (): Downloading https://repo1.maven.org/maven2/org/joda/joda-money/0.11/joda-money-0.11.jar.sha1\n",
+      "=> 2 (): Downloading https://repo1.maven.org/maven2/org/joda/joda-money/0.11/joda-money-0.11.jar\n",
+      "===> 1 (joda-money-0.11.jar.sha1): Is 40 total bytes\n",
+      "===> 1 (joda-money-0.11.jar.sha1): Downloaded 40 bytes (100.00%)\n",
+      "=> 1 (joda-money-0.11.jar.sha1): Finished downloading\n",
+      "===> 2 (joda-money-0.11.jar): Is 63725 total bytes\n",
+      "===> 2 (joda-money-0.11.jar): Downloaded 15713 bytes (24.66%)\n",
+      "===> 2 (joda-money-0.11.jar): Downloaded 32097 bytes (50.37%)\n",
+      "===> 2 (joda-money-0.11.jar): Downloaded 48481 bytes (76.08%)\n",
+      "===> 2 (joda-money-0.11.jar): Downloaded 63725 bytes (100.00%)\n",
+      "=> 2 (joda-money-0.11.jar): Finished downloading\n",
+      "-> New file at /tmp/.ivy2/https/repo1.maven.org/maven2/org/joda/joda-money/0.11/joda-money-0.11.jar\n"
+     ]
+    }
+   ],
+   "source": [
+    "%AddDeps org.joda joda-money 0.11 --transitive --trace --verbose"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 16,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "AUD"
+      ]
+     },
+     "execution_count": 16,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "org.joda.money.CurrencyUnit.AUD"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "collapsed": true
+   },
+   "source": [
+    "## Cell Magics<a name=\"cell-magics\"></a><span style=\"float: right; font-size: 0.5em\"><a href=\"#top\">Top</a></span>\n",
+    "Cell magics are magics which take the whole cell as their argument. They take the following form:\n",
+    "\n",
+    "```\n",
+    "%%magicname\n",
+    "line1\n",
+    "line2\n",
+    "...\n",
+    "```"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "collapsed": true
+   },
+   "source": [
+    "### %%Html<a name=\"html\"></a><span style=\"float: right; font-size: 0.5em\"><a href=\"#top\">Top</a></span>\n",
+    "The `%%HTML` magic allows you to return HTML."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 17,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<p>\n",
+       "Hello, <strong>Magics</strong>!\n",
+       "</p>"
+      ]
+     },
+     "execution_count": 17,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "%%html\n",
+    "<p>\n",
+    "Hello, <strong>Magics</strong>!\n",
+    "</p>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "collapsed": true
+   },
+   "source": [
+    "### %%JavaScript<a name=\"javascript\"></a><span style=\"float: right; font-size: 0.5em\"><a href=\"#top\">Top</a></span>\n",
+    "The `%%JavaScript` magic allows to return JavaScript. The JavaScript code will run in the notebook."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 18,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "application/javascript": [
+       "alert(\"Hello, Magics!\")"
+      ]
+     },
+     "execution_count": 18,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "%%JavaScript\n",
+    "alert(\"Hello, Magics!\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "collapsed": true
+   },
+   "source": [
+    "### %%PySpark<a name=\"pyspark\"></a><span style=\"float: right; font-size: 0.5em\"><a href=\"#top\">Top</a></span>\n",
+    "The `%%PySpark` exposes an environment with and a python interpreter and a shared `SparkContext`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "4950\n"
+      ]
+     },
+     "execution_count": 19,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "%%PySpark\n",
+    "from operator import add\n",
+    "print(sc.parallelize(range(1, 100)).reduce(add))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "collapsed": true
+   },
+   "source": [
+    "### %%SparkR<a name=\"sparkr\"></a><span style=\"float: right; font-size: 0.5em\"><a href=\"#top\">Top</a></span>\n",
+    "The `%%SparkR` exposes an environment with and an R interpreter and a shared `SparkContext`."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "* installing *binary* package ‘SparkR’ ...\n",
+      "* DONE (SparkR)\n",
+      "Loading required package: methods\n",
+      "\n",
+      "Attaching package: ‘SparkR’\n",
+      "\n",
+      "The following objects are masked from ‘package:stats’:\n",
+      "\n",
+      "    filter, na.omit\n",
+      "\n",
+      "The following objects are masked from ‘package:base’:\n",
+      "\n",
+      "    intersect, rbind, sample, subset, summary, table, transform\n",
+      "\n",
+      "NULL\n",
+      "[1] \"Received Id 680be9df-a384-455d-b548-405560772cc1 Code df <- createDataFrame(sqlContext, faithful)\\nhead(df)\"\n",
+      "[1] \"Code expr df <- createDataFrame(sqlContext, faithful)\"\n",
+      "[2] \"Code expr head(df)\"                                   \n",
+      "[1] \"Result type character 7\"\n",
+      "[1] \"Success 680be9df-a384-455d-b548-405560772cc1   eruptions waiting\"\n",
+      "[2] \"Success 680be9df-a384-455d-b548-405560772cc1 1     3.600      79\"\n",
+      "[3] \"Success 680be9df-a384-455d-b548-405560772cc1 2     1.800      54\"\n",
+      "[4] \"Success 680be9df-a384-455d-b548-405560772cc1 3     3.333      74\"\n",
+      "[5] \"Success 680be9df-a384-455d-b548-405560772cc1 4     2.283      62\"\n",
+      "[6] \"Success 680be9df-a384-455d-b548-405560772cc1 5     4.533      85\"\n",
+      "[7] \"Success 680be9df-a384-455d-b548-405560772cc1 6     2.883      55\"\n",
+      "[1] \"Marking success with output: eruptions waiting\\n1     3.600      79\\n2     1.800      54\\n3     3.333      74\\n4     2.283      62\\n5     4.533      85\\n6     2.883      55\"\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "eruptions waiting\n",
+       "1     3.600      79\n",
+       "2     1.800      54\n",
+       "3     3.333      74\n",
+       "4     2.283      62\n",
+       "5     4.533      85\n",
+       "6     2.883      55"
+      ]
+     },
+     "execution_count": 20,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "%%SparkR\n",
+    "df <- createDataFrame(sqlContext, faithful)\n",
+    "head(df)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "collapsed": true
+   },
+   "source": [
+    "### %%SparkSQL<a name=\"sparksql\"></a><span style=\"float: right; font-size: 0.5em\"><a href=\"#top\">Top</a></span>\n",
+    "The `%%SparkSQL` magic allows for SQL queries to be performed against tables saved in spark."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 21,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "import sqlContext.implicits._\n",
+    "case class Record(key: String, value: Int)\n",
+    "val df = sc.parallelize(1 to 10).map(x => Record(x.toString, x)).toDF()\n",
+    "df.registerTempTable(\"MYTABLE\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 22,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "+---+-----+\n",
+       "|key|value|\n",
+       "+---+-----+\n",
+       "|  6|    6|\n",
+       "|  7|    7|\n",
+       "|  8|    8|\n",
+       "|  9|    9|\n",
+       "| 10|   10|\n",
+       "+---+-----+\n",
+       "\n"
+      ]
+     },
+     "execution_count": 22,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "%%SQL\n",
+    "SELECT * FROM MYTABLE WHERE value >= 6"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "+---+-----+\n",
+       "|key|value|\n",
+       "+---+-----+\n",
+       "|  4|    4|\n",
+       "|  5|    5|\n",
+       "|  6|    6|\n",
+       "|  7|    7|\n",
+       "|  8|    8|\n",
+       "|  9|    9|\n",
+       "| 10|   10|\n",
+       "+---+-----+\n",
+       "\n"
+      ]
+     },
+     "execution_count": 23,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "%%SQL\n",
+    "SELECT * FROM MYTABLE WHERE value >= 4"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Toree - Scala",
+   "language": "scala",
+   "name": "toree_scala"
+  },
+  "language_info": {
+   "name": "scala"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 0
+}