You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lenya.apache.org by an...@apache.org on 2009/01/29 10:49:42 UTC
svn commit: r738808 [3/3] - in /lenya/trunk: legal/
src/modules/editors/config/ src/modules/editors/resources/codemirror/
src/modules/editors/resources/codemirror/0.60/
src/modules/editors/resources/codemirror/0.60/css/
src/modules/editors/resources/co...
Added: lenya/trunk/src/modules/editors/resources/codemirror/0.60/manual.html
URL: http://svn.apache.org/viewvc/lenya/trunk/src/modules/editors/resources/codemirror/0.60/manual.html?rev=738808&view=auto
==============================================================================
--- lenya/trunk/src/modules/editors/resources/codemirror/0.60/manual.html (added)
+++ lenya/trunk/src/modules/editors/resources/codemirror/0.60/manual.html Thu Jan 29 09:49:40 2009
@@ -0,0 +1,469 @@
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <title>CodeMirror user manual</title>
+ <link rel="stylesheet" type="text/css" href="css/docs.css"/>
+ </head>
+ <body>
+ <h1>CodeMirror user manual</h1>
+
+ <h2>Basic Usage</h2>
+
+ <p>Inside the editor, the tab key is used to re-indent the current
+ selection (or the current line when nothing is selected), and
+ pressing enter will, apart from inserting a line break,
+ automatically indent the new line. Pressing control-enter will
+ cause the whole buffer to be re-coloured, which can be helpful
+ when some colouring has become out-of-date without the editor
+ noticing it.</p>
+
+ <p>The editor sports an undo/redo system, accessible with
+ control-z (undo) and control-y (redo). Safari will not allow
+ client scripts to capture control-z presses, but you can use
+ control-backspace instead on that browser.</p>
+
+ <p>The key-combination control-[ triggers a paren-blink: If the
+ cursor is directly after a '(', ')', '[', ']', '{', or '}', the
+ editor looks for the matching character, and highlights these
+ characters for a moment. There is an option to enable this to
+ happen any time the user types something or moves the cursor.</p>
+
+ <p>To use CodeMirror in a document, you should add a script tag to
+ load <a href="js/codemirror.js"><code>codemirror.js</code></a>. This
+ adds two objects to your environment, <code>CodeMirror</code> and
+ <code>CodeMirrorConfig</code>. The first is the interface to the
+ editor, the second can be used to configure it. (Note that this is
+ the only name-space pollution you can expect from CodeMirror --
+ all other cruft is kept inside the IFRAMEs that it creates when
+ you open an editor.)</p>
+
+ <p>To add an editor to a document, you must choose a place, a
+ parser, and a style-sheet for it. For example, to append an
+ XML editor to the body of the document, you do:</p>
+
+ <pre class="code">var editor = new CodeMirror(document.body, {
+ parserfile: "parsexml.js",
+ stylesheet: "xmlcolors.css"
+});</pre>
+
+ <p>The first argument to the <code>CodeMirror</code> constructor
+ can be a DOM node, in which case the editor gets appended to that
+ node, or a function, which will be called with the IFRAME node as
+ argument, and which is expected to place that node somewhere in
+ the document.</p>
+
+ <p>The second (optional) argument is an object that specifies
+ options. A set of default options (see below) is present in the
+ <code>CodeMirrorConfig</code> object, but each instance of the
+ editor can be given a set of specific options to override these
+ defaults. In this case, we specified that the parser should be
+ loaded from the <a
+ href="js/parsexml.js"><code>"parsexml.js"</code></a> file, and
+ that <a href="css/xmlcolors.css"><code>"xmlcolors.css"</code></a>
+ should be used to specify the colours of the code.</p>
+
+ <p>Another example:</p>
+
+ <pre class="code">var editor = new CodeMirror(CodeMirror.replace("inputfield"), {
+ parserfile: ["tokenizejavascript.js", "parsejavascript.js"],
+ path: "lib/codemirror/js/",
+ stylesheet: "lib/codemirror/css/jscolors.css",
+ content: document.getElementById("inputfield").value
+});</pre>
+
+ <p>Here we use the utility function
+ <code>CodeMirror.replace</code> to create a function that will
+ replace a node in the current document (given either directly or
+ by ID) with the editor. We also select the JavaScript parser this
+ time, and give a <code>path</code> option to tell the editor that
+ its files are not located in the same directory as the current
+ HTML page, but in <code>"lib/codemirror/"</code>.</p>
+
+ <p>There is a function
+ <code>CodeMirror.isProbablySupported()</code> that causes some
+ 1998-style browser detection to happen, returning
+ <code>false</code> if CodeMirror is probably not supported on the
+ browser, <code>true</code> if it probably is, and
+ <code>null</code> if it has no idea. As the name suggests, this is
+ not something you can rely on, but it's usually better than
+ nothing.</p>
+
+ <p>Another utility function, <code>CodeMirror.fromTextArea</code>,
+ will, given a textarea node or the id of such a node, hide the
+ textarea and replace it with a CodeMirror frame. If the textarea
+ was part of a form, an <code>onsubmit</code> handler will be
+ registered with this form, which will load the content of the
+ editor into the textarea, so that it can be submitted as normal.
+ This function optionally takes a configuration object as second
+ argument.</p>
+
+ <pre class="code">var editor = CodeMirror.fromTextArea("inputfield", {
+ parserfile: ["tokenizejavascript.js", "parsejavascript.js"],
+ path: "lib/codemirror/js/",
+ stylesheet: "lib/codemirror/css/jscolors.css"
+});</pre>
+
+ <p>The reason that the script path has to be configured is that
+ CodeMirror will load in a bunch of extra files when an editor is
+ created (the parser script, among others). To be able to do this,
+ it has to know where to find them. These are all the JavaScript
+ files that are part of CodeMirror itself:</p>
+
+ <dl>
+ <dt><a href="js/codemirror.js"><code>codemirror.js</code></a></dt>
+ <dd>Main interface, takes care of default configuration and the
+ definition of editor frames. Include this in your HTML
+ document.</dd>
+ <dt><a href="js/editor.js"><code>editor.js</code></a></dt> <dd>The
+ code that takes care of reacting to user input, colouring text,
+ and indenting lines.</dd>
+ <dt><a href="js/util.js"><code>util.js</code></a></dt> <dd>A few
+ generic utility functions.</dd>
+ <dt><a
+ href="js/undo.js"><code>undo.js</code></a></dt>
+ <dd>Implements the undo history for the editor.</dd>
+ <dt><a
+ href="js/stringstream.js"><code>stringstream.js</code></a></dt>
+ <dd>Objects for wrapping the textual input to the parser.</dd>
+ <dt><a href="js/select.js"><code>select.js</code></a></dt> <dd>Some
+ helper utilities for working with selected text and cursor
+ positions.</dd>
+ <dt><a href="js/tokenize.js"><code>tokenize.js</code></a></dt>
+ <dd>Helper framework for writing tokenisers.</dd>
+ </dl>
+
+ <p>Most of these are rather full of comments, which can be useful
+ when you are trying to figure out how they work, but wastes a lot
+ of bandwidth in a production system. Take a look at the
+ description of the <code>basefiles</code> option below if you want
+ to concatenate and minimise the library.</p>
+
+ <p>Apart from these, there are files that implement the various
+ parsers. These all start with either <code>parse</code> or
+ <code>tokenize</code>.</p>
+
+ <h2>Configuration</h2>
+
+ <p>There are three ways to configure CodeMirror:</p>
+
+ <ul>
+ <li>If you define a global <code>CodeMirrorConfig</code> object
+ before loading <a
+ href="js/codemirror.js"><code>codemirror.js</code></a>, the
+ configuration options in that object will override the
+ defaults.</li>
+ <li>By assigning to the properties of the
+ <code>CodeMirrorConfig</code> object, configuration defaults can
+ be overridden after loading <a
+ href="js/codemirror.js"><code>codemirror.js</code></a>.</li>
+ <li>The <code>CodeMirror</code> constructor can be given a second
+ argument, an object, which will override some options for just
+ that editor. Options not mentioned in this object will default to
+ the values in the <code>CodeMirrorConfig</code> object.</li>
+ </ul>
+
+ <p>The options that can be specified are these (most have sensible
+ defaults specified in <a
+ href="js/codemirror.js"><code>codemirror.js</code></a>):</p>
+
+ <dl>
+
+ <dt><code>stylesheet</code></dt><dd>The file name of the style-sheet
+ that should be used to colour the code in the editor frame. See <a
+ href="css/jscolors.css"><code>jscolors.css</code></a> for an
+ example.</dd>
+
+ <dt><code>path</code></dt><dd>The path that is prefixed to
+ script file names when they are loaded into an IFRAME. (Note that
+ this is not applied to the style-sheet file name.)</dd>
+
+ <dt><code>parserfile</code></dt><dd>A file name string, or an
+ array of strings that name the files containing the parser. See
+ below for the interface that such a parser should
+ implement.</dd>
+
+ <dt><code>basefiles</code></dt><dd>An array of strings naming
+ the files containing the base CodeMirror functionality. Defaults
+ to <code>["util.js", "stringstream.js", "select.js", "undo.js",
+ "editor.js", "tokenize.js"]</code>, but if you put them all into
+ a single file to reduce latency, or add some functionality, you
+ might have to adjust that.</dd>
+
+ <dt><code>linesPerPass</code></dt><dd>Specifies the maximum
+ amount of lines that the highlighter tries to colour in one
+ shot. Setting this too high will cause the code to 'freeze' the
+ browser for noticeable intervals. Defaults to 15.</dd>
+
+ <dt><code>passDelay</code></dt><dd>Gives the amount of
+ milliseconds between colouring passes. Defaults to 200.</dd>
+
+ <dt><code>continuousScanning</code></dt><dd>Configure continuous
+ scanning of the document. When <code>false</code>, scanning is
+ disabled. When set to a number, say <code>N</code>, a
+ 'background' process will scan <code>linesPerPass</code> (see
+ above) lines of the document every <code>N</code> milliseconds,
+ regardless of whether anything changed. This makes sure
+ non-local changes propagate through the document, and will help
+ keep everything consistent. It does add extra processing cost,
+ even for an idle editor. Default value is
+ <code>false</code>.</dd>
+
+ <dt><code>autoMatchParens</code></dt><dd>When <code>true</code>,
+ will cause parens to be matched every time a key is pressed or
+ the user clicks on the document. Defaults to <code>false</code>.
+ Might be expensive for big documents.</dd>
+
+ <dt><code>saveFunction</code></dt><dd>If given a function
+ value, that function will be invoked when the user presses
+ control-s. You should advise your Opera users to use
+ control-shift-s instead, since plain control-s will bring up the
+ 'save page' dialog. Defaults to <code>null</code>.</dd>
+
+ <dt><code>undoDepth</code></dt><dd>Maximum length of the undo
+ history. Default is 20.</dd>
+
+ <dt><code>onChange</code></dt><dd>An optional function of zero
+ arguments that gets called whenever the document is changed.
+ Happens at undo-commit time, it is not instantanious.</dd>
+
+ <dt><code>undoDelay</code></dt><dd>When nothing is done in the
+ editor for this amount of milliseconds, pending changes get
+ added to the undo history. Setting this lower will give the undo
+ functionality a finer granularity. Defaults to 800.</dd>
+
+ <dt><code>width</code>, <code>height</code></dt><dd>The size of
+ the editor frame, given as a style-sheet quantities (for example
+ <code>"600px"</code> or <code>"100%"</code>).</dd>
+
+ <dt><code>disableSpellcheck</code></dt><dd>Should the editor
+ disable spell-checking on browsers that support it (Firefox 2+).
+ Default is <code>true</code>, since for most code spell-checking
+ is useless.</dd>
+
+ <dt><code>textWrapping</code></dt><dd>Can be used to disable or
+ enable text-wrapping in the editor frame. Default is
+ <code>true</code>.</dd>
+
+ <dt><code>dumbTabs</code></dt><dd>When this is set to
+ <code>true</code>, pressing tab will insert four spaces rather
+ than re-indenting the selected lines. Default
+ <code>false</code>.</dd>
+
+ <dt><code>readOnly</code></dt><dd>When set to <code>true</code>,
+ the document is not editable.</dd>
+
+ <dt><code>initCallback</code></dt><dd>If set to a function, this
+ will be called (without arguments) after the editor has finished
+ initialising.</dd>
+
+ <dt><code>cursorActivity</code></dt><dd>A function that is
+ called every time the cursor moves, with the top-level node that
+ the cursor is inside or next to as an argument. Can be used to
+ have some controls react to the context of the cursor.</dd>
+
+ <dt><code>activeTokens</code></dt><dd>Can be set to a function
+ that will be called with <code>(spanNode, tokenObject,
+ editor)</code> arguments whenever a new token node is being
+ added to the document. Can be used to do things like add event
+ handlers to nodes. Should <em>not</em> change the DOM structure
+ of the node (so no turning the span into a link), since this
+ will greatly confuse the editor.</dd>
+
+ <dt><code>parserConfig</code></dt><dd>A value that is passed
+ along to the parser to configure it. What this value may look
+ like depends on the parser used.</dd>
+
+ <dt><code>content</code></dt><dd>The starting content of the
+ editor. You'll probably not want to provide a global default for
+ this, but add it to the <code>options</code> object passed to
+ individual editors as they are created.</dd>
+
+ </dl>
+
+ <h2>Programming interface</h2>
+
+ <p>To be as flexible as possible, CodeMirror implements a very
+ plain editable field, without any accompanying buttons, bells, and
+ whistles. <code>CodeMirror</code> objects do, however, provide a
+ number of methods that make it possible to add extra functionality
+ around the editor. <a
+ href="js/mirrorframe.js"><code>mirrorframe.js</code></a> provides a
+ basic example of their usage.</p>
+
+ <dl>
+
+ <dt><code>getCode()</code> →
+ <code>string</code></dt><dd>Returns the current content of the
+ editor, as a string.</dd>
+
+ <dt><code>setCode(string)</code></dt><dd>Replaces the current
+ content of the editor with the given value.</dd>
+
+ <dt><code>focus()</code></dt><dd>Gives focus to the editor
+ frame.</dd>
+
+ <dt><code>currentLine()</code> →
+ <code>number</code></dt><dd>Returns the line on which the cursor
+ is currently sitting. <span class="warn">(Deprecated, see the
+ line-based interface below)</span></dd>
+
+ <dt><code>jumpToLine(number)</code></dt><dd>Moves the cursor to
+ the start of the given line. <span
+ class="warn">(Deprecated)</span></dd>
+
+ <dt><code>selection()</code> →
+ <code>string</code></dt><dd>Returns the text that is currently
+ selected in the editor.</dd>
+
+ <dt><code>replaceSelection(string)</code></dt><dd>Replaces the
+ currently selected text with the given string. Will also cause
+ the editor frame to gain focus.</dd>
+
+ <dt><code>reindent()</code></dt><dd>Automatically re-indent the
+ whole document.</dd>
+
+ <dt><code>getSearchCursor(string, atCursor)</code> →
+ <code>cursor</code></dt><dd>The first argument indicates the
+ string that should be searched for, and the second indicates
+ whether searching should start at the cursor (<code>true</code>)
+ or at the start of the document (<code>false</code>). Returns an
+ object that provides an interface for searching. Call its
+ <code>findNext()</code> method to search for an occurrence of
+ the given string. This returns <code>true</code> if something is
+ found, or <code>false</code> if the end of document is reached.
+ When an occurrence has been found, you can call
+ <code>select()</code> to select it, or
+ <code>replace(string)</code> to replace it with a given string.
+ Note that letting the user change the document, or
+ programmatically changing it in any way except for calling
+ <code>replace</code> on the cursor itself, might cause a cursor
+ object to skip back to the beginning of the document.</dd>
+ </dl>
+
+ <p>For detailed interaction with the content of the editor,
+ CodeMirror exposes a line-oriented interface, which allows you to
+ inspect and manipulate the document line by line. Line handles
+ should be considered opaque (they are in fact the <code>BR</code>
+ nodes at the start of the line), except that the value
+ <code>false</code> (but <em>not</em> <code>null</code>) always
+ denotes an invalid value. Since changing the document might cause
+ some line handles to become invalid, every function that takes
+ them as argument can throw
+ <code>CodeMirror.InvalidLineHandle</code>. These are the relevant
+ methods:</p>
+
+ <dl>
+ <dt><code>cursorPosition(start)</code> →
+ object</dt><dd>Retrieve a {line, character} object representing
+ the cursor position. <code>start</code> defaults to
+ <code>true</code> and determines if the startpoint or the
+ endpoint of the selection is used.</dd>
+ <dt><code>firstLine()</code> → handle</dt><dd>Get the
+ first line of the document.</dd>
+ <dt><code>lastLine()</code> → handle</dt><dd>The
+ last line.</dd>
+ <dt><code>nextLine(line)</code> → handle</dt><dd>Get the
+ line after the given one, or <code>false</code> if that was the
+ last line.</dd>
+ <dt><code>prevLine(line)</code> → handle</dt><dd>Find the
+ line before the given one, return <code>false</code> if that was
+ the first line.</dd>
+ <dt><code>nthLine(number)</code> → handle</dt><dd>Find the
+ Nth line of the document. Note that the first line counts as
+ one, not zero. Returns <code>false</code> if there is no such
+ line.</dd>
+ <dt><code>prevLine(line)</code> → handle</dt><dd>Find the
+ line before the given one, return <code>false</code> if that was
+ the first line.</dd>
+ <dt><code>lineContent(line)</code> →
+ string</dt><dd>Retrieve the content of the line.</dd>
+ <dt><code>setLineContent(line, string)</code></dt><dd>Replace
+ the content of the line with the given string.</dd>
+ <dt><code>selectLines(startLine, startOffset, endLine,
+ endOffset)</code></dt><dd>Move the selection to a specific
+ point. <code>endLine</code> and <code>endOffset</code> can be
+ omitted to just place the cursor somewhere without selecting any
+ text.</dd>
+ <dt><code>insertIntoLine(line, position,
+ text)</code></dt><dd>Insert a piece of text into a line.
+ <code>position</code> can be an integer, specifying the position
+ in the line where the text should be inserted, or the string
+ <code>"end"</code>, for the end of the line.</dd>
+ </dl>
+
+ <h2>Writing a parser</h2>
+
+ <p>A parser is implemented by one or more files (see
+ <code>parserfile</code> above) which, when loaded, add a
+ <code>Parser</code> object to the <code>Editor</code> object
+ defined by <a href="js/editor.js"><code>editor.js</code></a>. This
+ object must support the following interface:</p>
+
+ <dl>
+
+ <dt><code>make(stream)</code></dt><dd>A function that, given a
+ string stream (see <a
+ href="js/stringstream.js"><code>stringstream.js</code></a>),
+ creates a parser. The behaviour of this parser is described
+ below.</dd>
+
+ <dt><code>electricChars</code></dt><dd>An optional string
+ containing the characters that, when typed, should cause the
+ indentation of the current line to be recomputed (for example
+ <code>"{}"</code> for c-like languages).</dd>
+
+ <dt><code>configure(object)</code></dt><dd>An optional function
+ that can be used to configure the parser. If it exists, and an
+ editor is given a <code>parserConfig</code> option, it will be
+ called with the value of that option.</dd>
+
+ <dt><code>firstIndentation(chars, current,
+ direction)</code></dt><dd>An optional function that is used to
+ determine the proper indentation of the first line of a
+ document. When not provided, <code>0</code> is used.</dd>
+ </dl>
+
+ <p>When the <code>make</code> method is called with a string
+ stream, it should return a MochiKit-style iterator: an object with
+ a <code>next</code> method, which will raise
+ <code>StopIteration</code> when it is at its end (see <a
+ href="http://bob.pythonmac.org/archives/2005/07/06/iteration-in-javascript/">this</a>
+ for details). This iterator, when called, will consume input from
+ the string stream, and produce a token object.</p>
+
+ <p>Token objects represent a single significant piece of the text
+ that is being edited. A token object must have a
+ <code>value</code> property holding the text it stands for, and a
+ <code>style</code> property with the CSS class that should be used
+ to colour this element. This can be anything, except that any
+ whitespace at the start of a line should <em>always</em> have
+ class <code>"whitespace"</code>: The editor must be able to
+ recognize these when it indents lines. Furthermore, each newline
+ character <em>must</em> have its own separate token, which has an
+ <code>indentation</code> property holding a function that can be
+ used to determine the proper indentation level for the next line.
+ This function optionally takes the text in the first token of the
+ next line, the current indentation of the line, and the
+ 'direction' of the indentation as arguments, which it can use to
+ adjust the indentation level. The direction argument is only
+ useful for modes in which lines do not have a fixed indentation,
+ and can be modified by multiple tab presses. It is
+ <code>null</code> for 'default' indentations (like what happens
+ when the user presses enter), <code>true</code> for regular tab
+ presses, and <code>false</code> for control-tab or shift-tab.</p>
+
+ <p>So far this should be pretty easy. The hard part is that this
+ iterator must also have a <code>copy</code> method. This method,
+ called without arguments, returns a function representing the
+ current state of the parser. When this state function is later
+ called with a string stream as its argument, it returns a parser
+ object that resumes parsing using the old state and the new input
+ stream. It may assume that only one parser is active at a time,
+ and can clobber the state of the old parser if it wants.</p>
+
+ <p>For examples, see <a
+ href="js/parsejavascript.js"><code>parsejavascript.js</code></a>,
+ <a href="js/parsexml.js"><code>parsexml.js</code></a>, and <a
+ href="js/parsecss.js"><code>parsecss.js</code></a>.</p>
+
+ </body>
+</html>
Added: lenya/trunk/src/modules/editors/resources/codemirror/0.60/mixedtest.html
URL: http://svn.apache.org/viewvc/lenya/trunk/src/modules/editors/resources/codemirror/0.60/mixedtest.html?rev=738808&view=auto
==============================================================================
--- lenya/trunk/src/modules/editors/resources/codemirror/0.60/mixedtest.html (added)
+++ lenya/trunk/src/modules/editors/resources/codemirror/0.60/mixedtest.html Thu Jan 29 09:49:40 2009
@@ -0,0 +1,52 @@
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <script src="js/codemirror.js" type="text/javascript"></script>
+ <title>CodeMirror: HTML mixed-mode demonstration</title>
+ <link rel="stylesheet" type="text/css" href="css/docs.css"/>
+ </head>
+ <body style="padding: 20px;">
+
+<p>This is a simple demonstration of the HTML mixed-mode indentation
+module for <a href="index.html">CodeMirror</a>. Script tags use the JS
+parser, style tags use the CSS parser.</p>
+
+<div style="border: 1px solid black; padding: 3px;">
+<textarea id="code" cols="120" rows="30">
+<html>
+ <head>
+ <title>HTML Example</title>
+ <script type="text/javascript">
+ function foo(bar, baz) {
+ alert("quux");
+ return bar + baz + 1;
+ }
+ </script>
+ <style type="text/css">
+ div.border {
+ border: 1px solid black;
+ padding: 3px;
+ }
+ #foo code {
+ font-family: courier, monospace;
+ font-size: 80%;
+ color: #448888;
+ }
+ </style>
+ </head>
+ <body>
+ <p>Duh</p>
+ </body>
+</html>
+</textarea>
+</div>
+
+<script type="text/javascript">
+ var editor = CodeMirror.fromTextArea('code', {
+ height: "350px",
+ parserfile: ["parsexml.js", "parsecss.js", "tokenizejavascript.js", "parsejavascript.js", "parsehtmlmixed.js"],
+ stylesheet: ["css/xmlcolors.css", "css/jscolors.css", "css/csscolors.css"],
+ path: "js/"
+ });
+</script>
+ </body>
+</html>
Added: lenya/trunk/src/modules/editors/resources/codemirror/0.60/sparqltest.html
URL: http://svn.apache.org/viewvc/lenya/trunk/src/modules/editors/resources/codemirror/0.60/sparqltest.html?rev=738808&view=auto
==============================================================================
--- lenya/trunk/src/modules/editors/resources/codemirror/0.60/sparqltest.html (added)
+++ lenya/trunk/src/modules/editors/resources/codemirror/0.60/sparqltest.html Thu Jan 29 09:49:40 2009
@@ -0,0 +1,41 @@
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <script src="js/codemirror.js" type="text/javascript"></script>
+ <title>CodeMirror: Sparql demonstration</title>
+ <link rel="stylesheet" type="text/css" href="css/docs.css"/>
+ </head>
+ <body style="padding: 20px;">
+
+<p>Demonstration of <a href="index.html">CodeMirror</a>'s Sparql
+highlighter.</p>
+
+<div class="border">
+<textarea id="code" cols="120" rows="30">
+PREFIX a: <http://www.w3.org/2000/10/annotation-ns#>
+PREFIX dc: <http://purl.org/dc/elements/1.1/>
+PREFIX foaf: <http://xmlns.com/foaf/0.1/>
+
+# Comment!
+
+SELECT ?given ?family
+WHERE {
+ ?annot a:annotates <http://www.w3.org/TR/rdf-sparql-query/> .
+ ?annot dc:creator ?c .
+ OPTIONAL {?c foaf:given ?given ;
+ foaf:family ?family } .
+ FILTER isBlank(?c)
+}
+</textarea>
+</div>
+
+<script type="text/javascript">
+ var editor = CodeMirror.fromTextArea('code', {
+ height: "250px",
+ parserfile: "parsesparql.js",
+ stylesheet: "css/sparqlcolors.css",
+ path: "js/"
+ });
+</script>
+
+ </body>
+</html>
Added: lenya/trunk/src/modules/editors/resources/codemirror/0.60/story.html
URL: http://svn.apache.org/viewvc/lenya/trunk/src/modules/editors/resources/codemirror/0.60/story.html?rev=738808&view=auto
==============================================================================
--- lenya/trunk/src/modules/editors/resources/codemirror/0.60/story.html (added)
+++ lenya/trunk/src/modules/editors/resources/codemirror/0.60/story.html Thu Jan 29 09:49:40 2009
@@ -0,0 +1,642 @@
+<html xmlns="http://www.w3.org/1999/xhtml">
+ <head>
+ <title>Implementing a syntax-higlighting JavaScript editor in JavaScript</title>
+ <style type="text/css">
+ body {
+ font-family: tahoma, arial, sans-serif;
+ padding: 3em 8em;
+ }
+ h1 {
+ text-align: center;
+ margin: 0;
+ }
+ h2 {
+ font-size: 130%;
+ }
+ code {
+ font-family: courier, monospace;
+ font-size: 80%;
+ color: #448888;
+ }
+ p {
+ margin: 1em 0;
+ }
+ pre.code {
+ margin: 1.1em 12px;
+ border: 1px solid #CCCCCC;
+ padding: .4em;
+ font-family: courier, monospace;
+ }
+ </style>
+ <link rel="stylesheet" type="text/css" href="css/jscolors.css"/>
+ </head>
+ <body>
+ <h1 style="font-size: 180%;">Implementing a syntax-higlighting JavaScript editor in JavaScript</h1>
+ <h1 style="font-size: 110%;">or</h1>
+ <h1 style="font-size: 130%; margin-bottom: 3em;">A brutal odyssey to the dark side of the DOM tree</h1>
+
+ <p style="font-size: 80%">
+ <b>Topic</b>: JavaScript, advanced browser weirdness, cool programming techniques<br/>
+ <b>Audience</b>: Programmers, especially JavaScript programmers<br/>
+ <b>Author</b>: Marijn Haverbeke<br/>
+ <b>Date</b>: May 24th 2007
+ </p>
+
+ <p>In one of his (very informative) <a
+ href="http://video.yahoo.com/?t=t&p=douglascrockford">video
+ lectures</a>, Douglas Crockford remarks that writing JavaScript for
+ the web is 'programming in a hostile environment'. I had done my fair
+ share of weird workarounds, and even occasonally gave up an on idea
+ entirely because browsers just wouldn't support it, but before this
+ project I never really realized just how powerless a programmer can be
+ in the face of buggy, incompatible, and poorly designed platforms.</p>
+
+ <p>The plan was not ridiculously ambitious. I wanted to 'enhance' a
+ textarea to the point where writing code in it is pleasant. This meant
+ automatic indentation and, if possible at all, syntax highlighting.</p>
+
+ <p>In this document I describe the story of implementing this, for your
+ education and amusement. A demonstration of the resulting program,
+ along with the source code, can be found at <a
+ href="index.html">my website</a>.</p>
+
+ <h2>Take one: Only indentation</h2>
+
+ <p>The very first attempt merely added auto-indentation to a textarea
+ element. It would scan backwards through the content of the area,
+ starting from the cursor, until it had enough information to decide
+ how to indent the current line. It took me a while to figure out a
+ decent model for indenting JavaScript code, but in the end this seems
+ to work:</p>
+
+ <ul>
+ <li>Code that sits inside a block is indented one unit (generally two
+ spaces) more than the statement or brace that opened the block.</li>
+ <li>A statement that is continued to the next line is indented one unit
+ more than the line that starts the statement.</li>
+ <li>When dealing with lists of arguments or the content of array and
+ object literals there are two possible models. If there is any text
+ directly after the opening brace, bracket, or parenthesis,
+ subsequent lines are aligned with this opening character. If the
+ opening character is followed by a newline (optionally with whitespace
+ or comments before it), the next line is indented one unit further
+ than the line that started the list.</li>
+ <li>And, obviously, if a statement follows another statement it is
+ indented the same amount as the one before it.</li>
+ </ul>
+
+ <p>When scanning backwards through code one has to take string values,
+ comments, and regular expressions (which are delimited by slashes)
+ into account, because braces and semicolons and such are not
+ significant when they appear inside them. Single-line ('//') comments
+ turned out to be rather inefficient to check for when doing a
+ backwards scan, since every time you encounter a newline you have to
+ go on to the next newline to determine whether this line ends in a
+ comment or not. Regular expressions are even worse ― without
+ contextual information they are impossible to distinguish from the
+ division operator, and I didn't get them working in this first
+ version.</p>
+
+ <p>To find out which line to indent, and to make sure that adding or
+ removing whitespace doesn't cause the cursor to jump in strange ways,
+ it is necessary to determine which text the user has selected. Even
+ though I was working with just a simple textarea at this point, this
+ was already a bit of a headache.</p>
+
+ <p>On W3C-standards-respecting browsers, textarea nodes have
+ <code>selectionStart</code> and <code>selectionEnd</code>
+ properties which nicely give you the amount of characters before
+ the start and end of the selection. Great!</p>
+
+ <p>Then, there is Internet Explorer. Internet Explorer also has an API
+ for looking at and manipulating selections. It gives you information
+ such as a detailed map of the space the selected lines take up on the
+ screen, in pixels, and of course the text inside the selection. It
+ does, however, not give you much of a clue on where the selection is
+ located in the document.</p>
+
+ <p>After some experimentation I managed to work out an elaborate
+ method for getting something similar to the
+ <code>selectionStart</code> and <code>selectionEnd</code> values
+ in other browsers. It worked like this:</p>
+
+ <ul>
+ <li>Get the <code>TextRange</code> object corresponding to the selection.</li>
+ <li>Record the length of the text inside it.</li>
+ <li>Make another <code>TextRange</code> that covers the whole textarea element.</li>
+ <li>Set the start of the first <code>TextRange</code> to the start of the second one.</li>
+ <li>Again get the length of the text in the first object.</li>
+ <li>Now <code>selectionEnd</code> is the second length, and <code>selectionStart</code> is
+ the second minus the first one.</li>
+ </ul>
+
+ <p>That seemed to work, but when resetting the selection after modifying
+ the content of the textarea I ran into another interesting feature of
+ these <code>TextRange</code>s: You can move their endpoints by a given number of
+ characters, which is useful when trying to set a cursor at the Nth
+ character of a textarea, but in this context, newlines are <em>not</em>
+ considered to be characters, so you'll always end up one character too
+ far for every newline you passed. Of course, you can count newlines
+ and compensate for this (though it is still not possible to position
+ the cursor right in front of a newline). Sheesh.</p>
+
+ <p>After ragging on Internet Explorer for a while, let us move on and rag
+ on Firefox a bit. It turns out that, in Firefox, getting and setting
+ the text content of a DOM element is unexplainably expensive,
+ especially when there is a lot of text involved. As soon as I tried to
+ use my indentation code to indent itself (some 400 lines), I found
+ myself waiting for over four seconds every time I pressed enter. That
+ seemed a little slow.</p>
+
+ <h2><code>designMode</code> it is</h2>
+
+ <p>The solution was obvious: Since the text inside a textarea can only be
+ manipulated as one single big string, I had to spread it out over
+ multiple nodes. How do you spread editable content over multiple
+ nodes? Right! <code>designMode</code> or <code>contentEditable</code>.</p>
+
+ <p>Now I wasn't entirely naive about <code>designMode</code>, I had been looking
+ into writing a non-messy WYSIWYG editor before, and at that time I had
+ concluded two things:</p>
+
+ <ul>
+ <li>It is impossible to prevent the user from inserting whichever HTML
+ junk he wants into the document.</li>
+ <li>In Internet Explorer, it is extemely hard to get a good view
+ on what nodes the user has selected.</li>
+ </ul>
+
+ <p>Basically, the good folks at Microsoft designed a really bad interface
+ for putting editable documents in pages, and the other browsers, not
+ wanting to be left behind, more or less copied that. And there isn't
+ much hope for a better way to do this appearing anytime soon. Wise
+ people probably use a Flash movie or (God forbid) a Java applet for
+ these kind of things, though those are not without drawbacks either.</p>
+
+ <p>Anyway, seeing how using an editable document would also make syntax
+ highlighting possible, I foolishly went ahead. There is something
+ perversely fascinating about trying to build a complicated system on a
+ lousy, unsuitable platform.</p>
+
+ <h2>A parser</h2>
+
+ <p>How does one do decent syntax highlighting? A very simple scanning can
+ tell the difference between strings, comments, keywords, and other
+ code. But this time I wanted to actually be able to recognize regular
+ expressions, so that I didn't have any blatant incorrect behaviour
+ anymore.</p>
+
+ <p>That brought me to the idea of doing a serious parse on the code. This
+ would not only make detecting regular expressions much easier, it
+ would also give me detailed information about the code, which can be
+ used to determine proper indentation levels, and to make subtle
+ distinctions in colouring, for example the difference between variable
+ names and property names.</p>
+
+ <p>And hey, when we're parsing the whole thing, it would even be possible
+ to make a distinction between local and global variables, and colour
+ them differently. If you've ever programmed JavaScript you can
+ probably imagine how useful this would be ― it is ridiculously easy
+ to accidentally create global instead of local variables. I don't
+ consider myself a JavaScript rookie anymore, but it was (embarrasingly
+ enough) only this week that I realized that my habit of typing <code>for
+ (name in object) ...</code> was creating a global variable <code>name</code>, and that
+ I should be typing <code>for (var name in object) ...</code> instead.</p>
+
+ <p>Re-parsing all the code the user has typed in every time he hits a key
+ is obviously not feasible. So how does one combine on-the-fly
+ highlighting with a serious parser? One option would be to split the
+ code into top-level statements (functions, variable definitions, etc.)
+ and parse these separately. This is horribly clunky though, especially
+ considering the fact that modern JavaScripters often put all the code
+ in a file in a single big object or function to prevent namespace
+ pollution.</p>
+
+ <p>I have always liked continuation-passing style and generators. So the
+ idea I came up with is this: An interruptable, resumable parser. This
+ is a parser that does not run through a whole document at once, but
+ parses on-demand, a little bit at a time. At any moment you can create
+ a copy of its current state, which can be resumed later. You start
+ parsing at the top of the code, and keep going as long as you like,
+ but throughout the document, for example at every end of line, you
+ store a copy of the current parser state. Later on, when line 106
+ changes, you grab the interrupted parser that was stored at the end of
+ line 105, and use it to re-parse line 106. It still knows exactly what
+ the context was at that point, which local variables were defined,
+ which unfinished statements were encountered, and so on.</p>
+
+ <p>But that, unfortunately, turned out to be not quite as easy as it
+ sounds.</p>
+
+ <h2>The DOM nodes underfoot</h2>
+
+ <p>Of course, when working inside an editable frame we don't just have to
+ deal with text. The code will be represented by some kind of DOM tree.
+ My first idea was to set the <code>whitespace: pre</code> style for the frame and
+ try to work with mostly text, with the occasional coloured <code>span</code>
+ element. It turned out that support for <code>whitespace: pre</code> in browsers,
+ especially in editable frames, is so hopelessly glitchy that this was
+ unworkable.</p>
+
+ <p>Next I tried a series of <code>div</code> elements, one per line, with <code>span</code>
+ elements inside them. This seemed to nicely reflect the structure of
+ the code in a shallowly hierarchical way. I soon realized, however,
+ that my code would be much more straightfoward when using no hierarchy
+ whatsoever ― a series of <code>span</code>s, with <code>br</code> tags at the end of every
+ line. This way, the DOM nodes form a flat sequence that corresponds to
+ the sequence of the text ― just extract text from <code>span</code> nodes and
+ substitute newlines for <code>br</code> nodes.</p>
+
+ <p>It would be a shame if the editor would fall apart as soon as someone
+ pastes some complicated HTML into it. I wanted it to be able to deal
+ with whatever mess it finds. This means using some kind of
+ HTML-normalizer that takes arbitrary HTML and flattens it into a
+ series of <code>br</code>s and <code>span</code> elements that contain a single text node.
+ Just like the parsing process, it would be best if this did not have
+ to done to the entire buffer every time something changes.</p>
+
+ <p>It took some banging my head against my keyboard, but I found a very
+ nice way to model this. It makes heavy use of generators, for which I
+ used <a href="http://www.mochikit.com">MochiKit</a>'s iterator
+ framework. Bob Ippolito explains the concepts in this library very
+ well in his <a
+ href="http://bob.pythonmac.org/archives/2005/07/06/iteration-in-javascript/">blog
+ post</a> about it. (Also notice some of the dismissive comments at the
+ bottom of that post. They say "I don't think I really want to learn
+ this, so I'll make up some silly reason to condemn it.")</p>
+
+ <p>The highlighting process consists of the following elements:
+ normalizing the DOM tree, extracting the text from the DOM tree,
+ tokenizing this text, parsing the tokens, and finally adjusting the
+ DOM nodes to reflect the structure of the code.</p>
+
+ <p>The first two, I put into a single generator. It scans the DOM
+ tree, fixing anything that is not a simple top-level
+ <code>span</code> or <code>br</code>, and it produces the text
+ content of the nodes (or a newline in case of a <code>br</code>)
+ as its output ― each time it is called, it yields a string.
+ Continuation passing style was a good way to model this process in
+ an iterator, which has to be processed one step at a time. Look at
+ this simplified version:</p>
+
+ <pre class="code"><span class="js-keyword">function</span> <span class="js-variable">traverseDOM</span>(<span class="js-variabledef">start</span>){
+ <span class="js-keyword">var</span> <span class="js-variabledef">cc</span> = <span class="js-keyword">function</span>(){<span class="js-variable">scanNode</span>(<span class="js-localvariable">start</span>, <span class="js-variable">stop</span>);};
+ <span class="js-keyword">function</span> <span class="js-variabledef">stop</span>(){
+ <span class="js-localvariable">cc</span> = <span class="js-localvariable">stop</span>;
+ <span class="js-keyword">throw</span> <span class="js-variable">StopIteration</span>;
+ }
+ <span class="js-keyword">function</span> <span class="js-variabledef">yield</span>(<span class="js-variabledef">value</span>, <span class="js-variabledef">c</span>){
+ <span class="js-localvariable">cc</span> = <span class="js-localvariable">c</span>;
+ <span class="js-keyword">return</span> <span class="js-localvariable">value</span>;
+ }
+
+ <span class="js-keyword">function</span> <span class="js-variabledef">scanNode</span>(<span class="js-variabledef">node</span>, <span class="js-variabledef">c</span>){
+ <span class="js-keyword">if</span> (<span class="js-localvariable">node</span>.<span class="js-property">nextSibling</span>)
+ <span class="js-keyword">var</span> <span class="js-variabledef">nextc</span> = <span class="js-keyword">function</span>(){<span class="js-localvariable">scanNode</span>(<span class="js-localvariable">node</span>.<span class="js-property">nextSibling</span>, <span class="js-localvariable">c</span>);};
+ <span class="js-keyword">else</span>
+ <span class="js-keyword">var</span> <span class="js-variabledef">nextc</span> = <span class="js-localvariable">c</span>;
+
+ <span class="js-keyword">if</span> (<span class="js-comment">/* node is proper span element */</span>)
+ <span class="js-keyword">return</span> <span class="js-localvariable">yield</span>(<span class="js-localvariable">node</span>.<span class="js-property">firstChild</span>.<span class="js-property">nodeValue</span>, <span class="js-localvariable">nextc</span>);
+ <span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-comment">/* node is proper br element */</span>)
+ <span class="js-keyword">return</span> <span class="js-localvariable">yield</span>(<span class="js-string">"\n"</span>, <span class="js-localvariable">nextc</span>);
+ <span class="js-keyword">else</span>
+ <span class="js-comment">/* flatten node, yield its textual content */</span>;
+ }
+
+ <span class="js-keyword">return</span> {<span class="js-property">next</span>: <span class="js-keyword">function</span>(){<span class="js-keyword">return</span> <span class="js-localvariable">cc</span>();}};
+}</pre>
+
+ <p>The variable <code>c</code> stands for 'continuation', and <code>cc</code> for 'current
+ continuation' ― that last variable is used to store the function to
+ continue with, when yielding a value to the outside world. Every time
+ control leaves this function, it has to make sure that <code>cc</code> is set to
+ a suitable value, which is what <code>yield</code> and <code>stop</code> take care of.</p>
+
+ <p>The object that is returned contains a <code>next</code> method, which is
+ MochiKit's idea of an iterator, and the initial continuation just
+ throws a <code>StopIteration</code>, which is how MochiKit signals that an
+ iterator has reached its end.</p>
+
+ <p>The first lines of <code>scanNode</code> extend the continuation with the task of
+ scanning the next node, if there is a next node. The rest of the
+ function decides what kind of value to <code>yield</code>. Note that this is a
+ rather trivial example of this technique, since the process of going
+ through these nodes is basically linear (it was much, much more
+ complex in earlier versions), but still the trick with the
+ continuations makes the code shorter and, for those in the know,
+ clearer than the equivalent 'storing the iterator state in variables'
+ approach.</p>
+
+ <p>The next iterator that the input passes through is the
+ tokenizer. Well, actually, there is another iterator in between
+ that isolates the tokenizer from the fact that the DOM traversal
+ yields a bunch of separate strings, and presents them as a single
+ character stream (with a convenient <code>peek</code> operation),
+ but this is not a very interesting one. What the tokenizer returns
+ is a stream of token objects, each of which has a
+ <code>value</code>, its textual content, a <code>type</code>, like
+ <code>"variable"</code>, <code>"operator"</code>, or just itself,
+ <code>"{"</code> for example, in the case of significant
+ punctuation or special keywords. They also have a
+ <code>style</code>, which is used later by the highlighter to give
+ their <code>span</code> elements a class name (the parser will
+ still adjust this in some cases).</p>
+
+ <p>At first I assumed the parser would have to talk back to the
+ tokenizer about the current context, in order to be able to
+ distinguish those accursed regular expressions from divisions, but
+ it seems that regular expressions are only allowed if the previous
+ (non-whitespace, non-comment) token was either an operator, a
+ keyword like <code>new</code> or <code>throw</code>, or a specific
+ kind of punctuation (<code>"[{}(,;:"</code>) that indicates a new
+ expression can be started here. This made things considerably
+ easier, since the 'regexp or no regexp' question could stay
+ entirely within the tokenizer.</p>
+
+ <p>The next step, then, is the parser. It does not do a very
+ thorough job because, firstly, it has to be fast, and secondly, it
+ should not go to pieces when fed an incorrect program. So only
+ superficial constructs are recognized, keywords that resemble each
+ other in syntax, such as <code>while</code> and <code>if</code>,
+ are treated in precisely the same way, as are <code>try</code> and
+ <code>else</code> ― the parser doesn't mind if an
+ <code>else</code> appears without an <code>if</code>. Stuff that
+ binds variables, <code>var</code>, <code>function</code>, and
+ <code>catch</code> to be precise, is treated with more care,
+ because the parser wants to know about local variables.</p>
+
+ <p>Inside the parser, three kinds of context are stored. Firstly, a set
+ of known local variables, which is used to adjust the style of
+ variable tokens. Every time the parser enters a function, a new set of
+ variables is created. If there was already such a set (entering an
+ inner function), a pointer to the old one is stored in the new one. At
+ the end of the function, the current variable set is 'popped' off and
+ the previous one is restored.</p>
+
+ <p>The second kind of context is the lexical context, this keeps track of
+ whether we are inside a statement, block, or list. Like the variable
+ context, it also forms a stack of contexts, with each one containing a
+ pointer to the previous ones so that they can be popped off again when
+ they are finished. This information is used for indentation. Every
+ time the parser encounters a newline token, it attaches the current
+ lexical context and a 'copy' of itself (more about that later) to this
+ token.</p>
+
+ <p>The third context is a continuation context. This parser does not use
+ straight continuation style, instead it uses a stack of actions that
+ have to be performed. These actions are simple functions, a kind of
+ minilanguage, they act on tokens, and decide what kind of new actions
+ should be pushed onto the stack. Here are some examples:</p>
+
+<pre class="code"><span class="js-keyword">function</span> <span class="js-variable">expression</span>(<span class="js-variabledef">type</span>){
+ <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> in <span class="js-variable">atomicTypes</span>) <span class="js-variable">cont</span>(<span class="js-variable">maybeoperator</span>);
+ <span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">"function"</span>) <span class="js-variable">cont</span>(<span class="js-variable">functiondef</span>);
+ <span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">"("</span>) <span class="js-variable">cont</span>(<span class="js-variable">pushlex</span>(<span class="js-string">"list"</span>), <span class="js-variable">expression</span>, <span class="js-variable">expect</span>(<span class="js-string">")"</span>), <span class="js-variable">poplex</span>);
+ <span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">"operator"</span>) <span class="js-variable">cont</span>(<span class="js-variable">expression</span>);
+ <span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">"["</span>) <span class="js-variable">cont</span>(<span class="js-variable">pushlex</span>(<span class="js-string">"list"</span>), <span class="js-variable">commasep</span>(<span class="js-variable">expression</span>), <span class="js-variable">expect</span>(<span class="js-string">"]"</span>), <span class="js-variable">poplex</span>);
+ <span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">"{"</span>) <span class="js-variable">cont</span>(<span class="js-variable">pushlex</span>(<span class="js-string">"list"</span>), <span class="js-variable">commasep</span>(<span class="js-variable">objprop</span>), <span class="js-variable">expect</span>(<span class="js-string">"}"</span>), <span class="js-variable">poplex</span>);
+ <span class="js-keyword">else</span> <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">"keyword c"</span>) <span class="js-variable">cont</span>(<span class="js-variable">expression</span>);
+}
+
+<span class="js-keyword">function</span> <span class="js-variable">block</span>(<span class="js-variabledef">type</span>){
+ <span class="js-keyword">if</span> (<span class="js-localvariable">type</span> == <span class="js-string">"}"</span>) <span class="js-variable">cont</span>();
+ <span class="js-keyword">else</span> <span class="js-variable">pass</span>(<span class="js-variable">statement</span>, <span class="js-variable">block</span>);
+}</pre>
+
+ <p>The function <code>cont</code> (for continue), will push the actions it is given
+ onto the stack (in reverse order, so that the first one will be popped
+ first). Actions such as <code>pushlex</code> and <code>poplex</code> merely adjust the
+ lexical environment, while others, such as <code>expression</code> itself, do
+ actual parsing. <code>pass</code>, as seen in <code>block</code>, is similar to <code>cont</code>, but
+ it does not 'consume' the current token, so the next action will again
+ see this same token. In <code>block</code>, this happens when the function
+ determines that we are not at the end of the block yet, so it pushes
+ the <code>statement</code> function which will interpret the current token as the
+ start of a statement.</p>
+
+ <p>These actions are called by a 'driver' function, which filters out the
+ whitespace and comments, so that the parser actions do not have to
+ think about those, and keeps track of some things like the indentation
+ of the current line and the column at which the current token ends,
+ which are stored in the lexical context and used for indentation.
+ After calling an action, if the action called <code>cont</code>, this driver
+ function will return the current token, if <code>pass</code> (or nothing) was
+ called, it will immediately continue with the next action.</p>
+
+ <p>This goes to show that it is viable to write a quite elaborate
+ minilanguage in a macro-less language like JavaScript. I don't think
+ it would be possible to do something like this without closures (or
+ similarly powerful abstraction) though, I've certainly never seen
+ anything like it in Java code.</p>
+
+ <p>The way a 'copy' of the parser was produced shows a nice usage
+ of closures. Like with the DOM transformer shown above, most of
+ the local state of the parser is held in a closure produced by
+ calling <code>parse(stream)</code>. The function
+ <code>copy</code>, which is local to the parser function, produces
+ a new closure, with copies of all the relevant variables:</p>
+
+<pre class="code"><span class="js-keyword">function</span> <span class="js-variable">copy</span>(){
+ <span class="js-keyword">var</span> <span class="js-variabledef">_context</span> = <span class="js-variable">context</span>, <span class="js-variabledef">_lexical</span> = <span class="js-variable">lexical</span>, <span class="js-variabledef">_actions</span> = <span class="js-variable">copyArray</span>(<span class="js-variable">actions</span>);
+
+ <span class="js-keyword">return</span> <span class="js-keyword">function</span>(<span class="js-variabledef">_tokens</span>){
+ <span class="js-variable">context</span> = <span class="js-localvariable">_context</span>;
+ <span class="js-variable">lexical</span> = <span class="js-localvariable">_lexical</span>;
+ <span class="js-variable">actions</span> = <span class="js-variable">copyArray</span>(<span class="js-localvariable">_actions</span>);
+ <span class="js-variable">tokens</span> = <span class="js-localvariable">_tokens</span>;
+ <span class="js-keyword">return</span> <span class="js-variable">parser</span>;
+ };
+}</pre>
+
+ <p>Where <code>parser</code> is the object that contains the <code>next</code> (driver)
+ function, and a reference to this <code>copy</code> function. When the function
+ that <code>copy</code> produces is called with a token stream as argument, it
+ updates the local variables in the parser closure, and returns the
+ corresponding iterator object.</p>
+
+ <p>Moving on, we get to the last stop in this chain of generators, the
+ actual highlighter. You can view this one as taking two streams as
+ input, on the one hand there is the stream of tokens from the parser,
+ and on the other hand there is the DOM tree as left by the DOM
+ transformer. If everything went correctly, these two should be
+ synchronized. The highlighter can look at the current token, see if
+ the <code>span</code> in the DOM tree corresponds to it (has the same text
+ content, and the correct class), and if not it can chop up the DOM
+ nodes to conform to the tokens.</p>
+
+ <p>Every time the parser yields a newline token, the highligher
+ encounters a <code>br</code> element in the DOM stream. It takes the copy of the
+ parser and the lexical context from this token and attaches them to
+ the DOM node. This way, a new highlighting process can be started from
+ that node by re-starting the copy of the parser with a new token
+ stream, which reads tokens from the DOM nodes starting at that <code>br</code>
+ element, and the indentation code can use the lexical context
+ information to determine the correct indentation at that point.</p>
+
+ <h2>Selection woes</h2>
+
+ <p>All the above can be done using the DOM interface that all major
+ browsers have in common, and which is relatively free of weird bugs
+ and abberrations. However, when the user is typing in new code, this
+ must also be highlighted. For this to happen, the program must know
+ where the cursor currently is, and because it mucks up the DOM tree,
+ it has to restore this cursor position after doing the highlighting.</p>
+
+ <p>Re-highlighting always happens per line, because the copy of the
+ parser is stored only at the end of lines. Doing this every time the
+ user presses a key is terribly slow and obnoxious, so what I did was
+ keep a list of 'dirty' nodes, and as soon as the user didn't type
+ anyting for 300 milliseconds the program starts re-highlighting these
+ nodes. If it finds more than ten lines must be re-parsed, it does only
+ ten and waits another 300 milliseconds before it continues, this way
+ the browser never freezes up entirely.</p>
+
+ <p>As mentioned earlier, Internet Explorer's selection model is not the
+ most practical one. My attempts to build a wrapper that makes it look
+ like the W3C model all stranded. In the end I came to the conclusion
+ that I only needed two operations:</p>
+
+ <ul>
+ <li>Creating a selection 'snapshot' that can be restored after
+ highlighting, in such a way that it still works if some of the nodes
+ that were selected are replaced by other nodes with the same
+ size but a different structure.</li>
+ <li>Finding the top-level node around or before the cursor, to mark it
+ dirty or to insert indentation whitespace at the start of that line.</li>
+ </ul>
+
+ <p>It turns out that the pixel-based selection model that Internet
+ Explorer uses, which always seemed completely ludricrous to me, is
+ perfect for the first case. Since the DOM transformation (generally)
+ does not change the position of things, storing the pixel offsets of
+ the selection makes it possible to restore that same selection, never
+ mind what happened to the underlying DOM structure.</p>
+
+ <p>[Later addition: Note that this, due to the very random design
+ of the <a
+ href="http://msdn2.microsoft.com/en-us/library/ms535872(VS.85).aspx#">TextRange
+ interface</a>, only really works when the whole selection falls
+ within the visible part of the document.]</p>
+
+ <p>Doing the same with the W3C selection model is a lot harder. What I
+ ended up with was this:</p>
+
+ <ul>
+ <li>Create an object pointing to the nodes at the start and end of the
+ selection, and the offset within those nodes. This is basically the
+ information that the <code>Range</code> object gives you.</li>
+ <li>Make references from these nodes back to that object.</li>
+ <li>When replacing (part of) a node with another one, check for such a
+ reference, and when it is present, check whether this new node will
+ get the selection. If it does, move the reference from the old to the
+ new node, if it does not, adjust the offset in the selection object to
+ reflect the fact that part of the old node has been replaced.</li>
+ </ul>
+
+ <p>Now in the second case (getting the top-level node at the
+ cursor) the Internet Explorer cheat does not work. In the W3C
+ model this is rather easy, you have to do some creative parent-
+ and sibling-pointer following to arrive at the correct top-level
+ node, but nothing weird. In Internet Explorer, all we have to go
+ on is the <code>parentElement</code> method on a
+ <code>TextRange</code>, which gives the first element that
+ completely envelops the selection. If the cursor is inside a text
+ node, this is good, that text node tells us where we are. If the
+ cursor is between nodes, for example between two <code>br</code>
+ nodes, you get to top-level node itself back, which is remarkably
+ useless. In cases like this I stoop to a rather ugly hack (which
+ fortunately turned out to be acceptably fast) ― I create a
+ temporary empty <code>span</code> with an ID inside the selection,
+ get a reference to this <code>span</code> by ID, take its
+ <code>previousSibling</code>, and remove it again.</p>
+
+ <p>Unfortunately, Opera's selection implementation is buggy, and it
+ will give wildly incorrect <code>Range</code> objects when the cursor
+ is between two nodes. This is a bit of a showstopper, and until I find
+ a workaround for that or it gets fixed, the highlighter doesn't work
+ properly in Opera.</p>
+
+ <p>Also, when one presses enter in a <code>designMode</code>
+ document in Firefox or Opera, a <code>br</code> tag is inserted.
+ In Internet Explorer, pressing enter causes some maniacal gnome to
+ come out and start wrapping all the content before and after the
+ cursor in <code>p</code> tags. I suppose there is something to be
+ said for that, in principle, though if you saw the tag soup of
+ <code>font</code>s and nested paragraphs Internet Explorer
+ generates you would soon enough forget all about principle.
+ Anyway, getting unwanted <code>p</code> tags slowed the
+ highlighter down terribly ― it had to overhaul the whole
+ DOM tree to remove them again, every time the user pressed enter.
+ Fortunately I could fix this by capturing the enter presses and
+ manually inserting a <code>br</code> tag at the cursor.</p>
+
+ <p>On the subject of Internet Explorer's tag soup, here is an interesting
+ anecdote: One time, when testing the effect that modifying the content
+ of a selection had, I inspected the DOM tree and found a <code>"/B"</code>
+ element. This was not a closing tag, there are no closing tags in the
+ DOM tree, just elements. The <code>nodeName</code> of this element was actually
+ <code>"/B"</code>. That was when I gave up any notions of ever understanding the
+ profound mystery that is Internet Explorer.</p>
+
+ <h2>Closing thoughts</h2>
+
+ <p>Well, I despaired at times, but I did end up with a working JavaScript
+ editor. I did not keep track of the amount of time I wasted on this,
+ but I would estimate it to be around fifty hours. Finding workarounds
+ for browser bugs can be a terribly nonlinear process. I just spent
+ half a day working on a weird glitch in Firefox that caused the cursor
+ in the editable frame to be displayed 3/4 line too high when it was at
+ the very end of the document. Then I found out that setting the
+ style.display of the iframe to "block" fixed this (why not?). I'm
+ amazed how often issues that seem hopeless do turn out to be
+ avoidable, even if it takes hours of screwing around and some truly
+ non-obvious ideas.</p>
+
+ <p>For a lot of things, JavaScript + DOM elements are a surprisingly
+ powerful platform. Simple interactive documents and forms can be
+ written in browsers with very little effort, generally less than with
+ most 'traditional' platforms (Java, Win32, things like WxWidgets).
+ Libraries like Dojo (and a similar monster I once wrote myself) even
+ make complex, composite widgets workable. However, when applications
+ go sufficiently beyond the things that browsers were designed for, the
+ available APIs do not give enough control, are nonstandard and buggy,
+ and are often poorly designed. Because of this, writing such
+ applications, when it is even possible, is <em>painful</em> process.</p>
+
+ <p>And who likes pain? Sure, when finding that crazy workaround,
+ subdueing the damn browser, and getting everything to work, there
+ is a certain macho thrill. But one can't help wondering how much
+ easier things like preventing the user from pasting pictures in
+ his source code would be on another platform. Maybe something like
+ Silverlight or whatever other new browser plugin gizmos people are
+ pushing these days will become the way to solve things like this
+ in the future. But, personally, I would prefer for those browser
+ companies to put some real effort into things like cleaning up and
+ standardising shady things like <code>designMode</code>, fixing
+ their bugs, and getting serious about ECMAScript 4.</p>
+
+ <p>Which is probably not realistically going to happen anytime soon.</p>
+
+ <hr/>
+
+ <p>Some interesting projects similar to this:</p>
+
+ <ul>
+ <li><a href="http://gpl.internetconnection.net/vi/">vi clone</a></li>
+ <li><a href="http://robrohan.com/projects/9ne/">Emacs clone</a></li>
+ <li><a href="http://codepress.sourceforge.net/">CodePress</a></li>
+ <li><a href="http://www.codeide.com">CodeIDE</a></li>
+ <li><a href="http://www.cdolivet.net/editarea">EditArea</a></li>
+ </ul>
+
+ <hr/>
+
+ <p>If you have any remarks, criticism, or hints related to the
+ above, drop me an e-mail at <a
+ href="mailto:marijnh@gmail.com">marijnh@gmail.com</a>. If you say
+ something generally interesting, I'll include your reaction here
+ at the bottom of this page.</p>
+
+ </body>
+</html>
Added: lenya/trunk/src/modules/editors/resources/css/codemirror.css
URL: http://svn.apache.org/viewvc/lenya/trunk/src/modules/editors/resources/css/codemirror.css?rev=738808&view=auto
==============================================================================
--- lenya/trunk/src/modules/editors/resources/css/codemirror.css (added)
+++ lenya/trunk/src/modules/editors/resources/css/codemirror.css Thu Jan 29 09:49:40 2009
@@ -0,0 +1,5 @@
+@import url(/modules/editors/codemirror/0.60/css/xmlcolors.css);
+
+.editbox {
+ font-size: 9pt;
+}
\ No newline at end of file
Added: lenya/trunk/src/modules/editors/resources/css/sourceEditor.css
URL: http://svn.apache.org/viewvc/lenya/trunk/src/modules/editors/resources/css/sourceEditor.css?rev=738808&view=auto
==============================================================================
--- lenya/trunk/src/modules/editors/resources/css/sourceEditor.css (added)
+++ lenya/trunk/src/modules/editors/resources/css/sourceEditor.css Thu Jan 29 09:49:40 2009
@@ -0,0 +1,6 @@
+.editorContentContainer {
+ clear: both;
+ width: auto;
+ border: solid 1px #CCCCCC;
+ margin: .5em 0;
+}
Added: lenya/trunk/src/modules/editors/resources/javascript/sourceEditor/codemirror.js
URL: http://svn.apache.org/viewvc/lenya/trunk/src/modules/editors/resources/javascript/sourceEditor/codemirror.js?rev=738808&view=auto
==============================================================================
--- lenya/trunk/src/modules/editors/resources/javascript/sourceEditor/codemirror.js (added)
+++ lenya/trunk/src/modules/editors/resources/javascript/sourceEditor/codemirror.js Thu Jan 29 09:49:40 2009
@@ -0,0 +1,44 @@
+/*
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
+
+var editor;
+
+function insertEditor() {
+ editor = CodeMirror.fromTextArea('editorContent', {
+ path: "/modules/editors/codemirror/0.60/js/",
+ parserfile: "parsexml.js",
+ stylesheet: "/modules/editors/css/codemirror.css",
+ height: "400px",
+ onChange: saveContent
+ });
+}
+
+function saveContent() {
+ if (editor) {
+ // Firefox
+ document.forms.oneform.content.innerHTML = editor.getCode();
+ // Safari
+ document.forms.oneform.content.value = editor.getCode();
+
+ document.getElementById("save1").disabled = null;
+ document.getElementById("save2").disabled = null;
+ }
+}
+
+function indent() {
+ editor.reindent();
+}
Added: lenya/trunk/src/modules/editors/resources/javascript/sourceEditor/lenya_glue.js
URL: http://svn.apache.org/viewvc/lenya/trunk/src/modules/editors/resources/javascript/sourceEditor/lenya_glue.js?rev=738808&view=auto
==============================================================================
--- lenya/trunk/src/modules/editors/resources/javascript/sourceEditor/lenya_glue.js (added)
+++ lenya/trunk/src/modules/editors/resources/javascript/sourceEditor/lenya_glue.js Thu Jan 29 09:49:40 2009
@@ -0,0 +1,56 @@
+org.apache.lenya.editors.setObjectData = function(objectData, windowName) {
+ var currentUsecase = usecaseMap[windowName];
+ var snippet = org.apache.lenya.editors.generateContentSnippet(currentUsecase, objectData);
+ org.apache.lenya.editors.insertContent(
+ document.forms['oneform'].elements['content'],
+ snippet
+ );
+ usecaseMap[windowName] = undefined; // we're done!
+ objectData[windowName] = undefined; // we're done!
+}
+
+org.apache.lenya.editors.getObjectData = function(windowName) {
+ return objectData[windowName];
+}
+
+function triggerUsecase(usecase) {
+ var windowName = org.apache.lenya.editors.generateUniqueWindowName();
+ var selectedText = org.apache.lenya.editors.getSelectedText(document.forms[0].elements['content']);
+ switch (usecase) {
+
+ case org.apache.lenya.editors.USECASE_INSERTLINK:
+ objectData[windowName] = new org.apache.lenya.editors.ObjectData({
+ url : "",
+ text : selectedText,
+ title : ""
+ });
+ break;
+
+ case org.apache.lenya.editors.USECASE_INSERTIMAGE:
+ objectData[windowName] = new org.apache.lenya.editors.ObjectData({
+ url : "",
+ text : selectedText,
+ title : "",
+ width : "",
+ height: ""
+ });
+ break;
+
+ case org.apache.lenya.editors.USECASE_INSERTASSET:
+ objectData[windowName] = new org.apache.lenya.editors.ObjectData({
+ url : "",
+ text : selectedText,
+ title : ""
+ })
+ break;
+ }
+ org.apache.lenya.editors.openUsecaseWindow(usecase, windowName);
+ usecaseMap[windowName] = usecase;
+ /* alert("Stored values for new window " + windowName + ":\n"
+ + "objectData[windowName] = '" + objectData[windowName] + "'\n"
+ + "usecaseMap[windowName] = '" + usecaseMap[windowName] + "'"
+ ); */
+}
+
+var objectData = new Array();
+var usecaseMap = new Array();
Modified: lenya/trunk/src/modules/editors/usecases/forms/oneform.jx
URL: http://svn.apache.org/viewvc/lenya/trunk/src/modules/editors/usecases/forms/oneform.jx?rev=738808&r1=738807&r2=738808&view=diff
==============================================================================
--- lenya/trunk/src/modules/editors/usecases/forms/oneform.jx (original)
+++ lenya/trunk/src/modules/editors/usecases/forms/oneform.jx Thu Jan 29 09:49:40 2009
@@ -27,10 +27,13 @@
<page:title><i18n:text>Edit</i18n:text></page:title>
<page:head>
<script type="text/javascript" src="/modules/editors/javascript/org.apache.lenya.editors.js"/>
- <script type="text/javascript" src="/modules/editors/javascript/oneform_lenya_glue.js"/>
+ <script type="text/javascript" src="/modules/editors/javascript/sourceEditor/lenya_glue.js"/>
<script type="text/javascript" src="/modules/editors/javascript/disablebackspace.js"/>
+ <script type="text/javascript" src="/modules/editors/codemirror/0.60/js/codemirror.js"/>
+ <script type="text/javascript" src="/modules/editors/javascript/sourceEditor/codemirror.js"/>
+ <link rel="stylesheet" type="text/css" href="/modules/editors/css/sourceEditor.css"/>
</page:head>
- <page:body>
+ <page:body onload="insertEditor()">
<jx:import uri="fallback://lenya/modules/usecase/templates/messages.jx"/>
@@ -81,9 +84,13 @@
</div>
<div style="clear: both;">
- <input type="submit" value="Save" name="submit" i18n:attr="value"/>
+ <input id="save1" disabled="disabled" type="submit" value="Save" name="submit" i18n:attr="value"/>
<input type="submit" value="Cancel" name="cancel" i18n:attr="value"/>
+ <!--
<input type="submit" value="Reformat" name="reformat" i18n:attr="value"/>
+ -->
+ <input type="button" value="Reformat" name="reformat" i18n:attr="value"
+ onclick="indent()"/>
<input type="button" value="insertLink.submit" i18n:attr="value"
onclick="triggerUsecase('insertLink')"/>
<input type="button" value="insertAsset.insertImage" i18n:attr="value"
@@ -92,12 +99,11 @@
onclick="triggerUsecase('insertAsset')"/>
</div>
- <div style="clear: both;">
+ <div class="editorContentContainer">
<jx:choose>
<jx:when test="${usecase.getParameter('content')}">
- <textarea name="content" cols="120" rows="80">
- <jx:out value="${usecase.getParameter('content')}"/>
- </textarea>
+ <textarea name="content" style="display: none"/>
+ <textarea id="editorContent" cols="120" rows="80">${usecase.getParameter('content')}</textarea>
</jx:when>
<jx:otherwise>
<cinclude:includexml>
@@ -108,7 +114,7 @@
</div>
<div>
- <input type="submit" value="Save" name="submit" i18n:attr="value"/>
+ <input id="save2" disabled="disabled" type="submit" value="Save" name="submit" i18n:attr="value"/>
<input type="submit" value="Cancel" name="cancel" i18n:attr="value"/>
</div>
Modified: lenya/trunk/src/modules/editors/usecases/forms/oneform.xsl
URL: http://svn.apache.org/viewvc/lenya/trunk/src/modules/editors/usecases/forms/oneform.xsl?rev=738808&r1=738807&r2=738808&view=diff
==============================================================================
--- lenya/trunk/src/modules/editors/usecases/forms/oneform.xsl (original)
+++ lenya/trunk/src/modules/editors/usecases/forms/oneform.xsl Thu Jan 29 09:49:40 2009
@@ -32,7 +32,8 @@
<xsl:template match="/">
<input type="hidden" name="namespaces"><xsl:attribute name="value"><xsl:apply-templates mode="namespaces" /></xsl:attribute></input>
- <textarea name="content" cols="120" rows="80">
+ <textarea name="content" style="display: none"/>
+ <textarea id="editorContent" cols="120" rows="80">
<xsl:apply-templates mode="mixedcontent" />
</textarea>
</xsl:template>
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@lenya.apache.org
For additional commands, e-mail: commits-help@lenya.apache.org