You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@systemml.apache.org by Matthias Boehm <mb...@us.ibm.com> on 2016/04/03 06:34:37 UTC
Discussion SYSTEMML-593 MLContext Resign
thanks Deron for initiating the discussion around the rework of our
MLContext API (https://issues.apache.org/jira/browse/SYSTEMML-593). Here
are a couple of thoughts:
(1) Simplicity: Given that the primary usecase of MLContext calls a script
exactly once, I'm wondering if the separation into Script, ScriptFactory,
ScriptExecutor and MLContext adds unnecessary complexity by requiring more
code to setup. It would be great to see old vs new examples side by side.
Also rather than introducing another exception class, couldn't we just
reuse DMLException by making it an uncaught exception?
(2) Compilation: ScriptExecutor would also be yet another replicate of our
compilation chain (beside DMLScript, Debugger, JMLC, MLContext). Please,
keep in mind that we are about to consolidate this, centralizing these
calls via a configurable compilation chain because it really becomes a
maintenance nightmare (as recently seen when we reworked our thread-local
configuration management).
(3) Data Representation / Converters: Making data conversions and
input/output handling easier is certainly useful. However, isn't this new
class hierarchy redundant to our already existing hierarchy of Data,
MatrixObject, FrameObject, and ScalarObject?
(4) Consolidating MLContext and JMLC: This is a good idea since MLContext
is anyway "derived" from JMLC and both rely on the same concepts for
input/output handling and program modification (see JMLCUtils). Down the
road I would like to see a convergence to something like Script and
PreparedScript in the spirit of JDBC's Statement and PreparedStatement
(btw, that's how we created JMLC). Let's keep in mind that there are
already a number of users working against both MLContext and JMLC, so we
should support them separately until our major 1.0 release.
Regards,
Matthias
Re: Discussion SYSTEMML-593 MLContext Resign
Posted by Matthias Boehm <mb...@us.ibm.com>.
Thanks Deron - those are all valid arguments. However, the summary of
exceptions actually makes a good case for cleaning up the entire exception
handling. I created SYSTEMML-616 to track this. Let's decide on MLContext
once this is done.
Regards,
Matthias
From: Deron Eriksson <de...@gmail.com>
To: dev@systemml.incubator.apache.org
Date: 04/04/2016 02:38 PM
Subject: Re: Discussion SYSTEMML-593 MLContext Resign
Hi Matthias,
On Sat, Apr 2, 2016 at 9:34 PM, Matthias Boehm <mb...@us.ibm.com> wrote:
>
> Also rather than introducing another exception class, couldn't we just
> reuse DMLException by making it an uncaught exception?
>
Thank you for the feedback. My thoughts for creating an uncaught
MLContextException center around a few issues:
1) The exception happens through the MLContext API, so to me the naming
makes sense.
2) Placement of the new exception is in a org.apache.sysml.api.mlcontext
package, along with all other primary classes that the user interacts with
when using the MLContext API. This standardization is useful for things
such as Javadocs (all API classes are under this single package) and code
maintainability (all primary API classes are centralized in a single
place).
3) This exception would handle errors that happen at the MLContext level
and additionally be a wrapper for deeper levels of exceptions.
4) The naming of DMLException can be confusing to an end user. If a user
runs PyDML and gets a DMLException, this is a little disconcerting.
5) It seems to me the cleanest least-invasive solution that gives the user
the simplest experience is a new runtime exception. Given the
interrelations of existing exception classes, almost all of which are
actually caught exceptions, a new runtime exception avoids any retrofitting
needed of existing exception handling. In the MLContext API, you just catch
all existing caught exceptions and rethrow as MLContextExceptions, or
generate new MLContextExceptions where appropriate.
Throwable
|-Exception
|-DMLException (323 matches)
|-AppException (not used)
|-DMLDebuggerException (13 matches)
|-DMLRuntimeException (3,689 matches)
|-CacheException (82 matches)
|-CacheIOException (26 matches)
|-CacheStatusException (18 matches)
|-DMLScriptException (39 matches)
|-DMLUnsupportedOperationException (1,068 matches)
|-HopsException (748 matches)
|-LanguageException (390 matches)
|-LopsException (596 matches)
|-ParseException (320 matches)
|-DMLParseException (50 matches)
|-RuntimeException
|-PackageRuntimeException (70 matches)
Since there are already 15 existing exception classes, adding an additional
exception class for a programmatic API doesn't seem to me to be an
unnecessary proliferation of classes.
Deron
Re: Discussion SYSTEMML-593 MLContext Resign
Posted by Deron Eriksson <de...@gmail.com>.
Hi Matthias,
On Sat, Apr 2, 2016 at 9:34 PM, Matthias Boehm <mb...@us.ibm.com> wrote:
>
> Also rather than introducing another exception class, couldn't we just
> reuse DMLException by making it an uncaught exception?
>
Thank you for the feedback. My thoughts for creating an uncaught
MLContextException center around a few issues:
1) The exception happens through the MLContext API, so to me the naming
makes sense.
2) Placement of the new exception is in a org.apache.sysml.api.mlcontext
package, along with all other primary classes that the user interacts with
when using the MLContext API. This standardization is useful for things
such as Javadocs (all API classes are under this single package) and code
maintainability (all primary API classes are centralized in a single place).
3) This exception would handle errors that happen at the MLContext level
and additionally be a wrapper for deeper levels of exceptions.
4) The naming of DMLException can be confusing to an end user. If a user
runs PyDML and gets a DMLException, this is a little disconcerting.
5) It seems to me the cleanest least-invasive solution that gives the user
the simplest experience is a new runtime exception. Given the
interrelations of existing exception classes, almost all of which are
actually caught exceptions, a new runtime exception avoids any retrofitting
needed of existing exception handling. In the MLContext API, you just catch
all existing caught exceptions and rethrow as MLContextExceptions, or
generate new MLContextExceptions where appropriate.
Throwable
|-Exception
|-DMLException (323 matches)
|-AppException (not used)
|-DMLDebuggerException (13 matches)
|-DMLRuntimeException (3,689 matches)
|-CacheException (82 matches)
|-CacheIOException (26 matches)
|-CacheStatusException (18 matches)
|-DMLScriptException (39 matches)
|-DMLUnsupportedOperationException (1,068 matches)
|-HopsException (748 matches)
|-LanguageException (390 matches)
|-LopsException (596 matches)
|-ParseException (320 matches)
|-DMLParseException (50 matches)
|-RuntimeException
|-PackageRuntimeException (70 matches)
Since there are already 15 existing exception classes, adding an additional
exception class for a programmatic API doesn't seem to me to be an
unnecessary proliferation of classes.
Deron
Re: Discussion SYSTEMML-593 MLContext Resign
Posted by Frederick R Reiss <fr...@us.ibm.com>.
Luciano Resende <lu...@gmail.com> wrote on 04/02/2016 10:20:06 PM:
> From: Luciano Resende <lu...@gmail.com>
> To: dev@systemml.incubator.apache.org
> Cc: Deron Eriksson/San Francisco/IBM@IBMUS
> Date: 04/02/2016 10:20 PM
> Subject: Re: Discussion SYSTEMML-593 MLContext Resign
>
> On Sat, Apr 2, 2016 at 9:34 PM, Matthias Boehm <mb...@us.ibm.com> wrote:
>
> >
> >
> > thanks Deron for initiating the discussion around the rework of our
> > MLContext API (https://issues.apache.org/jira/browse/SYSTEMML-593).
Here
> > are a couple of thoughts:
> >
> > (1) Simplicity: Given that the primary usecase of MLContext calls a
script
> > exactly once, I'm wondering if the separation into Script,
ScriptFactory,
> > ScriptExecutor and MLContext adds unnecessary complexity by requiring
more
> > code to setup. It would be great to see old vs new examples side by
side.
> > Also rather than introducing another exception class, couldn't we just
> > reuse DMLException by making it an uncaught exception?
> >
>
> Simplicity and easy of use will, in my view, dictate use adoption. One
way
> I usually do to accomplish this is to start by building an app or test
> scenario and use that to model the user experience when using the APIs.
> This might help identify how many steps the user needs to handle before
> actually using the api (e.g. Script, ScriptFactory, etc) and if those
steps
> are really necessary... But, to be successful with this approach, you
> really need to come with a clean mind, and really think as a user trying
to
> use the API.
>
> Having said that, sometimes we do need a fine grained api, but sometimes
> that might be used internally (private apis) and be exposed to the user
> with a more coarse grained api that hides much of the details and expose
a
> simple programming model to the user.
>
> Also, I like the option to have a fluent API, which makes the code more
> readable and easy to use.
I think the proposed approach moves in the direction of greater
cleanliness, doesn't add complexity, and makes some tasks less complex. For
example, it's easier to write an application that calls out to multiple DML
scripts if you have an abstraction for a "prepared" script, a la a prepared
SQL statement.
Regarding Matthias's question about old vs new examples of the "run a
single script once use" case, here you go:
OLD:
String str = "x=$X; A=read($Ain); B=A+x; write(B, 'temp');";
ml.reset();
ml.registerInput("X", 10);
ml.registerInput("Ain", csv, sc.textFile("m.csv"));
ml.registerOutput("B");
MLOutput output = ml.execute(str);
MLMatrix matrix = output.getMLMatrix(ml, sqlContext, "B");
NEW:
String str = "x=$X; A=read($Ain); B=A+x; write(B, 'temp');";
Script script = ScriptFactory.dml(str);
script.in("$X", 10).in("A", sc.textFile("m.csv")).regOut("B");
ml.execute(script);
BinaryBlockMatrix matrix = script.out("B");
The new version has about the same amount of code.
One question I have about the design, though: What happens to outputs of
the most recent run when someone runs a new script? Are we leaking
persisted RDDs or unpersisting RDDs that the calling application might have
a pointer to?
>
> https://en.wikipedia.org/wiki/Fluent_interface#Java
>
>
> >
> > (2) Compilation: ScriptExecutor would also be yet another replicate of
our
> > compilation chain (beside DMLScript, Debugger, JMLC, MLContext).
Please,
> > keep in mind that we are about to consolidate this, centralizing these
> > calls via a configurable compilation chain because it really becomes a
> > maintenance nightmare (as recently seen when we reworked our
thread-local
> > configuration management).
Agreed, we should include the necessary refactoring to centralize the
top-level control of the compilation chain into the work proposed in this
design doc.
> >
> > (3) Data Representation / Converters: Making data conversions and
> > input/output handling easier is certainly useful. However, isn't this
new
> > class hierarchy redundant to our already existing hierarchy of Data,
> > MatrixObject, FrameObject, and ScalarObject?
> >
>
> Is there a way to hide some of the input/output registration calls by
some
> convention over api calls ?
>
>
> >
> > (4) Consolidating MLContext and JMLC: This is a good idea since
MLContext
> > is anyway "derived" from JMLC and both rely on the same concepts for
> > input/output handling and program modification (see JMLCUtils). Down
the
> > road I would like to see a convergence to something like Script and
> > PreparedScript in the spirit of JDBC's Statement and PreparedStatement
> > (btw, that's how we created JMLC). Let's keep in mind that there are
> > already a number of users working against both MLContext and JMLC, so
we
> > should support them separately until our major 1.0 release.
> >
> >
> > Regards,
> > Matthias
> >
>
>
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
Re: Discussion SYSTEMML-593 MLContext Resign
Posted by Luciano Resende <lu...@gmail.com>.
On Sat, Apr 2, 2016 at 9:34 PM, Matthias Boehm <mb...@us.ibm.com> wrote:
>
>
> thanks Deron for initiating the discussion around the rework of our
> MLContext API (https://issues.apache.org/jira/browse/SYSTEMML-593). Here
> are a couple of thoughts:
>
> (1) Simplicity: Given that the primary usecase of MLContext calls a script
> exactly once, I'm wondering if the separation into Script, ScriptFactory,
> ScriptExecutor and MLContext adds unnecessary complexity by requiring more
> code to setup. It would be great to see old vs new examples side by side.
> Also rather than introducing another exception class, couldn't we just
> reuse DMLException by making it an uncaught exception?
>
Simplicity and easy of use will, in my view, dictate use adoption. One way
I usually do to accomplish this is to start by building an app or test
scenario and use that to model the user experience when using the APIs.
This might help identify how many steps the user needs to handle before
actually using the api (e.g. Script, ScriptFactory, etc) and if those steps
are really necessary... But, to be successful with this approach, you
really need to come with a clean mind, and really think as a user trying to
use the API.
Having said that, sometimes we do need a fine grained api, but sometimes
that might be used internally (private apis) and be exposed to the user
with a more coarse grained api that hides much of the details and expose a
simple programming model to the user.
Also, I like the option to have a fluent API, which makes the code more
readable and easy to use.
https://en.wikipedia.org/wiki/Fluent_interface#Java
>
> (2) Compilation: ScriptExecutor would also be yet another replicate of our
> compilation chain (beside DMLScript, Debugger, JMLC, MLContext). Please,
> keep in mind that we are about to consolidate this, centralizing these
> calls via a configurable compilation chain because it really becomes a
> maintenance nightmare (as recently seen when we reworked our thread-local
> configuration management).
>
> (3) Data Representation / Converters: Making data conversions and
> input/output handling easier is certainly useful. However, isn't this new
> class hierarchy redundant to our already existing hierarchy of Data,
> MatrixObject, FrameObject, and ScalarObject?
>
Is there a way to hide some of the input/output registration calls by some
convention over api calls ?
>
> (4) Consolidating MLContext and JMLC: This is a good idea since MLContext
> is anyway "derived" from JMLC and both rely on the same concepts for
> input/output handling and program modification (see JMLCUtils). Down the
> road I would like to see a convergence to something like Script and
> PreparedScript in the spirit of JDBC's Statement and PreparedStatement
> (btw, that's how we created JMLC). Let's keep in mind that there are
> already a number of users working against both MLContext and JMLC, so we
> should support them separately until our major 1.0 release.
>
>
> Regards,
> Matthias
>
--
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/
Re: Discussion SYSTEMML-593 MLContext Resign
Posted by Deron Eriksson <de...@gmail.com>.
Hi Matthias,
On Sat, Apr 2, 2016 at 9:34 PM, Matthias Boehm <mb...@us.ibm.com> wrote:
>
> (1) Simplicity: Given that the primary usecase of MLContext calls a script
> exactly once, I'm wondering if the separation into Script, ScriptFactory,
> ScriptExecutor and MLContext adds unnecessary complexity by requiring more
> code to setup. It would be great to see old vs new examples side by side.
>
>
With the introduction of notebooks, we might need to be calling scripts
more than once in the near future.
The current API is very procedural. For example, there are 24 execute() and
executeScript() methods on the MLContext class. Encapsulating concepts such
as 'Script' can bring a significant amount of power, flexibility, and
actually simplicity to the system. These 24 execute methods can be replaced
by a single execute(Script script) method on MLContext. We can also include
a second execute method, execute(Script script, ScriptExecutor
scriptExecutor) so that an advanced user can easily modify the execution
steps. The existing API is not extensible in this way. A user would need to
modify the source code of the MLContext
executeUsingSimplifiedCompilationChain in order to do this, whereas with
this redesign a user can subclass ScriptExecutor and modify it as needed.
If a user wants to use the default execution, the user can just call the
MLContext execute(Script script) method.
This is an example of simplifying the end user experience (replace 24
execute methods with 1 execute method). However, it is also nice to add
extensibility (via a second execute method that takes a ScriptExecutor) for
advanced cases. A normal user probably would not really care about
ScriptExecutor and wouldn't need to use it.
As another potential benefit of Script objects, we could conceivably do
things like encapsulate a namespace into the script object, or have a
script object encapsulate a list of other script objects, etc.
As a further example of simplicity, the current 18 registerInput methods on
MLContext can be replaced by a single Script in(String str, Object obj)
method. Chaining method calls by returning a Script object from the method
call (script.in("$a", 5).in("$b", true)...) is a convenient way of setting
multiple inputs in a single line of code consisting of multiple method
calls.
In terms of user interaction, at its most basic, a script consists of some
text (string), has a type (DML or PYDML), can have inputs, and can have
outputs. If things are broken down any further and we lose the
encapsulation, then we have a procedural API with lots of registerInputs
and executes, for example.
WRT factories, they can be a very useful design pattern. Here's an example
of creating a DML script from a String, a DML script from a file, a PYDML
script from a URL, and a PYDML script from an input stream. That's 4
scripts from four different sources in four lines of code with no
boilerplate or boolean flags.
Script scr1 = ScriptFactory.createDMLScriptFromString("print('hi');");
Script scr2 = ScriptFactory.createDMLScriptFromFile("ex.dml");
Script scr3 = ScriptFactory.createPYDMLScriptFromUrl("
http://example.com/alg.pydml");
Script scr4 = ScriptFactory.createPYDMLScriptFromInputStream(myInputStream);
As for other code examples, we can replace MLContext's 18 registerInputs:
registerInput(String, DataFrame)
registerInput(String, DataFrame, boolean)
registerInput(String, JavaPairRDD<LongWritable, Text>, String, long, long,
long, FileFormatProperties)
registerInput(String, JavaPairRDD<MatrixIndexes, MatrixBlock>, long, long)
registerInput(String, JavaPairRDD<MatrixIndexes, MatrixBlock>, long, long,
int, int)
registerInput(String, JavaPairRDD<MatrixIndexes, MatrixBlock>, long, long,
int, int, long)
registerInput(String, JavaPairRDD<MatrixIndexes, MatrixBlock>,
MatrixCharacteristics)
registerInput(String, JavaRDD<String>, String)
registerInput(String, JavaRDD<String>, String, boolean, String, boolean,
double)
registerInput(String, JavaRDD<String>, String, boolean, String, boolean,
double, long, long, long)
registerInput(String, JavaRDD<String>, String, long, long)
registerInput(String, JavaRDD<String>, String, long, long, long)
registerInput(String, MLMatrix)
registerInput(String, RDD<String>, String)
registerInput(String, RDD<String>, String, boolean, String, boolean, double)
registerInput(String, RDD<String>, String, boolean, String, boolean,
double, long, long, long)
registerInput(String, RDD<String>, String, long, long)
registerInput(String, RDD<String>, String, long, long, long)
with one Script in(String, Object) method (and perhaps another in() method
for a Scala immutable Map).
We can replace the existing 24 execute and executeScript methods on
MLContext:
execute(String)
execute(String, ArrayList<String>)
execute(String, ArrayList<String>, boolean)
execute(String, ArrayList<String>, boolean, String)
execute(String, ArrayList<String>, String)
execute(String, boolean)
execute(String, boolean, String)
execute(String, HashMap<String, String>)
execute(String, HashMap<String, String>, boolean)
execute(String, HashMap<String, String>, boolean, String)
execute(String, HashMap<String, String>, String)
execute(String, Map<String, String>)
execute(String, Map<String, String>, boolean)
execute(String, String)
execute(String, String[])
execute(String, String[], boolean)
execute(String, String[], boolean, String)
execute(String, String[], String)
executeScript(String)
executeScript(String, HashMap<String, String>)
executeScript(String, HashMap<String, String>, String)
executeScript(String, Map<String, String>)
executeScript(String, Map<String, String>, String)
executeScript(String, String)
with one MLContext execute(Script script) method.
I think the following example from the design doc is quite concise. We have
(1) a String of DML, (2) we create a Script object for it, (3) we set
inputs and outputs (both input parameters and binding to variables), (4) we
execute the script, (5) we get back the results.
1) String str = "x=$X; A=read($Ain); B=A+x; write(B, 'temp');";
2) Script script = ScriptFactory.dml(str);
3) script.in("$X", 10).in("A", sc.textFile("m.csv")).regOut("B");
4) ml.execute(script);
5) BinaryBlockMatrix bbm = script.out("B");
These 5 steps can even be combined into a single line through method
chaining, which is interesting but not very readable. However, we could
combine 1&2 above and 4&5 above to provide a 3-line example that's still
readable.
1) Script script = ScriptFactory.dml("x=$X; A=read($Ain); B=A+x; write(B,
'temp');");
2) script.in("$X", 10).in("A", sc.textFile("m.csv")).regOut("B");
3) BinaryBlockMatrix bbm = ml.execute(script).out("B");
The only thing that seems 'extra' to me is registering the output ("B"). I
don't know if it's possible internally, but it might be nice to not require
a user to register an output.
The equivalent using the current API would be something like:
ml.reset();
ml.registerInput("A", sc.textFile("m.csv"), "csv");
HashMap<String, String> cmdLineArgs = new HashMap<String, String>();
cmdLineArgs.put("X", "10");
cmdLineArgs.put("Ain", " ");
ml.registerOutput("B");
MLOutput output = ml.executeScript("x=$X; A=read($Ain); B=A+x; write(B,
'temp');", cmdLineArgs);
JavaPairRDD<MatrixIndexes, MatrixBlock> binaryBlockedRDD =
output.getBinaryBlockedRDD("B");
Although these examples are basically logically equivalent and do the same
thing, personally I have a much easier time conceptualizing the example
using a Script object.
Deron