You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@any23.apache.org by "Simone Tripodi (Created) (JIRA)" <ji...@apache.org> on 2012/04/01 20:32:26 UTC
[jira] [Created] (ANY23-71) improve the current CLI engine
improve the current CLI engine
------------------------------
Key: ANY23-71
URL: https://issues.apache.org/jira/browse/ANY23-71
Project: Apache Any23
Issue Type: Improvement
Affects Versions: 0.7.0
Reporter: Simone Tripodi
Fix For: 0.7.0
Actual CLI - even if nicely working - can be improved in therms of of both internal architecture and user interface.
I see two main "issues" on current CLI:
* on UI, the CLI exposes internal details, since the {{ToolRunner}} requires the classname of the tool has to be executed;
* on internals, each Tool has to parse the the chunk of the command line, which can be automated.
So my proposal is to automate, via the already working plugins discovery, the CLI arguments parsing AND implementing a svn/git commands-based alike interface.
My preferred choice for that is [JCommande|http://www.jcommander.org/] because:
* it allows binding CLI arguments to Java properties via Annotations - no more manual parsing;
* it already supports a complex syntax to implement [commands|http://jcommander.org/#Complex];
* commands aliases can be expressed via annotations - no more needs to expose internals;
Patch with proposal is coming
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ANY23-71) improve the current CLI engine
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ANY23-71?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13246120#comment-13246120 ]
Hudson commented on ANY23-71:
-----------------------------
Integrated in Any23-trunk #160 (See [https://builds.apache.org/job/Any23-trunk/160/])
Improved CLI support, introduced usage of JCommander. The commandline parsing logic has been moved into ToolRunner as well as the production of help and version messages. This commit includes a set of minor fixes on the patch provided by stripodi (Simone Tripodi) and it is related to issue #ANY23-71. (Revision 1308786)
Result = UNSTABLE
mostarda :
Files :
* /incubator/any23/trunk/core/pom.xml
* /incubator/any23/trunk/core/src/main/java/org/apache/any23/cli/ExtractorDocumentation.java
* /incubator/any23/trunk/core/src/main/java/org/apache/any23/cli/MicrodataParser.java
* /incubator/any23/trunk/core/src/main/java/org/apache/any23/cli/MimeDetector.java
* /incubator/any23/trunk/core/src/main/java/org/apache/any23/cli/PluginVerifier.java
* /incubator/any23/trunk/core/src/main/java/org/apache/any23/cli/Rover.java
* /incubator/any23/trunk/core/src/main/java/org/apache/any23/cli/Tool.java
* /incubator/any23/trunk/core/src/main/java/org/apache/any23/cli/ToolRunner.java
* /incubator/any23/trunk/core/src/main/java/org/apache/any23/cli/Version.java
* /incubator/any23/trunk/core/src/main/java/org/apache/any23/cli/VocabPrinter.java
* /incubator/any23/trunk/core/src/main/java/org/apache/any23/writer/WriterRegistry.java
* /incubator/any23/trunk/core/src/main/resources/default-configuration.properties
* /incubator/any23/trunk/core/src/test/java/org/apache/any23/cli/ExtractorDocumentationTest.java
* /incubator/any23/trunk/core/src/test/java/org/apache/any23/cli/RoverTest.java
* /incubator/any23/trunk/core/src/test/java/org/apache/any23/cli/ToolRunnerTest.java
* /incubator/any23/trunk/core/src/test/java/org/apache/any23/cli/ToolTestBase.java
* /incubator/any23/trunk/core/src/test/java/org/apache/any23/cli/VersionTest.java
* /incubator/any23/trunk/plugins/basic-crawler/pom.xml
* /incubator/any23/trunk/plugins/basic-crawler/src/main/java/org/apache/any23/cli/Crawler.java
* /incubator/any23/trunk/plugins/basic-crawler/src/test/java/org/apache/any23/cli/CrawlerTest.java
* /incubator/any23/trunk/pom.xml
> improve the current CLI engine
> ------------------------------
>
> Key: ANY23-71
> URL: https://issues.apache.org/jira/browse/ANY23-71
> Project: Apache Any23
> Issue Type: Improvement
> Affects Versions: 0.7.0
> Reporter: Simone Tripodi
> Assignee: Michele Mostarda
> Fix For: 0.7.0
>
> Attachments: enhanced_cli.patch
>
>
> Actual CLI - even if nicely working - can be improved in therms of of both internal architecture and user interface.
> I see two main "issues" on current CLI:
> * on UI, the CLI exposes internal details, since the {{ToolRunner}} requires the classname of the tool has to be executed;
> * on internals, each Tool has to parse the the chunk of the command line, which can be automated.
> So my proposal is to automate, via the already working plugins discovery, the CLI arguments parsing AND implementing a svn/git commands-based alike interface.
> My preferred choice for that is [JCommande|http://www.jcommander.org/] because:
> * it allows binding CLI arguments to Java properties via Annotations - no more manual parsing;
> * it already supports a complex syntax to implement [commands|http://jcommander.org/#Complex];
> * commands aliases can be expressed via annotations - no more needs to expose internals;
> Patch with proposal is coming
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ANY23-71) improve the current CLI engine
Posted by "Simone Tripodi (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ANY23-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simone Tripodi updated ANY23-71:
--------------------------------
Component/s: core
> improve the current CLI engine
> ------------------------------
>
> Key: ANY23-71
> URL: https://issues.apache.org/jira/browse/ANY23-71
> Project: Apache Any23
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.7.0
> Reporter: Simone Tripodi
> Assignee: Michele Mostarda
> Fix For: 0.7.0
>
> Attachments: enhanced_cli.patch
>
>
> Actual CLI - even if nicely working - can be improved in therms of of both internal architecture and user interface.
> I see two main "issues" on current CLI:
> * on UI, the CLI exposes internal details, since the {{ToolRunner}} requires the classname of the tool has to be executed;
> * on internals, each Tool has to parse the the chunk of the command line, which can be automated.
> So my proposal is to automate, via the already working plugins discovery, the CLI arguments parsing AND implementing a svn/git commands-based alike interface.
> My preferred choice for that is [JCommande|http://www.jcommander.org/] because:
> * it allows binding CLI arguments to Java properties via Annotations - no more manual parsing;
> * it already supports a complex syntax to implement [commands|http://jcommander.org/#Complex];
> * commands aliases can be expressed via annotations - no more needs to expose internals;
> Patch with proposal is coming
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (ANY23-71) improve the current CLI engine
Posted by "Michele Mostarda (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ANY23-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michele Mostarda reassigned ANY23-71:
-------------------------------------
Assignee: Michele Mostarda
> improve the current CLI engine
> ------------------------------
>
> Key: ANY23-71
> URL: https://issues.apache.org/jira/browse/ANY23-71
> Project: Apache Any23
> Issue Type: Improvement
> Affects Versions: 0.7.0
> Reporter: Simone Tripodi
> Assignee: Michele Mostarda
> Fix For: 0.7.0
>
> Attachments: enhanced_cli.patch
>
>
> Actual CLI - even if nicely working - can be improved in therms of of both internal architecture and user interface.
> I see two main "issues" on current CLI:
> * on UI, the CLI exposes internal details, since the {{ToolRunner}} requires the classname of the tool has to be executed;
> * on internals, each Tool has to parse the the chunk of the command line, which can be automated.
> So my proposal is to automate, via the already working plugins discovery, the CLI arguments parsing AND implementing a svn/git commands-based alike interface.
> My preferred choice for that is [JCommande|http://www.jcommander.org/] because:
> * it allows binding CLI arguments to Java properties via Annotations - no more manual parsing;
> * it already supports a complex syntax to implement [commands|http://jcommander.org/#Complex];
> * commands aliases can be expressed via annotations - no more needs to expose internals;
> Patch with proposal is coming
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (ANY23-71) improve the current CLI engine
Posted by "Michele Mostarda (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ANY23-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michele Mostarda resolved ANY23-71.
-----------------------------------
Resolution: Fixed
Fixed @ r1308786. Thanks to Simone Tripodi.
> improve the current CLI engine
> ------------------------------
>
> Key: ANY23-71
> URL: https://issues.apache.org/jira/browse/ANY23-71
> Project: Apache Any23
> Issue Type: Improvement
> Affects Versions: 0.7.0
> Reporter: Simone Tripodi
> Assignee: Michele Mostarda
> Fix For: 0.7.0
>
> Attachments: enhanced_cli.patch
>
>
> Actual CLI - even if nicely working - can be improved in therms of of both internal architecture and user interface.
> I see two main "issues" on current CLI:
> * on UI, the CLI exposes internal details, since the {{ToolRunner}} requires the classname of the tool has to be executed;
> * on internals, each Tool has to parse the the chunk of the command line, which can be automated.
> So my proposal is to automate, via the already working plugins discovery, the CLI arguments parsing AND implementing a svn/git commands-based alike interface.
> My preferred choice for that is [JCommande|http://www.jcommander.org/] because:
> * it allows binding CLI arguments to Java properties via Annotations - no more manual parsing;
> * it already supports a complex syntax to implement [commands|http://jcommander.org/#Complex];
> * commands aliases can be expressed via annotations - no more needs to expose internals;
> Patch with proposal is coming
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (ANY23-71) improve the current CLI engine
Posted by "Lewis John McGibbney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ANY23-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney closed ANY23-71.
-------------------------------------
Bulk close for 0.7.0-incubating release
> improve the current CLI engine
> ------------------------------
>
> Key: ANY23-71
> URL: https://issues.apache.org/jira/browse/ANY23-71
> Project: Apache Any23
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.7.0
> Reporter: Simone Tripodi
> Assignee: Michele Mostarda
> Fix For: 0.7.0
>
> Attachments: enhanced_cli.patch
>
>
> Actual CLI - even if nicely working - can be improved in therms of of both internal architecture and user interface.
> I see two main "issues" on current CLI:
> * on UI, the CLI exposes internal details, since the {{ToolRunner}} requires the classname of the tool has to be executed;
> * on internals, each Tool has to parse the the chunk of the command line, which can be automated.
> So my proposal is to automate, via the already working plugins discovery, the CLI arguments parsing AND implementing a svn/git commands-based alike interface.
> My preferred choice for that is [JCommande|http://www.jcommander.org/] because:
> * it allows binding CLI arguments to Java properties via Annotations - no more manual parsing;
> * it already supports a complex syntax to implement [commands|http://jcommander.org/#Complex];
> * commands aliases can be expressed via annotations - no more needs to expose internals;
> Patch with proposal is coming
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (ANY23-71) improve the current CLI engine
Posted by "Simone Tripodi (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/ANY23-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Simone Tripodi updated ANY23-71:
--------------------------------
Attachment: enhanced_cli.patch
The attached patch contains a proposal for implementation wich replaces commons-cli (and custom args parsing) with JCommander; I also reduced generated sh binaries to a single /any23 script, wich is the single entry point able to launch all any23 commands that are plugged via plugins discovery.
The {{Version}} command has been replaced with the {{-v|--version}} option.
A sample usage print:
{code}
$ sh any23 -h
Usage: any23 [options] [command] [command options]
Options:
-h, --help Display help information.
Default: false
-p, --plugins-dir The Any23 plugins directory.
Default: /Users/simonetripodi/.any23/plugins
-X, --verbose Produce execution verbose output.
Default: false
-v, --version Display version information.
Default: false
Commands:
extractor Utility for obtaining documentation about metadata extractors.
Usage: extractor [options] Extractor name
Options:
-a, --all shows a report about all available extractors
Default: false
-i, --input shows example input for the given extractor
Default: false
-l, --list shows the names of all available extractors
Default: false
-o, --outut shows example output for the given extractor
Default: false
microdata Commandline Tool for extracting Microdata from file/HTTP source.
Usage: microdata [options] Input document URL, {http://path/to/resource.html|file:/path/to/local.file}
mimes MIME Type Detector Tool.
Usage: mimes [options] Input document URL, {http://path/to/resource.html|file:///path/to/local.file|inline:// some inline content}
verify Utility for plugin management verification.
Usage: verify [options] plugins-dir
rover Any23 Command Line Tool.
Usage: rover [options] input URIs {<url>|<file>}+
Options:
-d, --defaultns Override the default namespace used to produce
statements.
-e, --extractors a comma-separated list of extractors, e.g.
rdf-xml,rdf-turtle
Default: []
-f, --format the output format
Default: turtle
-l, --log Produce log within a file.
-n, --nesting Disable production of nesting triples.
Default: false
-t, --notrivial Filter trivial statements (e.g. CSS related ones).
Default: false
-o, --output Specify Output file (defaults to standard output)
-p, --pedantic Validate and fixes HTML content detecting commons
issues.
Default: false
-s, --stats Print out extraction statistics.
Default: false
vocab Prints out the RDF Schema of the vocabularies used by Any23.
Usage: vocab [options]
Options:
-f, --format Vocabulary output format
Default: NQuads
{code}
see the version:
{code}
$ ah any23 -v
Apache Any23 0.7.0-incubating-SNAPSHOT (trunk@r1304362; 2012-04-01 19:03:41+0200)
Java version: 1.6.0_29, vendor: Apple Inc.
Java home: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
Default locale: en_US, platform encoding: MacRoman
OS name: "Mac OS X", version: "10.7.3", arch: "x86_64", family: "mac"
{code}
execute a command:
{code}
$ sh any23 mimes http://www.google.com/
------------------------------------------------------------------------
Apache Any23 :: mimes
------------------------------------------------------------------------
text/html
------------------------------------------------------------------------
Apache Any23 SUCCESS
Total time: 4s
Finished at: Sun Apr 01 20:39:29 CEST 2012
Final Memory: 16M/493M
------------------------------------------------------------------------
{code}
Unfortunately while rearranging things I got an issue in the tests:
{code}
-------------------------------------------------------------------------------
Test set: org.apache.any23.cli.RoverTest
-------------------------------------------------------------------------------
Tests run: 3, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 8.055 sec <<< FAILURE!
testRunMultiFiles(org.apache.any23.cli.RoverTest) Time elapsed: 0.155 sec <<< ERROR!
org.openrdf.rio.RDFParseException: Unexpected end of line. [line 27, column 304]
at org.openrdf.rio.helpers.RDFParserBase.reportFatalError(RDFParserBase.java:525)
at org.apache.any23.io.nquads.NQuadsParser.parseLine(NQuadsParser.java:249)
at org.apache.any23.io.nquads.NQuadsParser.parse(NQuadsParser.java:100)
at org.apache.any23.rdf.RDFUtils.parseRDF(RDFUtils.java:396)
at org.apache.any23.rdf.RDFUtils.parseRDF(RDFUtils.java:413)
at org.apache.any23.rdf.RDFUtils.parseRDF(RDFUtils.java:429)
at org.apache.any23.cli.RoverTest.runWithMultiSourcesAndVerify(RoverTest.java:123)
at org.apache.any23.cli.RoverTest.testRunMultiFiles(RoverTest.java:54)
{code}
Can one of the original Any23 developer take a look at it, if interested on the proposal, please?
Many thanks in advance, all the best!
> improve the current CLI engine
> ------------------------------
>
> Key: ANY23-71
> URL: https://issues.apache.org/jira/browse/ANY23-71
> Project: Apache Any23
> Issue Type: Improvement
> Affects Versions: 0.7.0
> Reporter: Simone Tripodi
> Fix For: 0.7.0
>
> Attachments: enhanced_cli.patch
>
>
> Actual CLI - even if nicely working - can be improved in therms of of both internal architecture and user interface.
> I see two main "issues" on current CLI:
> * on UI, the CLI exposes internal details, since the {{ToolRunner}} requires the classname of the tool has to be executed;
> * on internals, each Tool has to parse the the chunk of the command line, which can be automated.
> So my proposal is to automate, via the already working plugins discovery, the CLI arguments parsing AND implementing a svn/git commands-based alike interface.
> My preferred choice for that is [JCommande|http://www.jcommander.org/] because:
> * it allows binding CLI arguments to Java properties via Annotations - no more manual parsing;
> * it already supports a complex syntax to implement [commands|http://jcommander.org/#Complex];
> * commands aliases can be expressed via annotations - no more needs to expose internals;
> Patch with proposal is coming
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira