You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@opennlp.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/12/05 08:41:00 UTC
[jira] [Commented] (OPENNLP-1154) change the XML format for feature
generator config in NameFinder and POS Tagger
[ https://issues.apache.org/jira/browse/OPENNLP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16278203#comment-16278203 ]
ASF GitHub Bot commented on OPENNLP-1154:
-----------------------------------------
kottmann commented on a change in pull request #286: OPENNLP-1154: change the XML format for feature generator config in N…
URL: https://github.com/apache/opennlp/pull/286#discussion_r154877269
##########
File path: opennlp-tools/src/main/java/opennlp/tools/util/featuregen/GeneratorFactory.java
##########
@@ -110,527 +119,232 @@ AdaptiveFeatureGenerator create(Element generatorElement,
FeatureGeneratorResourceProvider resourceManager) throws InvalidFormatException;
}
- /**
- * @see AggregatedFeatureGenerator
- */
- static class AggregatedFeatureGeneratorFactory implements XmlFeatureGeneratorFactory {
-
- public AdaptiveFeatureGenerator create(Element generatorElement,
- FeatureGeneratorResourceProvider resourceManager) throws InvalidFormatException {
-
- Collection<AdaptiveFeatureGenerator> aggregatedGenerators = new LinkedList<>();
+ public static abstract class AbstractXmlFeatureGeneratorFactory {
- NodeList childNodes = generatorElement.getChildNodes();
+ // TODO: set the following members to final once back-compat support is over
+ protected Element generatorElement;
+ protected FeatureGeneratorResourceProvider resourceManager;
- for (int i = 0; i < childNodes.getLength(); i++) {
- Node childNode = childNodes.item(i);
- if (childNode instanceof Element) {
- Element aggregatedGeneratorElement = (Element) childNode;
- aggregatedGenerators.add(
- GeneratorFactory.createGenerator(aggregatedGeneratorElement, resourceManager));
- }
- }
+ // to respect the order <generator/> in AggregatedFeatureGenerator, let's use LinkedHashMap
+ protected LinkedHashMap<String, Object> args;
- return new AggregatedFeatureGenerator(aggregatedGenerators.toArray(
- new AdaptiveFeatureGenerator[aggregatedGenerators.size()]));
+ // TODO: just remove this constructor when back-compat is no longer needed
+ protected AbstractXmlFeatureGeneratorFactory() {
}
- static void register(Map<String, XmlFeatureGeneratorFactory> factoryMap) {
- factoryMap.put("generators", new AggregatedFeatureGeneratorFactory());
+ protected AbstractXmlFeatureGeneratorFactory(Element generatorElement,
+ FeatureGeneratorResourceProvider resourceManager) {
+ this.generatorElement = generatorElement;
+ this.resourceManager = resourceManager;
+ args = new LinkedHashMap<>();
}
- }
-
- /**
- * @see CachedFeatureGenerator
- */
- static class CachedFeatureGeneratorFactory implements XmlFeatureGeneratorFactory {
- private CachedFeatureGeneratorFactory() {
+ public Map<String, ArtifactSerializer<?>>
+ getArtifactSerializerMapping() throws InvalidFormatException {
+ return null;
}
- public AdaptiveFeatureGenerator create(Element generatorElement,
- FeatureGeneratorResourceProvider resourceManager) throws InvalidFormatException {
-
- Element cachedGeneratorElement = null;
-
- NodeList kids = generatorElement.getChildNodes();
-
- for (int i = 0; i < kids.getLength(); i++) {
- Node childNode = kids.item(i);
-
+ final void init() throws InvalidFormatException {
+ int generators = 0;
+ NodeList childNodes = generatorElement.getChildNodes();
+ for (int i = 0; i < childNodes.getLength(); i++) {
+ Node childNode = childNodes.item(i);
if (childNode instanceof Element) {
- cachedGeneratorElement = (Element) childNode;
- break;
+ Element elem = (Element)childNode;
+ String type = elem.getTagName();
+ if (type.equals("generator")) {
+ String key = "generator#" + Integer.toString(generators++);
+ args.put(key, buildGenerator(elem, resourceManager));
+ }
+ else {
+ String name = elem.getAttribute("name");
+ Node cn = elem.getFirstChild();
+ Text text = (Text)cn;
+ if (type.equals("int")) {
Review comment:
Here a switch case will look nicer.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> change the XML format for feature generator config in NameFinder and POS Tagger
> -------------------------------------------------------------------------------
>
> Key: OPENNLP-1154
> URL: https://issues.apache.org/jira/browse/OPENNLP-1154
> Project: OpenNLP
> Issue Type: Improvement
> Components: Name Finder
> Affects Versions: 1.8.3
> Reporter: Koji Sekiguchi
> Assignee: Koji Sekiguchi
>
> NameFinder provides many kinds of feature generator (factories). Users can define their config via XML which looks like:
> {code:xml}
> <generators>
> <cache>
> <generators>
> <window prevLength = "2" nextLength = "2">
> <tokenclass/>
> </window>
> <window prevLength = "2" nextLength = "2">
> <token/>
> </window>
> <definition/>
> <prevmap/>
> <bigram/>
> <sentence begin="true" end="false"/>
> </generators>
> </cache>
> </generators>
> {code}
> If a user wants to implement their own feature generator, he can use <custom .../>, but if he wants to have two or more feature generators at once, he may be able to implement it by providing a wrapper feature generator which wraps two or more feature generators that he originally wants to have, but it is not good.
> I'd like to suggest that we make the config format more flexible like below:
> {code:xml}
> <generator class="opennlp.tools.util.featuregen.AggregatedFeatureGeneratorFactory">
> <args>
> <generator class="opennlp.tools.util.featuregen.CachedFeatureGeneratorFactory">
> <args>
> <generator class="opennlp.tools.util.featuregen.AggregatedFeatureGeneratorFactory">
> <args>
> <generator class="opennlp.tools.util.featuregen.WindowFeatureGeneratorFactory">
> <args>
> <int name="prevLength">2</int>
> <int name="nextLength">2</int>
> <generator class="opennlp.tools.util.featuregen.TokenClassFeatureGeneratorFactory"/>
> </args>
> </generator>
> <generator class="opennlp.tools.util.featuregen.WindowFeatureGeneratorFactory">
> <args>
> <int name="prevLength">2</int>
> <int name="nextLength">2</int>
> <generator class="opennlp.tools.util.featuregen.TokenFeatureGeneratorFactory"/>
> </args>
> </generator>
> </args>
> </generator>
> </args>
> </generator>
> </args>
> </generator>
> {code}
> If <args>...</args> is too noisy, I'm thinking another format as well:
> {code:xml}
> <generator class="opennlp.tools.util.featuregen.AggregatedFeatureGeneratorFactory">
> <generator class="opennlp.tools.util.featuregen.CachedFeatureGeneratorFactory">
> <generator class="opennlp.tools.util.featuregen.AggregatedFeatureGeneratorFactory">
> <generator class="opennlp.tools.util.featuregen.WindowFeatureGeneratorFactory">
> <int name="prevLength">2</int>
> <int name="nextLength">2</int>
> <generator class="opennlp.tools.util.featuregen.TokenClassFeatureGeneratorFactory"/>
> </generator>
> <generator class="opennlp.tools.util.featuregen.WindowFeatureGeneratorFactory">
> <int name="prevLength">2</int>
> <int name="nextLength">2</int>
> <generator class="opennlp.tools.util.featuregen.TokenFeatureGeneratorFactory"/>
> </generator>
> </generator>
> </generator>
> </generator>
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)