You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@joshua.apache.org by "Matt Post (JIRA)" <ji...@apache.org> on 2016/07/27 20:44:20 UTC
[jira] [Updated] (JOSHUA-284) Phrase-based decoding changes
[ https://issues.apache.org/jira/browse/JOSHUA-284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Matt Post updated JOSHUA-284:
-----------------------------
Description:
Joshua's phrase-based decoding creates a lot of complications in the pipeline.
Currently, phrase-based rules are simply left-branching Hiero rules. This means that, prior to packing or loading, rules have to have a nonterminal prepended to them. For example, Thrax will extract
[X] ||| yo quiero ||| i want ||| ...
This has to be changed to
[X] ||| [X,1] yo quiero ||| [X,1] yo quiero ||| ...
This means, for one, that phrase tables share a format but are specific to either the hiero or phrase-based decoder.
Another problem is that the alignments have to be adjusted when packing grammars from Moses or Thrax format, since a symbol is being added. Basically, this choice introduces a host of incompatibilities that require special handling.
A better idea would be to change the phrase-based decoder a bit so that, instead of using left-branching phrase rules, it made use of proper glue rules, the same way Hiero does. The advantages are:
- both formalisms would use the same format
- both formalisms would have a glue grammar
- there should be no impact in running time
was:
Joshua's phrase-based decoding creates a lot of complications in the pipeline.
Currently, phrase-based rules are simply left-branching Hiero rules. This means that, prior to packing or loading, rules have to have a nonterminal prepended to them. For example, Thrax will extract
[X] ||| yo quiero ||| i want ||| ...
This has to be changed to
[X] ||| [X,1] yo quiero ||| [X,1] yo quiero ||| ...
This means, for one, that phrase tables share a format but are specific to either the hiero or phrase-based decoder.
A better idea would be to change the phrase-based decoder a bit so that, instead of using left-branching phrase rules, it made use of proper glue rules, the same way Hiero does. The advantages are:
- both formalisms would use the same format
- both formalisms would have a glue grammar
- there should be no impact in running time
> Phrase-based decoding changes
> -----------------------------
>
> Key: JOSHUA-284
> URL: https://issues.apache.org/jira/browse/JOSHUA-284
> Project: Joshua
> Issue Type: Bug
> Reporter: Matt Post
> Fix For: 6.1
>
>
> Joshua's phrase-based decoding creates a lot of complications in the pipeline.
> Currently, phrase-based rules are simply left-branching Hiero rules. This means that, prior to packing or loading, rules have to have a nonterminal prepended to them. For example, Thrax will extract
> [X] ||| yo quiero ||| i want ||| ...
> This has to be changed to
> [X] ||| [X,1] yo quiero ||| [X,1] yo quiero ||| ...
> This means, for one, that phrase tables share a format but are specific to either the hiero or phrase-based decoder.
> Another problem is that the alignments have to be adjusted when packing grammars from Moses or Thrax format, since a symbol is being added. Basically, this choice introduces a host of incompatibilities that require special handling.
> A better idea would be to change the phrase-based decoder a bit so that, instead of using left-branching phrase rules, it made use of proper glue rules, the same way Hiero does. The advantages are:
> - both formalisms would use the same format
> - both formalisms would have a glue grammar
> - there should be no impact in running time
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)