You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nlpcraft.apache.org by GitBox <gi...@apache.org> on 2020/05/08 06:19:10 UTC
[GitHub] [incubator-nlpcraft] Ifropc opened a new pull request #1: NLPCRAFT-11: Add auto-enrich using Bert and FastTest models
Ifropc opened a new pull request #1:
URL: https://github.com/apache/incubator-nlpcraft/pull/1
This pull request should resolve [NLPCRAFT-11: auto-enrich user models with synonyms](https://issues.apache.org/jira/projects/NLPCRAFT/issues/NLPCRAFT-11?filter=allopenissues)
Proposed approach uses Bert (RoBerta) model to generate synonyms for given context, masking target word. Afterwards, output is filtered with FastTest for specified context.
This feature could also be integrated with [NLPCRAFT-41](https://issues.apache.org/jira/projects/NLPCRAFT/issues/NLPCRAFT-41?filter=allopenissues)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [GitHub] [incubator-nlpcraft] Ifropc opened a new pull request
#1: NLPCRAFT-11: Add auto-enrich using Bert and FastTest models
Posted by Nikita Ivanov <mk...@gmail.com>.
NLPCraft-ers,
I know we don't have a formal written policy on becoming a committer yet.
In the time we can follow some general guidelines in [1]. Given the fact
Gleb has been working on this functionality for quite some time now (way
before our 1st release & website availability) I think we should vote to
make Gleb a committer (the first inaugural outside committer no less!). If
there are no objections I'll kick off the vote in a day or two.
Thanks,
--
Nikita Ivanov.
1. https://www.apache.org/foundation/getinvolved.html#become-a-committer
On Sun, May 10, 2020 at 4:57 AM Sergey Kamov <sk...@gmail.com> wrote:
> Thank you Gleb!
>
> I'll try to use tool in the coming days. If I have more have questions -
> I'll ask you (python is not my strong side)
>
> Probably we'll have to extend readme file.
>
> Sergey Kamov
>
>
> 09.05.2020 21:16, Ifropc пишет:
> > Hello everybody,
> >
> > I would like to explain more in details how proposed solution works and
> how it could be integrated into existing system.
> >
> > Algorithm: target word (to find synonyms for) is masked and passed to
> Bert model. Afterwards, Bert results are filtered with FastText. Minimal
> scores from Bert and FastText are configurable, and weights for results of
> this 2 models are configurable as well. Therefore, weights could be trained
> later (some real data is required though).
> > Moreover, this pipeline could be improved with adding additional models
> or filters, e.g. for some specific domain we can replace models or fit them
> with domain-specific data.
> >
> > Application: right now there are 2 ways to use this pipeline, "static"
> and "dynamic" approaches.
> > With "static" approach for Nlpcraft model and example sentences
> potential synonyms are generated to manually expand model.
> > "Dynamic" approach is to pass sentence to model, which return potential
> synonyms for the word. You can look at it as one more enricher, which
> spawns new tokens. Then, if user want to use it in their model, they can
> write a macro rule, i.e. "I want to have exactly word(s) A or any word,
> that model thinks is synonym to word A"
> >
> > Thanks,
> > Gleb.
> >
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Friday, May 8, 2020 8:56 AM, Sergey Makov <sm...@apache.org> wrote:
> >
> >> Hi Gleb,
> >>
> >> thank you very much for your contribution,
> >> it looks really promising!
> >>
> >> Regards,
> >> Sergey
> >>
> >> On Fri, May 8, 2020 at 6:52 PM Ifropc ifropc@protonmail.com.invalid
> wrote:
> >>
> >>> Hello everybody!
> >>> I created a pull request for implementing auto-enriching user models
> with synonyms (NLPCRAFT-11), please see details in PR on Github.
> >>> Comments are appreciated, if any.
> >>> Thanks,
> >>> Gleb.
> >>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> >>> On Thursday, May 7, 2020 11:19 PM, GitBox git@apache.org wrote:
> >>>> Ifropc opened a new pull request #1:
> >>>> URL: https://github.com/apache/incubator-nlpcraft/pull/1
> >>>> This pull request should resolve NLPCRAFT-11: auto-enrich user models
> with synonyms
> >>>> Proposed approach uses Bert (RoBerta) model to generate synonyms for
> given context, masking target word. Afterwards, output is filtered with
> FastTest for specified context.
> >>>> This feature could also be integrated with NLPCRAFT-41
> >>>>
> >>>> This is an automated message from the Apache Git Service.
> >>>> To respond to the message, please log on to GitHub and use the
> >>>> URL above to go to the specific comment.
> >>>> For queries about this service, please contact Infrastructure at:
> >>>> users@infra.apache.org
> >
Re: [GitHub] [incubator-nlpcraft] Ifropc opened a new pull request
#1: NLPCRAFT-11: Add auto-enrich using Bert and FastTest models
Posted by Sergey Kamov <sk...@gmail.com>.
Thank you Gleb!
I'll try to use tool in the coming days. If I have more have questions -
I'll ask you (python is not my strong side)
Probably we'll have to extend readme file.
Sergey Kamov
09.05.2020 21:16, Ifropc пишет:
> Hello everybody,
>
> I would like to explain more in details how proposed solution works and how it could be integrated into existing system.
>
> Algorithm: target word (to find synonyms for) is masked and passed to Bert model. Afterwards, Bert results are filtered with FastText. Minimal scores from Bert and FastText are configurable, and weights for results of this 2 models are configurable as well. Therefore, weights could be trained later (some real data is required though).
> Moreover, this pipeline could be improved with adding additional models or filters, e.g. for some specific domain we can replace models or fit them with domain-specific data.
>
> Application: right now there are 2 ways to use this pipeline, "static" and "dynamic" approaches.
> With "static" approach for Nlpcraft model and example sentences potential synonyms are generated to manually expand model.
> "Dynamic" approach is to pass sentence to model, which return potential synonyms for the word. You can look at it as one more enricher, which spawns new tokens. Then, if user want to use it in their model, they can write a macro rule, i.e. "I want to have exactly word(s) A or any word, that model thinks is synonym to word A"
>
> Thanks,
> Gleb.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Friday, May 8, 2020 8:56 AM, Sergey Makov <sm...@apache.org> wrote:
>
>> Hi Gleb,
>>
>> thank you very much for your contribution,
>> it looks really promising!
>>
>> Regards,
>> Sergey
>>
>> On Fri, May 8, 2020 at 6:52 PM Ifropc ifropc@protonmail.com.invalid wrote:
>>
>>> Hello everybody!
>>> I created a pull request for implementing auto-enriching user models with synonyms (NLPCRAFT-11), please see details in PR on Github.
>>> Comments are appreciated, if any.
>>> Thanks,
>>> Gleb.
>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>> On Thursday, May 7, 2020 11:19 PM, GitBox git@apache.org wrote:
>>>> Ifropc opened a new pull request #1:
>>>> URL: https://github.com/apache/incubator-nlpcraft/pull/1
>>>> This pull request should resolve NLPCRAFT-11: auto-enrich user models with synonyms
>>>> Proposed approach uses Bert (RoBerta) model to generate synonyms for given context, masking target word. Afterwards, output is filtered with FastTest for specified context.
>>>> This feature could also be integrated with NLPCRAFT-41
>>>>
>>>> This is an automated message from the Apache Git Service.
>>>> To respond to the message, please log on to GitHub and use the
>>>> URL above to go to the specific comment.
>>>> For queries about this service, please contact Infrastructure at:
>>>> users@infra.apache.org
>
Re: [GitHub] [incubator-nlpcraft] Ifropc opened a new pull request #1: NLPCRAFT-11: Add auto-enrich using Bert and FastTest models
Posted by Ifropc <if...@protonmail.com.INVALID>.
Hello everybody,
I would like to explain more in details how proposed solution works and how it could be integrated into existing system.
Algorithm: target word (to find synonyms for) is masked and passed to Bert model. Afterwards, Bert results are filtered with FastText. Minimal scores from Bert and FastText are configurable, and weights for results of this 2 models are configurable as well. Therefore, weights could be trained later (some real data is required though).
Moreover, this pipeline could be improved with adding additional models or filters, e.g. for some specific domain we can replace models or fit them with domain-specific data.
Application: right now there are 2 ways to use this pipeline, "static" and "dynamic" approaches.
With "static" approach for Nlpcraft model and example sentences potential synonyms are generated to manually expand model.
"Dynamic" approach is to pass sentence to model, which return potential synonyms for the word. You can look at it as one more enricher, which spawns new tokens. Then, if user want to use it in their model, they can write a macro rule, i.e. "I want to have exactly word(s) A or any word, that model thinks is synonym to word A"
Thanks,
Gleb.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Friday, May 8, 2020 8:56 AM, Sergey Makov <sm...@apache.org> wrote:
> Hi Gleb,
>
> thank you very much for your contribution,
> it looks really promising!
>
> Regards,
> Sergey
>
> On Fri, May 8, 2020 at 6:52 PM Ifropc ifropc@protonmail.com.invalid wrote:
>
> > Hello everybody!
> > I created a pull request for implementing auto-enriching user models with synonyms (NLPCRAFT-11), please see details in PR on Github.
> > Comments are appreciated, if any.
> > Thanks,
> > Gleb.
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Thursday, May 7, 2020 11:19 PM, GitBox git@apache.org wrote:
> > > Ifropc opened a new pull request #1:
> > > URL: https://github.com/apache/incubator-nlpcraft/pull/1
> > > This pull request should resolve NLPCRAFT-11: auto-enrich user models with synonyms
> > > Proposed approach uses Bert (RoBerta) model to generate synonyms for given context, masking target word. Afterwards, output is filtered with FastTest for specified context.
> > > This feature could also be integrated with NLPCRAFT-41
> > >
> > > This is an automated message from the Apache Git Service.
> > > To respond to the message, please log on to GitHub and use the
> > > URL above to go to the specific comment.
> > > For queries about this service, please contact Infrastructure at:
> > > users@infra.apache.org
Re: [GitHub] [incubator-nlpcraft] Ifropc opened a new pull request
#1: NLPCRAFT-11: Add auto-enrich using Bert and FastTest models
Posted by Sergey Makov <sm...@apache.org>.
Hi Gleb,
thank you very much for your contribution,
it looks really promising!
Regards,
Sergey
On Fri, May 8, 2020 at 6:52 PM Ifropc <if...@protonmail.com.invalid> wrote:
>
> Hello everybody!
>
> I created a pull request for implementing auto-enriching user models with synonyms (NLPCRAFT-11), please see details in PR on Github.
>
> Comments are appreciated, if any.
>
> Thanks,
> Gleb.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Thursday, May 7, 2020 11:19 PM, GitBox git@apache.org wrote:
>
> > Ifropc opened a new pull request #1:
> > URL: https://github.com/apache/incubator-nlpcraft/pull/1
> > This pull request should resolve NLPCRAFT-11: auto-enrich user models with synonyms
> > Proposed approach uses Bert (RoBerta) model to generate synonyms for given context, masking target word. Afterwards, output is filtered with FastTest for specified context.
> > This feature could also be integrated with NLPCRAFT-41
> >
> > This is an automated message from the Apache Git Service.
> > To respond to the message, please log on to GitHub and use the
> > URL above to go to the specific comment.
> > For queries about this service, please contact Infrastructure at:
> > users@infra.apache.org
>
>
Re: [GitHub] [incubator-nlpcraft] Ifropc opened a new pull request #1: NLPCRAFT-11: Add auto-enrich using Bert and FastTest models
Posted by Ifropc <if...@protonmail.com.INVALID>.
Hello everybody!
I created a pull request for implementing auto-enriching user models with synonyms (NLPCRAFT-11), please see details in PR on Github.
Comments are appreciated, if any.
Thanks,
Gleb.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, May 7, 2020 11:19 PM, GitBox git@apache.org wrote:
> Ifropc opened a new pull request #1:
> URL: https://github.com/apache/incubator-nlpcraft/pull/1
> This pull request should resolve NLPCRAFT-11: auto-enrich user models with synonyms
> Proposed approach uses Bert (RoBerta) model to generate synonyms for given context, masking target word. Afterwards, output is filtered with FastTest for specified context.
> This feature could also be integrated with NLPCRAFT-41
>
> This is an automated message from the Apache Git Service.
> To respond to the message, please log on to GitHub and use the
> URL above to go to the specific comment.
> For queries about this service, please contact Infrastructure at:
> users@infra.apache.org
[GitHub] [incubator-nlpcraft] skhdl commented on pull request #1: NLPCRAFT-11: Add auto-enrich using Bert and FastTest models
Posted by GitBox <gi...@apache.org>.
skhdl commented on pull request #1:
URL: https://github.com/apache/incubator-nlpcraft/pull/1#issuecomment-629625313
9. Could you add some base request validation and throw readable errors for such invalid requests?
Just for main cases (empty request, invalid indexes range) - now result is code 500, internal error.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nlpcraft] skhdl commented on pull request #1: NLPCRAFT-11: Add auto-enrich using Bert and FastTest models
Posted by GitBox <gi...@apache.org>.
skhdl commented on pull request #1:
URL: https://github.com/apache/incubator-nlpcraft/pull/1#issuecomment-629613105
Hi @Ifropc ! Thank your for your work, I have tried it and results are very impressive!
Below some remarks for discussion:
1. start_server.sh - I think that better to use python3 instead of python.
2. Is it possible to download data to user home folder (and use it from there)? For example under ~/<user-home>/nlpcraft/<bert?>
3. How can we clear installation files? Maybe we can add some flag like -reset to install_dependencies.sh or create new script like clear_dependencies.sh?
4. Should I have python3 already installed?
- If yes - could you describe it in readme? (versions etc)
- If no - we have to create ticket to check it on clear computer without any python related installations.
5. I think we can drop `#` usage at least from API (indexes parameters are enough and more flexible). Support of both modes confused.
6. I think we have to separate endpoints URLs (now we have only default one), because later we can need some additional endpoints and have to separate them.
7. Is it possible to add some new endpoint (or change current with some parameter) which can return suggestions for each word of sentence?
- Now we can do it sending N requests for each N sentence words, but it can be too slow because each request takes about 0.2 secs
- Will we have some performance improvements with such new endpoint ?
8. Can I control suggestions result count? (maybe via some parameter)
Let's discuss!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nlpcraft] Ifropc commented on pull request #1: NLPCRAFT-11: Add auto-enrich using Bert and FastTest models
Posted by GitBox <gi...@apache.org>.
Ifropc commented on pull request #1:
URL: https://github.com/apache/incubator-nlpcraft/pull/1#issuecomment-636272081
I'm closing this pull request. Further work would be done in branch NLPCRAFT-67
Let's continue discussion in [related issue](NLPCRAFT-67) on Jira
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nlpcraft] Ifropc closed pull request #1: NLPCRAFT-11: Add auto-enrich using Bert and FastTest models
Posted by GitBox <gi...@apache.org>.
Ifropc closed pull request #1:
URL: https://github.com/apache/incubator-nlpcraft/pull/1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nlpcraft] Ifropc commented on pull request #1: NLPCRAFT-11: Add auto-enrich using Bert and FastTest models
Posted by GitBox <gi...@apache.org>.
Ifropc commented on pull request #1:
URL: https://github.com/apache/incubator-nlpcraft/pull/1#issuecomment-629735553
Hi @skhdl , thanks you for your feedback.
1. Done
2. I think in general it's not very good approach, as it is not following XDG base directory specification. I understand that currently NLP craft do store some files there, so I think this should remain open for discussion and for now I leave it as is. In future, we may consider either moving NLP craft files to XDG supported directory, or make it configurable in some other way (environment variable, configuration in file, etc).
3. It's already done after successful installation. Are you talking about `/tmp/fastText/`
4. Yes, it's checked in installation script.
5. Done
6. Done. Please use **/synonyms** endpoint.
7. This part is a bit tricky. I am going to improve performance, but it might take some time to make proper batching into models, so it might be out of scope of current pull request and left as improvement for next iteration.
8. Yes, it was already implemented, but I forgot to add it as part of API. Please pass this value as **"limit"**
9. Done
Also I upgraded torch to latest version (1.5), please re-run install script.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-nlpcraft] syermakov commented on pull request #1: NLPCRAFT-11: Add auto-enrich using Bert and FastTest models
Posted by GitBox <gi...@apache.org>.
syermakov commented on pull request #1:
URL: https://github.com/apache/incubator-nlpcraft/pull/1#issuecomment-625923342
@Ifropc Thank you very much for sharing, this feature looks very promising!
I would suggest the following improvements:
- Add Apache license header to all files
- Use `logging` module instead of the `print` function
- Do we need matplotlib-related functions (`mk_graph`, etc) in `bertft.py`, isn't it better to move them directly to jupiter notebook?
- May be rename `enricher` folder to `syn_enricher` (just a proposal, ideas?)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org