You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@nutch.apache.org by lewis john mcgibbney <le...@apache.org> on 2021/06/03 21:44:49 UTC

DuplexWeb-Google - GoogleBot Crawler For Duplex / Google Assistant

Some interesting content for a short read :)

https://www.seroundtable.com/duplexweb-google-bot-31522.html?utm_source=search_engine_roundtable&utm_campaign=ser_newsletter_2021-06-03&utm_medium=email

-- 
http://home.apache.org/~lewismc/
http://people.apache.org/keys/committer/lewismc

Re: DuplexWeb-Google - GoogleBot Crawler For Duplex / Google Assistant

Posted by Sebastian Nagel <wa...@googlemail.com.INVALID>.

Thanks! Interesting that the dublexweb bot ignores the wildcard user agent rules by default.

On 6/3/21 11:44 PM, lewis john mcgibbney wrote:
> Some interesting content for a short read :)
> 
> https://www.seroundtable.com/duplexweb-google-bot-31522.html?utm_source=search_engine_roundtable&utm_campaign=ser_newsletter_2021-06-03&utm_medium=email
>