You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stormcrawler.apache.org by Tim Allison <ta...@apache.org> on 2025/02/05 14:31:27 UTC

playwright and headers?

This should be a question for our user@ mailing list, but I don't think
we've set that up yet?

I'm not sure if this is user error.

The playwright protocol does not appear to do anything with
"storeHttpHeaders" as the httpclient and okhttp protocols do (e.g.
https://github.com/apache/incubator-stormcrawler/blob/main/core/src/main/java/org/apache/stormcrawler/protocol/okhttp/HttpProtocol.java#L220
).

Is this intentional or an area for improvement?

Thank you.

Best,

        Tim

Re: playwright and headers?

Posted by Tim Allison <ta...@apache.org>.
Thank you!

On Thu, Feb 6, 2025 at 2:39 AM Richard Zowalla <rz...@apache.org> wrote:
>
> Hi Tim,
>
> I think, that it is just not implemented. Feel free to add code to support that configuration.
>
> Gruß
> Richard
>
> > Am 05.02.2025 um 15:31 schrieb Tim Allison <ta...@apache.org>:
> >
> > This should be a question for our user@ mailing list, but I don't think
> > we've set that up yet?
> >
> > I'm not sure if this is user error.
> >
> > The playwright protocol does not appear to do anything with
> > "storeHttpHeaders" as the httpclient and okhttp protocols do (e.g.
> > https://github.com/apache/incubator-stormcrawler/blob/main/core/src/main/java/org/apache/stormcrawler/protocol/okhttp/HttpProtocol.java#L220
> > ).
> >
> > Is this intentional or an area for improvement?
> >
> > Thank you.
> >
> > Best,
> >
> >        Tim
>

Re: playwright and headers?

Posted by Richard Zowalla <rz...@apache.org>.
Hi Tim,

I think, that it is just not implemented. Feel free to add code to support that configuration.

Gruß
Richard

> Am 05.02.2025 um 15:31 schrieb Tim Allison <ta...@apache.org>:
> 
> This should be a question for our user@ mailing list, but I don't think
> we've set that up yet?
> 
> I'm not sure if this is user error.
> 
> The playwright protocol does not appear to do anything with
> "storeHttpHeaders" as the httpclient and okhttp protocols do (e.g.
> https://github.com/apache/incubator-stormcrawler/blob/main/core/src/main/java/org/apache/stormcrawler/protocol/okhttp/HttpProtocol.java#L220
> ).
> 
> Is this intentional or an area for improvement?
> 
> Thank you.
> 
> Best,
> 
>        Tim