You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stormcrawler.apache.org by "jnioche (via GitHub)" <gi...@apache.org> on 2024/05/17 10:18:59 UTC
[PR] 1024 Playwright protocol implementation, fixes #1024 [incubator-stormcrawler]
jnioche opened a new pull request, #1228:
URL: https://github.com/apache/incubator-stormcrawler/pull/1228
Thank you for contributing to Apache StormCrawler.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ x] Is there a issue associated with this PR? Is it referenced in the commit message?
- [x ] Does your PR title start with `#XXXX` where `XXXX` is the issue number you are trying to resolve?
- [x ] Has your PR been rebased against the latest commit within the target branch (typically main)?
- [ x] Is your initial contribution a single, squashed commit?
- [ x] Is the code properly formatted with `mvn git-code-format:format-code -Dgcf.globPattern="**/*" -Dskip.format.code=false`?
### For code changes:
- [ x] Have you ensured that the full suite of tests is executed via `mvn clean verify`?
- [ x] Have you written or updated unit tests to verify your changes?
- [ x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [x ] If applicable, have you updated the LICENSE file, including the main LICENSE file?
- [ x] If applicable, have you updated the NOTICE file, including the main NOTICE file?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions for build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 1024 Playwright protocol implementation, fixes #1024 [incubator-stormcrawler]
Posted by "jnioche (via GitHub)" <gi...@apache.org>.
jnioche commented on code in PR #1228:
URL: https://github.com/apache/incubator-stormcrawler/pull/1228#discussion_r1604993382
##########
THIRD-PARTY.txt:
##########
@@ -136,6 +136,7 @@ List of third-party dependencies grouped by their license type.
* FindBugs-jsr305 (com.google.code.findbugs:jsr305:3.0.2 - http://findbugs.sourceforge.net/)
* Google Android Annotations Library (com.google.android:annotations:4.1.1.4 - http://source.android.com/)
* Graphite Integration for Metrics (io.dropwizard.metrics:metrics-graphite:3.2.6 - http://metrics.dropwizard.io/metrics-graphite/)
+ * Gson (com.google.code.gson:gson:2.8.9 - https://github.com/google/gson/gson)
Review Comment:
Not sure what generates it.
`mvn dependency:tree `
```
[INFO] org.apache.stormcrawler:stormcrawler-playwright:jar:3.1.0-SNAPSHOT
[INFO] +- com.microsoft.playwright:playwright:jar:1.43.0:compile
[INFO] | +- com.google.code.gson:gson:jar:2.10.1:compile
[INFO] | +- org.opentest4j:opentest4j:jar:1.3.0:compile
```
Maybe it came from using an older version of playwright but then it would be removed from THIRD PARTY when calling `mvn clean install` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 1024 Playwright protocol implementation, fixes #1024 [incubator-stormcrawler]
Posted by "rzo1 (via GitHub)" <gi...@apache.org>.
rzo1 commented on PR #1228:
URL: https://github.com/apache/incubator-stormcrawler/pull/1228#issuecomment-2117263513
A great addition to SC!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 1024 Playwright protocol implementation, fixes #1024 [incubator-stormcrawler]
Posted by "jnioche (via GitHub)" <gi...@apache.org>.
jnioche merged PR #1228:
URL: https://github.com/apache/incubator-stormcrawler/pull/1228
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] 1024 Playwright protocol implementation, fixes #1024 [incubator-stormcrawler]
Posted by "rzo1 (via GitHub)" <gi...@apache.org>.
rzo1 commented on code in PR #1228:
URL: https://github.com/apache/incubator-stormcrawler/pull/1228#discussion_r1604728687
##########
THIRD-PARTY.txt:
##########
@@ -136,6 +136,7 @@ List of third-party dependencies grouped by their license type.
* FindBugs-jsr305 (com.google.code.findbugs:jsr305:3.0.2 - http://findbugs.sourceforge.net/)
* Google Android Annotations Library (com.google.android:annotations:4.1.1.4 - http://source.android.com/)
* Graphite Integration for Metrics (io.dropwizard.metrics:metrics-graphite:3.2.6 - http://metrics.dropwizard.io/metrics-graphite/)
+ * Gson (com.google.code.gson:gson:2.8.9 - https://github.com/google/gson/gson)
Review Comment:
I think, that something is leaking `gson` here. Maybe we can fix to `2.9.0` ?
##########
THIRD-PARTY.txt:
##########
@@ -245,6 +247,9 @@ List of third-party dependencies grouped by their license type.
* parso (com.epam:parso:2.0.14 - https://github.com/epam/parso)
* PDFBox JBIG2 ImageIO plugin (org.apache.pdfbox:jbig2-imageio:3.0.4 - https://www.apache.org/jbig2-imageio/)
* perfmark:perfmark-api (io.perfmark:perfmark-api:0.25.0 - https://github.com/perfmark/perfmark)
+ * Playwright - Driver (com.microsoft.playwright:driver:1.41.2 - https://github.com/microsoft/playwright-java/driver)
Review Comment:
`1.41.2` -> `1.44.0` -> https://github.com/microsoft/playwright/releases/tag/v1.44.0
##########
external/playwright/pom.xml:
##########
@@ -0,0 +1,81 @@
+<?xml version="1.0" encoding="UTF-8"?>
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
+ <modelVersion>4.0.0</modelVersion>
+
+ <parent>
+ <groupId>org.apache.stormcrawler</groupId>
+ <artifactId>stormcrawler-external</artifactId>
+ <version>3.1.0-SNAPSHOT</version>
+ <relativePath>../pom.xml</relativePath>
+ </parent>
+
+ <artifactId>stormcrawler-playwright</artifactId>
+ <packaging>jar</packaging>
+
+ <name>stormcrawler-playwright</name>
+ <url>https://github.com/apache/incubator-stormcrawler/tree/master/external/playwright</url>
+ <description>Playwright-based protocol for StormCrawler</description>
+
+ <properties>
+ <playwright.version>1.43.0</playwright.version>
Review Comment:
1.44.0 ? At least this dependency does not match what is in the third party txt?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org