You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@stormcrawler.apache.org by "rzo1 (via GitHub)" <gi...@apache.org> on 2024/11/22 07:36:04 UTC
[PR] Fixes Issues mentioned in IPMC Vote [incubator-stormcrawler]
rzo1 opened a new pull request, #1417:
URL: https://github.com/apache/incubator-stormcrawler/pull/1417
See details: https://lists.apache.org/thread/xljxn3901mffhyjhdx7orwy938kf2vjv
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fixes Issues mentioned in IPMC Vote [incubator-stormcrawler]
Posted by "rzo1 (via GitHub)" <gi...@apache.org>.
rzo1 commented on code in PR #1417:
URL: https://github.com/apache/incubator-stormcrawler/pull/1417#discussion_r1854356820
##########
external/tika/pom.xml:
##########
@@ -60,6 +60,11 @@ under the License.
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
+ <!-- This comes in through the Tika Image Parser module and has an incompatible BSD license -->
Review Comment:
Great to have the link!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fixes Issues mentioned in IPMC Vote [incubator-stormcrawler]
Posted by "rzo1 (via GitHub)" <gi...@apache.org>.
rzo1 commented on PR #1417:
URL: https://github.com/apache/incubator-stormcrawler/pull/1417#issuecomment-2493108174
Replaced the RSS Feeds as well as the compressed Tripadvisor-based Sitemap tests with original stormcrawler.apache.org content.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fixes Issues mentioned in IPMC Vote [incubator-stormcrawler]
Posted by "rzo1 (via GitHub)" <gi...@apache.org>.
rzo1 commented on code in PR #1417:
URL: https://github.com/apache/incubator-stormcrawler/pull/1417#discussion_r1853435960
##########
core/src/test/java/org/apache/stormcrawler/bolt/FeedParserBoltTest.java:
##########
@@ -51,7 +51,10 @@ void testFeedParsing() throws IOException {
Metadata metadata = new Metadata();
// specify that it is a Feed file
metadata.setValue(FeedParserBolt.isFeedKey, "true");
- parse("http://www.guardian.com/Feed.xml", "guardian.rss", metadata);
+ parse(
+ "https://de.wikipedia.org/w/index.php?title=Apache_Software_Foundation&action=history&feed=rss",
Review Comment:
Thanks. Think we are good now. I replaced it with handmade rss based upon the sitemap of our actual stormcrawler website.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fixes Issues mentioned in IPMC Vote [incubator-stormcrawler]
Posted by "tballison (via GitHub)" <gi...@apache.org>.
tballison commented on code in PR #1417:
URL: https://github.com/apache/incubator-stormcrawler/pull/1417#discussion_r1854343012
##########
external/tika/pom.xml:
##########
@@ -60,6 +60,11 @@ under the License.
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
+ <!-- This comes in through the Tika Image Parser module and has an incompatible BSD license -->
Review Comment:
Apache Tika cleared the use of `jai-imageio-core` here: https://issues.apache.org/jira/browse/LEGAL-304.
I'm ok excluding it to make licensing cleaner for stormcrawler, and I don't think there will be large consequences for stormcrawler.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fixes Issues mentioned in IPMC Vote [incubator-stormcrawler]
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #1417:
URL: https://github.com/apache/incubator-stormcrawler/pull/1417#discussion_r1853416242
##########
core/src/test/java/org/apache/stormcrawler/bolt/FeedParserBoltTest.java:
##########
@@ -51,7 +51,10 @@ void testFeedParsing() throws IOException {
Metadata metadata = new Metadata();
// specify that it is a Feed file
metadata.setValue(FeedParserBolt.isFeedKey, "true");
- parse("http://www.guardian.com/Feed.xml", "guardian.rss", metadata);
+ parse(
+ "https://de.wikipedia.org/w/index.php?title=Apache_Software_Foundation&action=history&feed=rss",
Review Comment:
can we use some page listed under apache scope instead? not sure if using wikipedia might upset some as well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fixes Issues mentioned in IPMC Vote [incubator-stormcrawler]
Posted by "rzo1 (via GitHub)" <gi...@apache.org>.
rzo1 commented on code in PR #1417:
URL: https://github.com/apache/incubator-stormcrawler/pull/1417#discussion_r1853419849
##########
core/src/test/java/org/apache/stormcrawler/bolt/FeedParserBoltTest.java:
##########
@@ -51,7 +51,10 @@ void testFeedParsing() throws IOException {
Metadata metadata = new Metadata();
// specify that it is a Feed file
metadata.setValue(FeedParserBolt.isFeedKey, "true");
- parse("http://www.guardian.com/Feed.xml", "guardian.rss", metadata);
+ parse(
+ "https://de.wikipedia.org/w/index.php?title=Apache_Software_Foundation&action=history&feed=rss",
Review Comment:
Alternative is I will just create something random myself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fixes Issues mentioned in IPMC Vote [incubator-stormcrawler]
Posted by "rzo1 (via GitHub)" <gi...@apache.org>.
rzo1 commented on code in PR #1417:
URL: https://github.com/apache/incubator-stormcrawler/pull/1417#discussion_r1853421533
##########
core/src/test/java/org/apache/stormcrawler/bolt/FeedParserBoltTest.java:
##########
@@ -51,7 +51,10 @@ void testFeedParsing() throws IOException {
Metadata metadata = new Metadata();
// specify that it is a Feed file
metadata.setValue(FeedParserBolt.isFeedKey, "true");
- parse("http://www.guardian.com/Feed.xml", "guardian.rss", metadata);
+ parse(
+ "https://de.wikipedia.org/w/index.php?title=Apache_Software_Foundation&action=history&feed=rss",
Review Comment:
Let me create one for stormcrawler.apache.org (just to be save)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fixes Issues mentioned in IPMC Vote [incubator-stormcrawler]
Posted by "tballison (via GitHub)" <gi...@apache.org>.
tballison merged PR #1417:
URL: https://github.com/apache/incubator-stormcrawler/pull/1417
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fixes Issues mentioned in IPMC Vote [incubator-stormcrawler]
Posted by "rzo1 (via GitHub)" <gi...@apache.org>.
rzo1 commented on code in PR #1417:
URL: https://github.com/apache/incubator-stormcrawler/pull/1417#discussion_r1853417153
##########
core/src/test/java/org/apache/stormcrawler/bolt/FeedParserBoltTest.java:
##########
@@ -51,7 +51,10 @@ void testFeedParsing() throws IOException {
Metadata metadata = new Metadata();
// specify that it is a Feed file
metadata.setValue(FeedParserBolt.isFeedKey, "true");
- parse("http://www.guardian.com/Feed.xml", "guardian.rss", metadata);
+ parse(
+ "https://de.wikipedia.org/w/index.php?title=Apache_Software_Foundation&action=history&feed=rss",
Review Comment:
Do you have an example for an RSS feed on an apache site ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fixes Issues mentioned in IPMC Vote [incubator-stormcrawler]
Posted by "ayushtkn (via GitHub)" <gi...@apache.org>.
ayushtkn commented on code in PR #1417:
URL: https://github.com/apache/incubator-stormcrawler/pull/1417#discussion_r1853429177
##########
core/src/test/java/org/apache/stormcrawler/bolt/FeedParserBoltTest.java:
##########
@@ -51,7 +51,10 @@ void testFeedParsing() throws IOException {
Metadata metadata = new Metadata();
// specify that it is a Feed file
metadata.setValue(FeedParserBolt.isFeedKey, "true");
- parse("http://www.guardian.com/Feed.xml", "guardian.rss", metadata);
+ parse(
+ "https://de.wikipedia.org/w/index.php?title=Apache_Software_Foundation&action=history&feed=rss",
Review Comment:
maybe comdev has some
https://community.apache.org/blog/
the latest says
https://community.apache.org/blog/rss-feeds-activated.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@stormcrawler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org