You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@streams.apache.org by robdouglas <gi...@git.apache.org> on 2014/05/22 18:05:51 UTC
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
GitHub user robdouglas opened a pull request:
https://github.com/apache/incubator-streams/pull/25
STREAMS-93 | Updated the Hashtag POJO schema in tweet.json, added depend...
...ency on streams-processor-urls in streams-provider-twitter (to get LinkDetails POJO) and updated all Twitter serializers to add in generic extensions
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/robdouglas/incubator-streams STREAMS-93
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-streams/pull/25.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #25
----
commit e29eadc9c29bb0dbb1dc2115804e84823d5674b2
Author: Robert Douglas <rd...@w2odigital.com>
Date: 2014-05-22T16:04:03Z
STREAMS-93 | Updated the Hashtag POJO schema in tweet.json, added dependency on streams-processor-urls in streams-provider-twitter (to get LinkDetails POJO) and updated all Twitter serializers to add in generic extensions
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by mfranklin <gi...@git.apache.org>.
Github user mfranklin commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/25#discussion_r12974835
--- Diff: streams-contrib/streams-provider-twitter/pom.xml ---
@@ -46,6 +46,11 @@
</dependency>
<dependency>
<groupId>org.apache.streams</groupId>
+ <artifactId>streams-processor-urls</artifactId>
--- End diff --
I still see processor-urls here
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by robdouglas <gi...@git.apache.org>.
Github user robdouglas commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/25#discussion_r12961938
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/serializer/TwitterJsonActivitySerializer.java ---
@@ -81,6 +87,48 @@ else if( documentSubType == Delete.class )
throw new NotImplementedException();
}
+ public static Actor buildActorRetweet(Retweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActorTweet(Tweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActor(User user) {
+ Actor actor = new Actor();
+
+ actor.setId(formatId(
+ Optional.fromNullable(
+ user.getIdStr())
+ .or(Optional.of(user.getId().toString()))
+ .orNull()
+ ));
+ actor.setDisplayName(user.getScreenName());
+ if (user.getUrl()!=null){
+ actor.setUrl(user.getUrl());
+ }
+
+ actor.setDisplayName(user.getName());
+ actor.setSummary(user.getDescription());
+
+ Map<String, Object> extensions = new HashMap<String, Object>();
+ extensions.put("location", user.getLocation());
--- End diff --
Would you propose taking this out then? Twitter4J will just return a String that is user set for the actor's location
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by mfranklin <gi...@git.apache.org>.
Github user mfranklin commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/25#discussion_r12961379
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/serializer/TwitterJsonActivitySerializer.java ---
@@ -81,6 +87,48 @@ else if( documentSubType == Delete.class )
throw new NotImplementedException();
}
+ public static Actor buildActorRetweet(Retweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActorTweet(Tweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActor(User user) {
+ Actor actor = new Actor();
+
+ actor.setId(formatId(
+ Optional.fromNullable(
+ user.getIdStr())
+ .or(Optional.of(user.getId().toString()))
+ .orNull()
+ ));
+ actor.setDisplayName(user.getScreenName());
+ if (user.getUrl()!=null){
+ actor.setUrl(user.getUrl());
+ }
+
+ actor.setDisplayName(user.getName());
+ actor.setSummary(user.getDescription());
+
+ Map<String, Object> extensions = new HashMap<String, Object>();
+ extensions.put("location", user.getLocation());
+ extensions.put("posts", user.getStatusesCount());
+ extensions.put("favorites", user.getFavouritesCount());
--- End diff --
In your proposal to the list you also had a perspectival flag, right?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by mfranklin <gi...@git.apache.org>.
Github user mfranklin commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/25#discussion_r12961535
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/serializer/TwitterJsonTweetActivitySerializer.java ---
@@ -78,35 +80,71 @@ public Activity deserialize(String serialized) throws ActivitySerializerExceptio
activity.setProvider(getProvider());
activity.setTitle("");
activity.setContent(tweet.getText());
- activity.setUrl("http://twitter.com/" + tweet.getIdStr());
+ activity.setUrl("http://twitter.com/" + tweet.getUser().getIdStr() + "/status/" + tweet.getIdStr());
activity.setLinks(getLinks(tweet));
addTwitterExtension(activity, mapper.convertValue(tweet, ObjectNode.class));
addLocationExtension(activity, tweet);
+ addTwitterExtensions(activity, tweet);
+
return activity;
}
+ public static void addTwitterExtensions(Activity activity, Tweet tweet) {
--- End diff --
This seems like it should be a utility function rather than duplicated logic with the class above.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by mfranklin <gi...@git.apache.org>.
Github user mfranklin commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/25#discussion_r12961473
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/serializer/TwitterJsonActivitySerializer.java ---
@@ -81,6 +87,48 @@ else if( documentSubType == Delete.class )
throw new NotImplementedException();
}
+ public static Actor buildActorRetweet(Retweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActorTweet(Tweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActor(User user) {
+ Actor actor = new Actor();
+
+ actor.setId(formatId(
+ Optional.fromNullable(
+ user.getIdStr())
+ .or(Optional.of(user.getId().toString()))
+ .orNull()
+ ));
+ actor.setDisplayName(user.getScreenName());
+ if (user.getUrl()!=null){
+ actor.setUrl(user.getUrl());
+ }
+
+ actor.setDisplayName(user.getName());
+ actor.setSummary(user.getDescription());
+
+ Map<String, Object> extensions = new HashMap<String, Object>();
+ extensions.put("location", user.getLocation());
+ extensions.put("posts", user.getStatusesCount());
+ extensions.put("favorites", user.getFavouritesCount());
--- End diff --
NM, I see now this is on the actor object
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by mfranklin <gi...@git.apache.org>.
Github user mfranklin commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/25#discussion_r13083214
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/serializer/TwitterJsonActivitySerializer.java ---
@@ -81,6 +87,86 @@ else if( documentSubType == Delete.class )
throw new NotImplementedException();
}
+ public static Actor buildActorRetweet(Retweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActorTweet(Tweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActor(User user) {
+ Actor actor = new Actor();
+
+ actor.setId(formatId(
+ Optional.fromNullable(
+ user.getIdStr())
+ .or(Optional.of(user.getId().toString()))
+ .orNull()
+ ));
+ actor.setDisplayName(user.getScreenName());
--- End diff --
If it is in the extensions already, I suggest leave it there for now, but we should have a separate conversation on the list about how we handle this
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by mfranklin <gi...@git.apache.org>.
Github user mfranklin commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/25#discussion_r12961348
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/serializer/TwitterJsonActivitySerializer.java ---
@@ -81,6 +87,48 @@ else if( documentSubType == Delete.class )
throw new NotImplementedException();
}
+ public static Actor buildActorRetweet(Retweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActorTweet(Tweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActor(User user) {
+ Actor actor = new Actor();
+
+ actor.setId(formatId(
+ Optional.fromNullable(
+ user.getIdStr())
+ .or(Optional.of(user.getId().toString()))
+ .orNull()
+ ));
+ actor.setDisplayName(user.getScreenName());
+ if (user.getUrl()!=null){
+ actor.setUrl(user.getUrl());
+ }
+
+ actor.setDisplayName(user.getName());
+ actor.setSummary(user.getDescription());
+
+ Map<String, Object> extensions = new HashMap<String, Object>();
+ extensions.put("location", user.getLocation());
--- End diff --
I am not sure the Location properties in streams currently match 1:1 with twitter...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-streams/pull/25
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by robdouglas <gi...@git.apache.org>.
Github user robdouglas commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/25#discussion_r12961501
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/serializer/TwitterJsonActivitySerializer.java ---
@@ -81,6 +87,48 @@ else if( documentSubType == Delete.class )
throw new NotImplementedException();
}
+ public static Actor buildActorRetweet(Retweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActorTweet(Tweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActor(User user) {
+ Actor actor = new Actor();
+
+ actor.setId(formatId(
+ Optional.fromNullable(
+ user.getIdStr())
+ .or(Optional.of(user.getId().toString()))
+ .orNull()
+ ));
+ actor.setDisplayName(user.getScreenName());
+ if (user.getUrl()!=null){
+ actor.setUrl(user.getUrl());
+ }
+
+ actor.setDisplayName(user.getName());
+ actor.setSummary(user.getDescription());
+
+ Map<String, Object> extensions = new HashMap<String, Object>();
+ extensions.put("location", user.getLocation());
+ extensions.put("posts", user.getStatusesCount());
+ extensions.put("favorites", user.getFavouritesCount());
--- End diff --
Correct, however the extensions here are for the actors. So it's only recording the total number of: posts, favorites and followers for this actor at this particular time. The perspectival flag is included in the addTwitterExtensions function in both the Tweet and Retweet serializers
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by mfranklin <gi...@git.apache.org>.
Github user mfranklin commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/25#discussion_r13079542
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/serializer/TwitterJsonActivitySerializer.java ---
@@ -81,6 +87,86 @@ else if( documentSubType == Delete.class )
throw new NotImplementedException();
}
+ public static Actor buildActorRetweet(Retweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActorTweet(Tweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActor(User user) {
+ Actor actor = new Actor();
+
+ actor.setId(formatId(
+ Optional.fromNullable(
+ user.getIdStr())
+ .or(Optional.of(user.getId().toString()))
+ .orNull()
+ ));
+ actor.setDisplayName(user.getScreenName());
--- End diff --
This is overridden below. I think we need to keep both in the object. In this case, an extensions may not make sense. I think we update the activity to have a handle property that stores screen names.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by robdouglas <gi...@git.apache.org>.
Github user robdouglas commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/25#discussion_r12961833
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/serializer/TwitterJsonTweetActivitySerializer.java ---
@@ -78,35 +80,71 @@ public Activity deserialize(String serialized) throws ActivitySerializerExceptio
activity.setProvider(getProvider());
activity.setTitle("");
activity.setContent(tweet.getText());
- activity.setUrl("http://twitter.com/" + tweet.getIdStr());
+ activity.setUrl("http://twitter.com/" + tweet.getUser().getIdStr() + "/status/" + tweet.getIdStr());
activity.setLinks(getLinks(tweet));
addTwitterExtension(activity, mapper.convertValue(tweet, ObjectNode.class));
addLocationExtension(activity, tweet);
+ addTwitterExtensions(activity, tweet);
+
return activity;
}
+ public static void addTwitterExtensions(Activity activity, Tweet tweet) {
--- End diff --
Valid point, pushed change
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by robdouglas <gi...@git.apache.org>.
Github user robdouglas commented on the pull request:
https://github.com/apache/incubator-streams/pull/25#issuecomment-43942479
Removed URL addition in extension attribute
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by robdouglas <gi...@git.apache.org>.
Github user robdouglas commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/25#discussion_r13081342
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/serializer/TwitterJsonActivitySerializer.java ---
@@ -81,6 +87,86 @@ else if( documentSubType == Delete.class )
throw new NotImplementedException();
}
+ public static Actor buildActorRetweet(Retweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActorTweet(Tweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActor(User user) {
+ Actor actor = new Actor();
+
+ actor.setId(formatId(
+ Optional.fromNullable(
+ user.getIdStr())
+ .or(Optional.of(user.getId().toString()))
+ .orNull()
+ ));
+ actor.setDisplayName(user.getScreenName());
--- End diff --
Should I take the screen name out of the actor's extensions then?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by mfranklin <gi...@git.apache.org>.
Github user mfranklin commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/25#discussion_r13085712
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/serializer/util/TwitterActivityUtil.java ---
@@ -174,10 +176,28 @@ public static Actor buildActor(Tweet tweet) {
.or(Optional.of(user.getId().toString()))
.orNull()
));
- actor.setDisplayName(user.getScreenName());
+
+ actor.setDisplayName(user.getName());
+ actor.setAdditionalProperty("handle", user.getScreenName());
+ actor.setSummary(user.getDescription());
+
if (user.getUrl()!=null){
actor.setUrl(user.getUrl());
}
+
+ Map<String, Object> extensions = new HashMap<String, Object>();
+ extensions.put("location", user.getLocation());
+ extensions.put("posts", user.getStatusesCount());
+ extensions.put("favorites", user.getFavouritesCount());
+ extensions.put("followers", user.getFollowersCount());
+
+ Map<String, Object> image = new HashMap<String, Object>();
+ image.put("url", user.getProfileImageUrlHttps());
--- End diff --
I believe the profile image should be a media link in the image property of the actor object per
http://activitystrea.ms/specs/json/1.0/#object
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by mfranklin <gi...@git.apache.org>.
Github user mfranklin commented on the pull request:
https://github.com/apache/incubator-streams/pull/25#issuecomment-43940562
I agree that it should not import URLs
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by mfranklin <gi...@git.apache.org>.
Github user mfranklin commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/25#discussion_r12972642
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/serializer/TwitterJsonActivitySerializer.java ---
@@ -81,6 +87,48 @@ else if( documentSubType == Delete.class )
throw new NotImplementedException();
}
+ public static Actor buildActorRetweet(Retweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActorTweet(Tweet tweet) {
+ return buildActor(tweet.getUser());
+ }
+
+ public static Actor buildActor(User user) {
+ Actor actor = new Actor();
+
+ actor.setId(formatId(
+ Optional.fromNullable(
+ user.getIdStr())
+ .or(Optional.of(user.getId().toString()))
+ .orNull()
+ ));
+ actor.setDisplayName(user.getScreenName());
+ if (user.getUrl()!=null){
+ actor.setUrl(user.getUrl());
+ }
+
+ actor.setDisplayName(user.getName());
+ actor.setSummary(user.getDescription());
+
+ Map<String, Object> extensions = new HashMap<String, Object>();
+ extensions.put("location", user.getLocation());
--- End diff --
Again, I was thinking the actual activity not user object. This is fine
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request: STREAMS-93 | Updated the Hashtag P...
Posted by mfranklin <gi...@git.apache.org>.
Github user mfranklin commented on the pull request:
https://github.com/apache/incubator-streams/pull/25#issuecomment-44280074
:+1:
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---