You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@streams.apache.org by steveblackmon <gi...@git.apache.org> on 2017/04/03 20:20:37 UTC
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
GitHub user steveblackmon opened a pull request:
https://github.com/apache/incubator-streams/pull/360
STREAMS-496: Remove twitter4j dependency from streams-provider-twitter
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/steveblackmon/incubator-streams STREAMS-496
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-streams/pull/360.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #360
----
commit 7dedbb42251e1c5214690e32f12a5efb5032dc92
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-03-20T06:35:33Z
STREAMS-496: WIP
commit a8fd796096296bb2e8af61b722c4a261acbba987
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-03-20T18:07:27Z
more WIP
commit 68d7976f928173dbd6de636c58d1a23dbe95eb15
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-03-24T18:39:52Z
STREAMS-496: additional pojos and interfaces needed to maintain features
commit c95bccd41b4f5f51d7e614e5927210458d1dbfbd
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-03-24T19:21:34Z
implementations using RestClient
commit 78f30284caefb09e06818a9f435c3f380ee986a3
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-03-25T16:38:06Z
STREAMS-496: WIP - twitter4j gone, src/main/java compiles
commit 0d172e76232e3d0bd48af8d6616716c444eb94f0
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-03-26T17:56:11Z
STREAMS-496: consolidate httpcomponents version selection
commit 434657439abc0dde2998a8cb213d8e8398f43ad6
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-03-26T17:56:34Z
first implementation of oauth signing
commit 1885090c6d683c8a9a6df7c64f933529378ee965
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-03-26T18:09:24Z
cleanup authentication. debugging \u2018Bad Authentication data.\u2019
commit edc2662fd68b9e1e04c20b8721a44a6ee9fb36e6
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-03-26T21:20:10Z
adding unit testing to debug authentication
commit 5a07c57b553f8c87af1d42e0bbef4f0795f17bf7
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-03-26T22:44:37Z
STREAMS-496: more unit tests, all passing, ITs still fail
commit c623f7139421049d5b536576c739b60e959cf11e
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-03-26T22:55:09Z
STREAMS-496: going through guide word-by-word and debugging line-by-line looking for mistakes
commit eddf003ba5b5555b10c3d51267c00225b6da4ef1
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-03-27T03:46:19Z
TwitterOAuthRequestInterceptorTest.testProcess
TwitterOAuthRequestInterceptorTest.testProcess is pretty much directly off the twitter dev guidelines and tests the entire auth process at once. passes. ITs still fail
commit 06845f2dd848c91971c10f194f0d5503d5f4a964
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-03-28T18:20:03Z
return empty array not null on error
commit 517b85f02e93c51d5fa36a704f7f0e0e2cdaed4e
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-04-03T16:01:25Z
oauth signing working
commit 069be4da745b9cd262159cca1982a9b52cb5cefc
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-04-03T19:56:01Z
all integration tests passing using juneau 6.1.0-incubating
commit d7cd07110d602d24a07aad8e8f390ff0d98f1e19
Author: Steve Blackmon @steveblackmon <sb...@apache.org>
Date: 2017-04-03T20:11:32Z
checkstyle clean up
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by jfrazee <gi...@git.apache.org>.
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111298262
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Friends.java ---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.twitter.pojo.Tweet;
+
+import java.util.List;
+
+/**
+ * Returns a collection of the most recent Tweets posted by the user indicated by the screen_name or user_id parameters.
--- End diff --
Same as above, javadoc needs fixed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams issue #360: STREAMS-496: Remove twitter4j dependency from ...
Posted by steveblackmon <gi...@git.apache.org>.
Github user steveblackmon commented on the issue:
https://github.com/apache/incubator-streams/pull/360
addressed all PR feedback and double-checked that build and tests pass. merging
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by jfrazee <gi...@git.apache.org>.
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111308292
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/TwitterOAuthRequestInterceptor.java ---
@@ -0,0 +1,235 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.twitter.TwitterOAuthConfiguration;
+
+import org.apache.http.HttpException;
+import org.apache.http.HttpRequest;
+import org.apache.http.HttpRequestInterceptor;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpRequestWrapper;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.protocol.HttpContext;
+import org.apache.http.util.EntityUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import sun.misc.BASE64Encoder;
+
+import java.io.IOException;
+import java.io.UnsupportedEncodingException;
+import java.net.URLDecoder;
+import java.net.URLEncoder;
+import java.security.GeneralSecurityException;
+import java.util.Calendar;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.SortedSet;
+import java.util.StringJoiner;
+import java.util.TreeSet;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import javax.crypto.Mac;
+import javax.crypto.SecretKey;
+import javax.crypto.spec.SecretKeySpec;
+
+/**
+ * Handles request signing to api.twitter.com
+ */
+public class TwitterOAuthRequestInterceptor implements HttpRequestInterceptor {
--- End diff --
I think this interceptor could be simplified a bunch using https://github.com/mttkay/signpost or stuff from https://oltu.apache.org. Fine to leave for another day though.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-streams/pull/360
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by steveblackmon <gi...@git.apache.org>.
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111306821
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Followers.java ---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.twitter.pojo.Tweet;
+
+import java.util.List;
+
+/**
+ * Returns a collection of the most recent Tweets posted by the user indicated by the screen_name or user_id parameters.
+ *
+ * @see <a href="https://dev.twitter.com/rest/reference/get/statuses/user_timeline">https://api.twitter.com/1.1/statuses/user_timeline.json</a>
+ */
+public interface Followers {
+
+ /**
+ * Returns a cursored collection of user IDs for every user following the specified user.
+ *
+ * @param parameters StatusesLookupRequest
+ * @return List < Tweet >
+ * @see <a href="https://api.twitter.com/1.1/followers/ids.json">https://api.twitter.com/1.1/followers/ids.json</a>
--- End diff --
good point, just made this change across all interfaces
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by steveblackmon <gi...@git.apache.org>.
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111310313
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Twitter.java ---
@@ -0,0 +1,521 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.jackson.StreamsJacksonMapper;
+import org.apache.streams.twitter.TwitterConfiguration;
+import org.apache.streams.twitter.pojo.Tweet;
+import org.apache.streams.twitter.pojo.User;
+import org.apache.streams.twitter.provider.TwitterProviderUtil;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.HttpRequestInterceptor;
+import org.apache.http.client.HttpRequestRetryHandler;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.params.ClientPNames;
+import org.apache.http.client.params.CookiePolicy;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.juneau.json.JsonParser;
+import org.apache.juneau.parser.ParseException;
+import org.apache.juneau.plaintext.PlainTextSerializer;
+import org.apache.juneau.rest.client.RestCall;
+import org.apache.juneau.rest.client.RestCallException;
+import org.apache.juneau.rest.client.RestClient;
+//import org.apache.juneau.rest.client.RestClientBuilder;
+import org.apache.juneau.rest.client.RetryOn;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URISyntaxException;
+import java.util.ArrayList;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.ConcurrentHashMap;
+
+/**
+ * Implementation of all twitter interfaces using juneau.
+ */
+public class Twitter implements Followers, Friends, Statuses, Users {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(Twitter.class);
+
+ private static Map<TwitterConfiguration, Twitter> INSTANCE_MAP = new ConcurrentHashMap<>();
+
+ private TwitterConfiguration configuration;
+
+ private ObjectMapper mapper;
+
+ private String rootUrl;
+
+ private CloseableHttpClient httpclient;
+
+ private HttpRequestInterceptor oauthInterceptor;
+
+ RestClient restClient;
+
+ private Twitter(TwitterConfiguration configuration) throws InstantiationException {
+ this.configuration = configuration;
+ this.rootUrl = TwitterProviderUtil.baseUrl(configuration);
+ this.oauthInterceptor = new TwitterOAuthRequestInterceptor(configuration.getOauth());
+ this.httpclient = HttpClientBuilder.create()
+ .addInterceptorFirst(oauthInterceptor)
+ .setDefaultRequestConfig(RequestConfig.custom()
+ .setConnectionRequestTimeout(5000)
+ .setConnectTimeout(5000)
+ .setSocketTimeout(5000)
+ .setCookieSpec("easy")
+ .build()
+ )
+ .setMaxConnPerRoute(20)
+ .setMaxConnTotal(100)
+ .build();
+
+// TODO: juneau-6.2.0-incubating
+// this.restClient = new RestClientBuilder()
+// .httpClient(httpclient, true)
+// .parser(JsonParser.class)
+// .rootUrl(rootUrl)
+// .retryable(
+// configuration.getRetryMax().intValue(),
+// configuration.getRetrySleepMs(),
+// new TwitterRetryHandler())
+// .build();
+ this.restClient = new RestClient()
+ .setHttpClient(httpclient)
+ .setParser(JsonParser.class)
+ .setRootUrl(rootUrl);
+
+ this.mapper = StreamsJacksonMapper.getInstance();
+ }
+
+ public static Twitter getInstance(TwitterConfiguration configuration) throws InstantiationException {
+ if (INSTANCE_MAP.containsKey(configuration) && INSTANCE_MAP.get(configuration) != null) {
+ return INSTANCE_MAP.get(configuration);
+ } else {
+ Twitter twitter = new Twitter(configuration);
+ INSTANCE_MAP.put(configuration, twitter);
+ return INSTANCE_MAP.get(configuration);
+ }
+ }
+
+ @Override
+ public List<Tweet> userTimeline(StatusesUserTimelineRequest parameters) {
+ try {
+// TODO: juneau-6.2.0-incubating
+// Statuses restStatuses = restClient.getRemoteableProxy("/statuses/user_timeline.json", Statuses.class);
+// List<Tweet> result = restStatuses.userTimeline(parameters);
+// return result;
+ URIBuilder uriBuilder = new URIBuilder()
+ .setPath("/statuses/user_timeline.json");
+ if( Objects.nonNull(parameters.getUserId()) && StringUtils.isNotBlank(parameters.getUserId().toString())) {
+ uriBuilder.addParameter("user_id", parameters.getUserId().toString());
+ }
+ if( Objects.nonNull(parameters.getScreenName()) && StringUtils.isNotBlank(parameters.getScreenName())) {
+ uriBuilder.addParameter("screen_name", parameters.getScreenName());
+ }
+ if( Objects.nonNull(parameters.getSinceId()) && StringUtils.isNotBlank(parameters.getSinceId().toString())) {
+ uriBuilder.addParameter("since_id", parameters.getSinceId().toString());
+ }
+ if( Objects.nonNull(parameters.getCount()) && StringUtils.isNotBlank(parameters.getCount().toString())) {
+ uriBuilder.addParameter("count", parameters.getCount().toString());
+ }
+ if( Objects.nonNull(parameters.getMaxId()) && StringUtils.isNotBlank(parameters.getMaxId().toString())) {
+ uriBuilder.addParameter("max_id", parameters.getMaxId().toString());
+ }
+ if( Objects.nonNull(parameters.getTrimUser()) && StringUtils.isNotBlank(parameters.getTrimUser().toString())) {
+ uriBuilder.addParameter("trim_user", parameters.getTrimUser().toString());
+ }
+ if( Objects.nonNull(parameters.getExcludeReplies()) && StringUtils.isNotBlank(parameters.getExcludeReplies().toString())) {
+ uriBuilder.addParameter("exclude_replies", parameters.getExcludeReplies().toString());
+ }
+ if( Objects.nonNull(parameters.getContributorDetails()) && StringUtils.isNotBlank(parameters.getContributorDetails().toString())) {
+ uriBuilder.addParameter("contributor_details", parameters.getContributorDetails().toString());
+ }
+ if( Objects.nonNull(parameters.getIncludeRts()) && StringUtils.isNotBlank(parameters.getIncludeRts().toString())) {
+ uriBuilder.addParameter("include_rts", parameters.getIncludeRts().toString());
+ }
+ RestCall restCall = restClient.doGet(uriBuilder.build().toString());
+ try {
+ String restResponseEntity = restCall
+ .setRetryable(configuration.getRetryMax().intValue(), configuration.getRetrySleepMs().intValue(), new TwitterRetryHandler())
+ .getResponseAsString();
+ ArrayNode resultArrayNode = mapper.readValue(restResponseEntity, ArrayNode.class);
+ List<Tweet> result = new ArrayList();
+ resultArrayNode.iterator().forEachRemaining(item -> result.add(mapper.convertValue(item, Tweet.class)));
+ return result;
+ } catch (RestCallException e) {
+ LOGGER.warn("RestCallException", e);
+ }
+ } catch (IOException e) {
+ LOGGER.warn("IOException", e);
+ } catch (URISyntaxException e) {
+ LOGGER.warn("URISyntaxException", e);
+ }
+ return new ArrayList<>();
+ }
+
+ @Override
+ public List<Tweet> lookup(StatusesLookupRequest parameters) {
+// TODO: juneau-6.2.0-incubating
+// Statuses restStatuses = restClient.getRemoteableProxy("/statuses/lookup.json", Statuses.class);
+// List<Tweet> result = restStatuses.lookup(parameters);
+// return result;
+ String ids = StringUtils.join(parameters.getId(), ',');
+ try {
+ URIBuilder uriBuilder = new URIBuilder()
+ .setPath("/statuses/lookup.json");
+ if( Objects.nonNull(parameters.getId()) && StringUtils.isNotBlank(parameters.getId().toString())) {
+ uriBuilder.addParameter("id", parameters.getId().toString());
+ }
+ if( Objects.nonNull(parameters.getTrimUser()) && StringUtils.isNotBlank(parameters.getTrimUser().toString())) {
+ uriBuilder.addParameter("trim_user", parameters.getTrimUser().toString());
+ }
+ if( Objects.nonNull(parameters.getIncludeEntities()) && StringUtils.isNotBlank(parameters.getIncludeEntities().toString())) {
+ uriBuilder.addParameter("include_entities", parameters.getIncludeEntities().toString());
+ }
+ if( Objects.nonNull(parameters.getMap()) && StringUtils.isNotBlank(parameters.getMap().toString())) {
+ uriBuilder.addParameter("map", parameters.getMap().toString());
+ }
+ RestCall restCall = restClient.doGet(uriBuilder.build().toString());
+ try {
+ String restResponseEntity = restCall
+ .setRetryable(configuration.getRetryMax().intValue(), configuration.getRetrySleepMs().intValue(), new TwitterRetryHandler())
+ .getResponseAsString();
+ ArrayNode resultArrayNode = mapper.readValue(restResponseEntity, ArrayNode.class);
+ List<Tweet> result = new ArrayList();
+ resultArrayNode.iterator().forEachRemaining(item -> result.add(mapper.convertValue(item, Tweet.class)));
+ //List<Tweet> result = restCall.getResponse(LinkedList.class, Tweet.class);
+ return result;
+ } catch (RestCallException e) {
+ LOGGER.warn("RestCallException", e);
+ }
+ } catch (IOException e) {
+ LOGGER.warn("IOException", e);
+ } catch (URISyntaxException e) {
+ LOGGER.warn("URISyntaxException", e);
+ }
+ return new ArrayList<>();
+ }
+
+ @Override
+ public Tweet show(StatusesShowRequest parameters) {
+// TODO: juneau-6.2.0-incubating
+// Statuses restStatuses = restClient.getRemoteableProxy("/statuses/show.json", Statuses.class);
+// Tweet result = restStatuses.show(parameters);
+// return result;
+ try {
+ URIBuilder uriBuilder = new URIBuilder()
+ .setPath("/statuses/show.json");
+ if (Objects.nonNull(parameters.getId()) && StringUtils.isNotBlank(parameters.getId().toString())) {
+ uriBuilder.addParameter("id", parameters.getId().toString());
+ }
+ if (Objects.nonNull(parameters.getTrimUser()) && StringUtils.isNotBlank(parameters.getTrimUser().toString())) {
+ uriBuilder.addParameter("trim_user", parameters.getTrimUser().toString());
+ }
+ if (Objects.nonNull(parameters.getIncludeEntities()) && StringUtils.isNotBlank(parameters.getIncludeEntities().toString())) {
+ uriBuilder.addParameter("include_entities", parameters.getIncludeEntities().toString());
+ }
+ if (Objects.nonNull(parameters.getIncludeMyRetweet()) && StringUtils.isNotBlank(parameters.getIncludeMyRetweet().toString())) {
+ uriBuilder.addParameter("include_my_retweet", parameters.getIncludeMyRetweet().toString());
+ }
+ RestCall restCall = restClient.doGet(uriBuilder.build().toString());
+ try {
+ String restResponseEntity = restCall
+ .setRetryable(configuration.getRetryMax().intValue(), configuration.getRetrySleepMs().intValue(), new TwitterRetryHandler())
+ .getResponseAsString();
+ //Tweet result = restCall.getResponse(Tweet.class);
+ Tweet result = mapper.readValue(restResponseEntity, Tweet.class);
+ return result;
+ } catch (RestCallException e) {
+ LOGGER.warn("RestCallException", e);
+ }
+ } catch (IOException e) {
+ LOGGER.warn("IOException", e);
+ } catch (URISyntaxException e) {
+ LOGGER.warn("URISyntaxException", e);
+ }
+ return null;
+ }
+
+ @Override
+ public FriendsIdsResponse ids(FriendsIdsRequest parameters) {
+// TODO: juneau-6.2.0-incubating
+// Friends restFriends = restClient.getRemoteableProxy("/friends/ids.json", Friends.class);
+// FriendsIdsResponse result = restFriends.ids(parameters);
+// return result;
+ try {
+ URIBuilder uriBuilder = new URIBuilder()
+ .setPath("/friends/ids.json");
+ if( Objects.nonNull(parameters.getCount()) && StringUtils.isNotBlank(parameters.getCount().toString())) {
+ uriBuilder.addParameter("count", parameters.getCount().toString());
+ }
+ if( Objects.nonNull(parameters.getCurser()) && StringUtils.isNotBlank(parameters.getCurser().toString())) {
--- End diff --
nice catch. that was wrong a bunch of places. all fixed now and i watch the providers page as expected.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by steveblackmon <gi...@git.apache.org>.
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111309069
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Twitter.java ---
@@ -0,0 +1,521 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.jackson.StreamsJacksonMapper;
+import org.apache.streams.twitter.TwitterConfiguration;
+import org.apache.streams.twitter.pojo.Tweet;
+import org.apache.streams.twitter.pojo.User;
+import org.apache.streams.twitter.provider.TwitterProviderUtil;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.HttpRequestInterceptor;
+import org.apache.http.client.HttpRequestRetryHandler;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.params.ClientPNames;
+import org.apache.http.client.params.CookiePolicy;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.juneau.json.JsonParser;
+import org.apache.juneau.parser.ParseException;
+import org.apache.juneau.plaintext.PlainTextSerializer;
+import org.apache.juneau.rest.client.RestCall;
+import org.apache.juneau.rest.client.RestCallException;
+import org.apache.juneau.rest.client.RestClient;
+//import org.apache.juneau.rest.client.RestClientBuilder;
+import org.apache.juneau.rest.client.RetryOn;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URISyntaxException;
+import java.util.ArrayList;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.ConcurrentHashMap;
+
+/**
+ * Implementation of all twitter interfaces using juneau.
+ */
+public class Twitter implements Followers, Friends, Statuses, Users {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(Twitter.class);
+
+ private static Map<TwitterConfiguration, Twitter> INSTANCE_MAP = new ConcurrentHashMap<>();
+
+ private TwitterConfiguration configuration;
+
+ private ObjectMapper mapper;
+
+ private String rootUrl;
+
+ private CloseableHttpClient httpclient;
+
+ private HttpRequestInterceptor oauthInterceptor;
+
+ RestClient restClient;
+
+ private Twitter(TwitterConfiguration configuration) throws InstantiationException {
+ this.configuration = configuration;
+ this.rootUrl = TwitterProviderUtil.baseUrl(configuration);
+ this.oauthInterceptor = new TwitterOAuthRequestInterceptor(configuration.getOauth());
+ this.httpclient = HttpClientBuilder.create()
+ .addInterceptorFirst(oauthInterceptor)
+ .setDefaultRequestConfig(RequestConfig.custom()
+ .setConnectionRequestTimeout(5000)
+ .setConnectTimeout(5000)
+ .setSocketTimeout(5000)
+ .setCookieSpec("easy")
+ .build()
+ )
+ .setMaxConnPerRoute(20)
+ .setMaxConnTotal(100)
+ .build();
+
+// TODO: juneau-6.2.0-incubating
+// this.restClient = new RestClientBuilder()
+// .httpClient(httpclient, true)
+// .parser(JsonParser.class)
+// .rootUrl(rootUrl)
+// .retryable(
+// configuration.getRetryMax().intValue(),
+// configuration.getRetrySleepMs(),
+// new TwitterRetryHandler())
+// .build();
+ this.restClient = new RestClient()
+ .setHttpClient(httpclient)
+ .setParser(JsonParser.class)
+ .setRootUrl(rootUrl);
+
+ this.mapper = StreamsJacksonMapper.getInstance();
+ }
+
+ public static Twitter getInstance(TwitterConfiguration configuration) throws InstantiationException {
+ if (INSTANCE_MAP.containsKey(configuration) && INSTANCE_MAP.get(configuration) != null) {
+ return INSTANCE_MAP.get(configuration);
+ } else {
+ Twitter twitter = new Twitter(configuration);
+ INSTANCE_MAP.put(configuration, twitter);
+ return INSTANCE_MAP.get(configuration);
+ }
+ }
+
+ @Override
+ public List<Tweet> userTimeline(StatusesUserTimelineRequest parameters) {
+ try {
+// TODO: juneau-6.2.0-incubating
+// Statuses restStatuses = restClient.getRemoteableProxy("/statuses/user_timeline.json", Statuses.class);
+// List<Tweet> result = restStatuses.userTimeline(parameters);
+// return result;
+ URIBuilder uriBuilder = new URIBuilder()
+ .setPath("/statuses/user_timeline.json");
+ if( Objects.nonNull(parameters.getUserId()) && StringUtils.isNotBlank(parameters.getUserId().toString())) {
--- End diff --
probably true for the string parameters, not true for the non-string parameters b/c the toString method can throw NPE
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by steveblackmon <gi...@git.apache.org>.
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111310112
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/TwitterOAuthRequestInterceptor.java ---
@@ -0,0 +1,235 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.twitter.TwitterOAuthConfiguration;
+
+import org.apache.http.HttpException;
+import org.apache.http.HttpRequest;
+import org.apache.http.HttpRequestInterceptor;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.client.methods.HttpRequestWrapper;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.protocol.HttpContext;
+import org.apache.http.util.EntityUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import sun.misc.BASE64Encoder;
+
+import java.io.IOException;
+import java.io.UnsupportedEncodingException;
+import java.net.URLDecoder;
+import java.net.URLEncoder;
+import java.security.GeneralSecurityException;
+import java.util.Calendar;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.SortedSet;
+import java.util.StringJoiner;
+import java.util.TreeSet;
+import java.util.UUID;
+import java.util.concurrent.ConcurrentHashMap;
+import javax.crypto.Mac;
+import javax.crypto.SecretKey;
+import javax.crypto.spec.SecretKeySpec;
+
+/**
+ * Handles request signing to api.twitter.com
+ */
+public class TwitterOAuthRequestInterceptor implements HttpRequestInterceptor {
--- End diff --
that would be terrific. writing and testing classes like this is not particularly rewarding (for me anyway).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by steveblackmon <gi...@git.apache.org>.
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111310493
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Twitter.java ---
@@ -0,0 +1,521 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.jackson.StreamsJacksonMapper;
+import org.apache.streams.twitter.TwitterConfiguration;
+import org.apache.streams.twitter.pojo.Tweet;
+import org.apache.streams.twitter.pojo.User;
+import org.apache.streams.twitter.provider.TwitterProviderUtil;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.HttpRequestInterceptor;
+import org.apache.http.client.HttpRequestRetryHandler;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.params.ClientPNames;
+import org.apache.http.client.params.CookiePolicy;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.juneau.json.JsonParser;
+import org.apache.juneau.parser.ParseException;
+import org.apache.juneau.plaintext.PlainTextSerializer;
+import org.apache.juneau.rest.client.RestCall;
+import org.apache.juneau.rest.client.RestCallException;
+import org.apache.juneau.rest.client.RestClient;
+//import org.apache.juneau.rest.client.RestClientBuilder;
+import org.apache.juneau.rest.client.RetryOn;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URISyntaxException;
+import java.util.ArrayList;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.ConcurrentHashMap;
+
+/**
+ * Implementation of all twitter interfaces using juneau.
+ */
+public class Twitter implements Followers, Friends, Statuses, Users {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(Twitter.class);
+
+ private static Map<TwitterConfiguration, Twitter> INSTANCE_MAP = new ConcurrentHashMap<>();
+
+ private TwitterConfiguration configuration;
+
+ private ObjectMapper mapper;
+
+ private String rootUrl;
+
+ private CloseableHttpClient httpclient;
+
+ private HttpRequestInterceptor oauthInterceptor;
+
+ RestClient restClient;
+
+ private Twitter(TwitterConfiguration configuration) throws InstantiationException {
+ this.configuration = configuration;
+ this.rootUrl = TwitterProviderUtil.baseUrl(configuration);
+ this.oauthInterceptor = new TwitterOAuthRequestInterceptor(configuration.getOauth());
+ this.httpclient = HttpClientBuilder.create()
+ .addInterceptorFirst(oauthInterceptor)
+ .setDefaultRequestConfig(RequestConfig.custom()
+ .setConnectionRequestTimeout(5000)
+ .setConnectTimeout(5000)
+ .setSocketTimeout(5000)
+ .setCookieSpec("easy")
+ .build()
+ )
+ .setMaxConnPerRoute(20)
+ .setMaxConnTotal(100)
+ .build();
+
+// TODO: juneau-6.2.0-incubating
+// this.restClient = new RestClientBuilder()
+// .httpClient(httpclient, true)
+// .parser(JsonParser.class)
+// .rootUrl(rootUrl)
+// .retryable(
+// configuration.getRetryMax().intValue(),
+// configuration.getRetrySleepMs(),
+// new TwitterRetryHandler())
+// .build();
+ this.restClient = new RestClient()
+ .setHttpClient(httpclient)
+ .setParser(JsonParser.class)
+ .setRootUrl(rootUrl);
+
+ this.mapper = StreamsJacksonMapper.getInstance();
+ }
+
+ public static Twitter getInstance(TwitterConfiguration configuration) throws InstantiationException {
+ if (INSTANCE_MAP.containsKey(configuration) && INSTANCE_MAP.get(configuration) != null) {
+ return INSTANCE_MAP.get(configuration);
+ } else {
+ Twitter twitter = new Twitter(configuration);
+ INSTANCE_MAP.put(configuration, twitter);
+ return INSTANCE_MAP.get(configuration);
+ }
+ }
+
+ @Override
+ public List<Tweet> userTimeline(StatusesUserTimelineRequest parameters) {
+ try {
+// TODO: juneau-6.2.0-incubating
+// Statuses restStatuses = restClient.getRemoteableProxy("/statuses/user_timeline.json", Statuses.class);
+// List<Tweet> result = restStatuses.userTimeline(parameters);
+// return result;
+ URIBuilder uriBuilder = new URIBuilder()
+ .setPath("/statuses/user_timeline.json");
+ if( Objects.nonNull(parameters.getUserId()) && StringUtils.isNotBlank(parameters.getUserId().toString())) {
--- End diff --
removed the redundant checks for String parameters
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by steveblackmon <gi...@git.apache.org>.
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111306857
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Followers.java ---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.twitter.pojo.Tweet;
+
+import java.util.List;
+
+/**
+ * Returns a collection of the most recent Tweets posted by the user indicated by the screen_name or user_id parameters.
--- End diff --
edited all interface headers for accuracy
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by steveblackmon <gi...@git.apache.org>.
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111306855
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Friends.java ---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.twitter.pojo.Tweet;
+
+import java.util.List;
+
+/**
+ * Returns a collection of the most recent Tweets posted by the user indicated by the screen_name or user_id parameters.
--- End diff --
edited all interface headers for accuracy
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by jfrazee <gi...@git.apache.org>.
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111307991
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Twitter.java ---
@@ -0,0 +1,521 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.jackson.StreamsJacksonMapper;
+import org.apache.streams.twitter.TwitterConfiguration;
+import org.apache.streams.twitter.pojo.Tweet;
+import org.apache.streams.twitter.pojo.User;
+import org.apache.streams.twitter.provider.TwitterProviderUtil;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.HttpRequestInterceptor;
+import org.apache.http.client.HttpRequestRetryHandler;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.params.ClientPNames;
+import org.apache.http.client.params.CookiePolicy;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.juneau.json.JsonParser;
+import org.apache.juneau.parser.ParseException;
+import org.apache.juneau.plaintext.PlainTextSerializer;
+import org.apache.juneau.rest.client.RestCall;
+import org.apache.juneau.rest.client.RestCallException;
+import org.apache.juneau.rest.client.RestClient;
+//import org.apache.juneau.rest.client.RestClientBuilder;
+import org.apache.juneau.rest.client.RetryOn;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URISyntaxException;
+import java.util.ArrayList;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.ConcurrentHashMap;
+
+/**
+ * Implementation of all twitter interfaces using juneau.
+ */
+public class Twitter implements Followers, Friends, Statuses, Users {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(Twitter.class);
+
+ private static Map<TwitterConfiguration, Twitter> INSTANCE_MAP = new ConcurrentHashMap<>();
+
+ private TwitterConfiguration configuration;
+
+ private ObjectMapper mapper;
+
+ private String rootUrl;
+
+ private CloseableHttpClient httpclient;
+
+ private HttpRequestInterceptor oauthInterceptor;
+
+ RestClient restClient;
+
+ private Twitter(TwitterConfiguration configuration) throws InstantiationException {
+ this.configuration = configuration;
+ this.rootUrl = TwitterProviderUtil.baseUrl(configuration);
+ this.oauthInterceptor = new TwitterOAuthRequestInterceptor(configuration.getOauth());
+ this.httpclient = HttpClientBuilder.create()
+ .addInterceptorFirst(oauthInterceptor)
+ .setDefaultRequestConfig(RequestConfig.custom()
+ .setConnectionRequestTimeout(5000)
+ .setConnectTimeout(5000)
+ .setSocketTimeout(5000)
+ .setCookieSpec("easy")
+ .build()
+ )
+ .setMaxConnPerRoute(20)
+ .setMaxConnTotal(100)
+ .build();
+
+// TODO: juneau-6.2.0-incubating
+// this.restClient = new RestClientBuilder()
+// .httpClient(httpclient, true)
+// .parser(JsonParser.class)
+// .rootUrl(rootUrl)
+// .retryable(
+// configuration.getRetryMax().intValue(),
+// configuration.getRetrySleepMs(),
+// new TwitterRetryHandler())
+// .build();
+ this.restClient = new RestClient()
+ .setHttpClient(httpclient)
+ .setParser(JsonParser.class)
+ .setRootUrl(rootUrl);
+
+ this.mapper = StreamsJacksonMapper.getInstance();
+ }
+
+ public static Twitter getInstance(TwitterConfiguration configuration) throws InstantiationException {
+ if (INSTANCE_MAP.containsKey(configuration) && INSTANCE_MAP.get(configuration) != null) {
+ return INSTANCE_MAP.get(configuration);
+ } else {
+ Twitter twitter = new Twitter(configuration);
+ INSTANCE_MAP.put(configuration, twitter);
+ return INSTANCE_MAP.get(configuration);
+ }
+ }
+
+ @Override
+ public List<Tweet> userTimeline(StatusesUserTimelineRequest parameters) {
+ try {
+// TODO: juneau-6.2.0-incubating
+// Statuses restStatuses = restClient.getRemoteableProxy("/statuses/user_timeline.json", Statuses.class);
+// List<Tweet> result = restStatuses.userTimeline(parameters);
+// return result;
+ URIBuilder uriBuilder = new URIBuilder()
+ .setPath("/statuses/user_timeline.json");
+ if( Objects.nonNull(parameters.getUserId()) && StringUtils.isNotBlank(parameters.getUserId().toString())) {
+ uriBuilder.addParameter("user_id", parameters.getUserId().toString());
+ }
+ if( Objects.nonNull(parameters.getScreenName()) && StringUtils.isNotBlank(parameters.getScreenName())) {
+ uriBuilder.addParameter("screen_name", parameters.getScreenName());
+ }
+ if( Objects.nonNull(parameters.getSinceId()) && StringUtils.isNotBlank(parameters.getSinceId().toString())) {
+ uriBuilder.addParameter("since_id", parameters.getSinceId().toString());
+ }
+ if( Objects.nonNull(parameters.getCount()) && StringUtils.isNotBlank(parameters.getCount().toString())) {
+ uriBuilder.addParameter("count", parameters.getCount().toString());
+ }
+ if( Objects.nonNull(parameters.getMaxId()) && StringUtils.isNotBlank(parameters.getMaxId().toString())) {
+ uriBuilder.addParameter("max_id", parameters.getMaxId().toString());
+ }
+ if( Objects.nonNull(parameters.getTrimUser()) && StringUtils.isNotBlank(parameters.getTrimUser().toString())) {
+ uriBuilder.addParameter("trim_user", parameters.getTrimUser().toString());
+ }
+ if( Objects.nonNull(parameters.getExcludeReplies()) && StringUtils.isNotBlank(parameters.getExcludeReplies().toString())) {
+ uriBuilder.addParameter("exclude_replies", parameters.getExcludeReplies().toString());
+ }
+ if( Objects.nonNull(parameters.getContributorDetails()) && StringUtils.isNotBlank(parameters.getContributorDetails().toString())) {
+ uriBuilder.addParameter("contributor_details", parameters.getContributorDetails().toString());
+ }
+ if( Objects.nonNull(parameters.getIncludeRts()) && StringUtils.isNotBlank(parameters.getIncludeRts().toString())) {
+ uriBuilder.addParameter("include_rts", parameters.getIncludeRts().toString());
+ }
+ RestCall restCall = restClient.doGet(uriBuilder.build().toString());
+ try {
+ String restResponseEntity = restCall
+ .setRetryable(configuration.getRetryMax().intValue(), configuration.getRetrySleepMs().intValue(), new TwitterRetryHandler())
+ .getResponseAsString();
+ ArrayNode resultArrayNode = mapper.readValue(restResponseEntity, ArrayNode.class);
+ List<Tweet> result = new ArrayList();
+ resultArrayNode.iterator().forEachRemaining(item -> result.add(mapper.convertValue(item, Tweet.class)));
+ return result;
+ } catch (RestCallException e) {
+ LOGGER.warn("RestCallException", e);
+ }
+ } catch (IOException e) {
+ LOGGER.warn("IOException", e);
+ } catch (URISyntaxException e) {
+ LOGGER.warn("URISyntaxException", e);
+ }
+ return new ArrayList<>();
+ }
+
+ @Override
+ public List<Tweet> lookup(StatusesLookupRequest parameters) {
+// TODO: juneau-6.2.0-incubating
+// Statuses restStatuses = restClient.getRemoteableProxy("/statuses/lookup.json", Statuses.class);
+// List<Tweet> result = restStatuses.lookup(parameters);
+// return result;
+ String ids = StringUtils.join(parameters.getId(), ',');
+ try {
+ URIBuilder uriBuilder = new URIBuilder()
+ .setPath("/statuses/lookup.json");
+ if( Objects.nonNull(parameters.getId()) && StringUtils.isNotBlank(parameters.getId().toString())) {
+ uriBuilder.addParameter("id", parameters.getId().toString());
+ }
+ if( Objects.nonNull(parameters.getTrimUser()) && StringUtils.isNotBlank(parameters.getTrimUser().toString())) {
+ uriBuilder.addParameter("trim_user", parameters.getTrimUser().toString());
+ }
+ if( Objects.nonNull(parameters.getIncludeEntities()) && StringUtils.isNotBlank(parameters.getIncludeEntities().toString())) {
+ uriBuilder.addParameter("include_entities", parameters.getIncludeEntities().toString());
+ }
+ if( Objects.nonNull(parameters.getMap()) && StringUtils.isNotBlank(parameters.getMap().toString())) {
+ uriBuilder.addParameter("map", parameters.getMap().toString());
+ }
+ RestCall restCall = restClient.doGet(uriBuilder.build().toString());
+ try {
+ String restResponseEntity = restCall
+ .setRetryable(configuration.getRetryMax().intValue(), configuration.getRetrySleepMs().intValue(), new TwitterRetryHandler())
+ .getResponseAsString();
+ ArrayNode resultArrayNode = mapper.readValue(restResponseEntity, ArrayNode.class);
+ List<Tweet> result = new ArrayList();
+ resultArrayNode.iterator().forEachRemaining(item -> result.add(mapper.convertValue(item, Tweet.class)));
+ //List<Tweet> result = restCall.getResponse(LinkedList.class, Tweet.class);
+ return result;
+ } catch (RestCallException e) {
+ LOGGER.warn("RestCallException", e);
+ }
+ } catch (IOException e) {
+ LOGGER.warn("IOException", e);
+ } catch (URISyntaxException e) {
+ LOGGER.warn("URISyntaxException", e);
+ }
+ return new ArrayList<>();
+ }
+
+ @Override
+ public Tweet show(StatusesShowRequest parameters) {
+// TODO: juneau-6.2.0-incubating
+// Statuses restStatuses = restClient.getRemoteableProxy("/statuses/show.json", Statuses.class);
+// Tweet result = restStatuses.show(parameters);
+// return result;
+ try {
+ URIBuilder uriBuilder = new URIBuilder()
+ .setPath("/statuses/show.json");
+ if (Objects.nonNull(parameters.getId()) && StringUtils.isNotBlank(parameters.getId().toString())) {
+ uriBuilder.addParameter("id", parameters.getId().toString());
+ }
+ if (Objects.nonNull(parameters.getTrimUser()) && StringUtils.isNotBlank(parameters.getTrimUser().toString())) {
+ uriBuilder.addParameter("trim_user", parameters.getTrimUser().toString());
+ }
+ if (Objects.nonNull(parameters.getIncludeEntities()) && StringUtils.isNotBlank(parameters.getIncludeEntities().toString())) {
+ uriBuilder.addParameter("include_entities", parameters.getIncludeEntities().toString());
+ }
+ if (Objects.nonNull(parameters.getIncludeMyRetweet()) && StringUtils.isNotBlank(parameters.getIncludeMyRetweet().toString())) {
+ uriBuilder.addParameter("include_my_retweet", parameters.getIncludeMyRetweet().toString());
+ }
+ RestCall restCall = restClient.doGet(uriBuilder.build().toString());
+ try {
+ String restResponseEntity = restCall
+ .setRetryable(configuration.getRetryMax().intValue(), configuration.getRetrySleepMs().intValue(), new TwitterRetryHandler())
+ .getResponseAsString();
+ //Tweet result = restCall.getResponse(Tweet.class);
+ Tweet result = mapper.readValue(restResponseEntity, Tweet.class);
+ return result;
+ } catch (RestCallException e) {
+ LOGGER.warn("RestCallException", e);
+ }
+ } catch (IOException e) {
+ LOGGER.warn("IOException", e);
+ } catch (URISyntaxException e) {
+ LOGGER.warn("URISyntaxException", e);
+ }
+ return null;
+ }
+
+ @Override
+ public FriendsIdsResponse ids(FriendsIdsRequest parameters) {
+// TODO: juneau-6.2.0-incubating
+// Friends restFriends = restClient.getRemoteableProxy("/friends/ids.json", Friends.class);
+// FriendsIdsResponse result = restFriends.ids(parameters);
+// return result;
+ try {
+ URIBuilder uriBuilder = new URIBuilder()
+ .setPath("/friends/ids.json");
+ if( Objects.nonNull(parameters.getCount()) && StringUtils.isNotBlank(parameters.getCount().toString())) {
+ uriBuilder.addParameter("count", parameters.getCount().toString());
+ }
+ if( Objects.nonNull(parameters.getCurser()) && StringUtils.isNotBlank(parameters.getCurser().toString())) {
--- End diff --
I left a comment on FollowingIdsRequest.json too, but I don't think this parameter is doing anything since the API param is 'cursor'.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by jfrazee <gi...@git.apache.org>.
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111306867
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Twitter.java ---
@@ -0,0 +1,521 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.jackson.StreamsJacksonMapper;
+import org.apache.streams.twitter.TwitterConfiguration;
+import org.apache.streams.twitter.pojo.Tweet;
+import org.apache.streams.twitter.pojo.User;
+import org.apache.streams.twitter.provider.TwitterProviderUtil;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import org.apache.commons.lang3.StringUtils;
+import org.apache.http.HttpRequestInterceptor;
+import org.apache.http.client.HttpRequestRetryHandler;
+import org.apache.http.client.config.RequestConfig;
+import org.apache.http.client.params.ClientPNames;
+import org.apache.http.client.params.CookiePolicy;
+import org.apache.http.client.utils.URIBuilder;
+import org.apache.http.impl.client.CloseableHttpClient;
+import org.apache.http.impl.client.HttpClientBuilder;
+import org.apache.juneau.json.JsonParser;
+import org.apache.juneau.parser.ParseException;
+import org.apache.juneau.plaintext.PlainTextSerializer;
+import org.apache.juneau.rest.client.RestCall;
+import org.apache.juneau.rest.client.RestCallException;
+import org.apache.juneau.rest.client.RestClient;
+//import org.apache.juneau.rest.client.RestClientBuilder;
+import org.apache.juneau.rest.client.RetryOn;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.IOException;
+import java.net.URISyntaxException;
+import java.util.ArrayList;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
+import java.util.Objects;
+import java.util.concurrent.ConcurrentHashMap;
+
+/**
+ * Implementation of all twitter interfaces using juneau.
+ */
+public class Twitter implements Followers, Friends, Statuses, Users {
+
+ private static final Logger LOGGER = LoggerFactory.getLogger(Twitter.class);
+
+ private static Map<TwitterConfiguration, Twitter> INSTANCE_MAP = new ConcurrentHashMap<>();
+
+ private TwitterConfiguration configuration;
+
+ private ObjectMapper mapper;
+
+ private String rootUrl;
+
+ private CloseableHttpClient httpclient;
+
+ private HttpRequestInterceptor oauthInterceptor;
+
+ RestClient restClient;
+
+ private Twitter(TwitterConfiguration configuration) throws InstantiationException {
+ this.configuration = configuration;
+ this.rootUrl = TwitterProviderUtil.baseUrl(configuration);
+ this.oauthInterceptor = new TwitterOAuthRequestInterceptor(configuration.getOauth());
+ this.httpclient = HttpClientBuilder.create()
+ .addInterceptorFirst(oauthInterceptor)
+ .setDefaultRequestConfig(RequestConfig.custom()
+ .setConnectionRequestTimeout(5000)
+ .setConnectTimeout(5000)
+ .setSocketTimeout(5000)
+ .setCookieSpec("easy")
+ .build()
+ )
+ .setMaxConnPerRoute(20)
+ .setMaxConnTotal(100)
+ .build();
+
+// TODO: juneau-6.2.0-incubating
+// this.restClient = new RestClientBuilder()
+// .httpClient(httpclient, true)
+// .parser(JsonParser.class)
+// .rootUrl(rootUrl)
+// .retryable(
+// configuration.getRetryMax().intValue(),
+// configuration.getRetrySleepMs(),
+// new TwitterRetryHandler())
+// .build();
+ this.restClient = new RestClient()
+ .setHttpClient(httpclient)
+ .setParser(JsonParser.class)
+ .setRootUrl(rootUrl);
+
+ this.mapper = StreamsJacksonMapper.getInstance();
+ }
+
+ public static Twitter getInstance(TwitterConfiguration configuration) throws InstantiationException {
+ if (INSTANCE_MAP.containsKey(configuration) && INSTANCE_MAP.get(configuration) != null) {
+ return INSTANCE_MAP.get(configuration);
+ } else {
+ Twitter twitter = new Twitter(configuration);
+ INSTANCE_MAP.put(configuration, twitter);
+ return INSTANCE_MAP.get(configuration);
+ }
+ }
+
+ @Override
+ public List<Tweet> userTimeline(StatusesUserTimelineRequest parameters) {
+ try {
+// TODO: juneau-6.2.0-incubating
+// Statuses restStatuses = restClient.getRemoteableProxy("/statuses/user_timeline.json", Statuses.class);
+// List<Tweet> result = restStatuses.userTimeline(parameters);
+// return result;
+ URIBuilder uriBuilder = new URIBuilder()
+ .setPath("/statuses/user_timeline.json");
+ if( Objects.nonNull(parameters.getUserId()) && StringUtils.isNotBlank(parameters.getUserId().toString())) {
--- End diff --
I think the nonNull() checks are redundant because isNotBlank() "Checks if a String is not empty (""), not null and not whitespace only."
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by jfrazee <gi...@git.apache.org>.
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111297739
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Followers.java ---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.twitter.pojo.Tweet;
+
+import java.util.List;
+
+/**
+ * Returns a collection of the most recent Tweets posted by the user indicated by the screen_name or user_id parameters.
--- End diff --
I think this is the wrong javadoc for this interface. Should probably say something about the follower IDs and user objects, right?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by jfrazee <gi...@git.apache.org>.
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111307908
--- Diff: streams-contrib/streams-provider-twitter/src/main/jsonschema/org/apache/streams/twitter/api/FollowingIdsRequest.json ---
@@ -0,0 +1,36 @@
+{
+ "$schema": "http://json-schema.org/draft-03/schema",
+ "$license": [
+ "http://www.apache.org/licenses/LICENSE-2.0"
+ ],
+ "id": "#",
+ "javaType" : "org.apache.streams.twitter.api.FollowingIdsRequest",
+ "javaInterfaces": ["java.io.Serializable"],
+ "properties": {
+ "id": {
+ "description": "The ID of the user for whom to return results for.",
+ "required": false,
+ "type": "integer"
+ },
+ "screen_name": {
+ "description": "The screen name of the user for whom to return results for.",
+ "required": false,
+ "type": "string"
+ },
+ "curser": {
--- End diff --
I think this needs to be 'cursor'.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by jfrazee <gi...@git.apache.org>.
Github user jfrazee commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111299132
--- Diff: streams-contrib/streams-provider-twitter/src/main/java/org/apache/streams/twitter/api/Followers.java ---
@@ -0,0 +1,52 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.streams.twitter.api;
+
+import org.apache.streams.twitter.pojo.Tweet;
+
+import java.util.List;
+
+/**
+ * Returns a collection of the most recent Tweets posted by the user indicated by the screen_name or user_id parameters.
+ *
+ * @see <a href="https://dev.twitter.com/rest/reference/get/statuses/user_timeline">https://api.twitter.com/1.1/statuses/user_timeline.json</a>
+ */
+public interface Followers {
+
+ /**
+ * Returns a cursored collection of user IDs for every user following the specified user.
+ *
+ * @param parameters StatusesLookupRequest
+ * @return List < Tweet >
+ * @see <a href="https://api.twitter.com/1.1/followers/ids.json">https://api.twitter.com/1.1/followers/ids.json</a>
--- End diff --
Do you want the see URLs here to link to the online docs or the URL for the resource? E.g., I was thinking this one would have been https://dev.twitter.com/rest/reference/get/followers/ids instead of https://api.twitter.com/1.1/followers/ids.json
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-streams pull request #360: STREAMS-496: Remove twitter4j dependenc...
Posted by steveblackmon <gi...@git.apache.org>.
Github user steveblackmon commented on a diff in the pull request:
https://github.com/apache/incubator-streams/pull/360#discussion_r111310308
--- Diff: streams-contrib/streams-provider-twitter/src/main/jsonschema/org/apache/streams/twitter/api/FollowingIdsRequest.json ---
@@ -0,0 +1,36 @@
+{
+ "$schema": "http://json-schema.org/draft-03/schema",
+ "$license": [
+ "http://www.apache.org/licenses/LICENSE-2.0"
+ ],
+ "id": "#",
+ "javaType" : "org.apache.streams.twitter.api.FollowingIdsRequest",
+ "javaInterfaces": ["java.io.Serializable"],
+ "properties": {
+ "id": {
+ "description": "The ID of the user for whom to return results for.",
+ "required": false,
+ "type": "integer"
+ },
+ "screen_name": {
+ "description": "The screen name of the user for whom to return results for.",
+ "required": false,
+ "type": "string"
+ },
+ "curser": {
--- End diff --
nice catch. that was wrong a bunch of places. all fixed now and i watch the providers page as expected.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---