You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by medale <gi...@git.apache.org> on 2017/01/03 03:32:39 UTC
[GitHub] flink pull request #3054: [Flink 5404] Consolidate and update S3 documentati...
GitHub user medale opened a pull request:
https://github.com/apache/flink/pull/3054
[Flink 5404] Consolidate and update S3 documentation
Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration.
If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the [How To Contribute guide](http://flink.apache.org/how-to-contribute.html).
In addition to going through the list, please provide a meaningful description of your changes.
- [ ] General
- The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
- The pull request addresses only one issue
- Each commit in the PR has a meaningful commit message (including the JIRA id)
- [ ] Documentation
- Documentation has been added for new functionality
- Old documentation affected by the pull request has been updated
- JavaDoc for public methods has been added
- [ ] Tests & Build
- Functionality added by the pull request is covered by tests
- `mvn clean verify` has been executed successfully locally or a Travis build has passed
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/medale/flink FLINK-5404
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3054.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3054
----
commit 24ca6a571cb2f709ce049f1901943fc3a6a0f811
Author: medale <me...@yahoo.com>
Date: 2017-01-03T03:13:56Z
[FLINK-5404] consolidate updated S3 information in aws.md
commit 75204096dbce7f8a62064da944304c1b6ab972a2
Author: medale <me...@yahoo.com>
Date: 2017-01-03T03:20:09Z
[FLINK-5404] correctly set site base url.
commit 7da5baa7dc95b915b736b03031387f6a47febeaa
Author: medale <me...@yahoo.com>
Date: 2017-01-03T03:27:23Z
[FLINK-5404] clarify link.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3054: [Flink 5404] Consolidate and update S3 documentation
Posted by uce <gi...@git.apache.org>.
Github user uce commented on the issue:
https://github.com/apache/flink/pull/3054
Thanks for the PR. Looks good to me. Merging this...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3054: [Flink 5404] Consolidate and update S3 documentation
Posted by tzulitai <gi...@git.apache.org>.
Github user tzulitai commented on the issue:
https://github.com/apache/flink/pull/3054
@medale The test failure is unrelated, it's ok to ignore that.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3054: [Flink 5404] Consolidate and update S3 documentati...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/flink/pull/3054
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3054: [Flink 5404] Consolidate and update S3 documentati...
Posted by tzulitai <gi...@git.apache.org>.
Github user tzulitai commented on a diff in the pull request:
https://github.com/apache/flink/pull/3054#discussion_r94746683
--- Diff: docs/setup/aws.md ---
@@ -130,13 +134,13 @@ This registers `NativeS3FileSystem` as the default FileSystem for URIs with the
#### Hadoop Configuration
-You can specify the [Hadoop configuration]({{ site.baseurl }}/setup/config.html#hdfs) in various ways, for examples by configuring the path to the Hadoop configuration directory in `flink-conf.yaml`:
+You can specify the [Hadoop configuration]({{ site.baseurl }}/setup/config.html#hdfs) in various ways, for example by configuring the path to the Hadoop configuration directory in `flink-conf.yaml`:
```
fs.hdfs.hadoopconf: /path/to/etc/hadoop
```
-This registers `path/to/etc/hadoop` as Hadoop's configuration directory with Flink.
+This registers `/path/to/etc/hadoop` as Hadoop's configuration directory with Flink. Flink will look for the "core-site.xml" and "hdfs-site.xml" files in the specified directory.
--- End diff --
Reference to core-site.xml and hdfs-site.xml here uses double apostrophes, while \` is used in other places.
Lets keep that styling consistent here :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3054: [Flink 5404] Consolidate and update S3 documentation
Posted by medale <gi...@git.apache.org>.
Github user medale commented on the issue:
https://github.com/apache/flink/pull/3054
@tzulitai thank you for the feedback and catching those stylistic elements! I made the changes and recommitted.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink pull request #3054: [Flink 5404] Consolidate and update S3 documentati...
Posted by tzulitai <gi...@git.apache.org>.
Github user tzulitai commented on a diff in the pull request:
https://github.com/apache/flink/pull/3054#discussion_r94746306
--- Diff: docs/dev/batch/connectors.md ---
@@ -52,33 +52,13 @@ interface. There are Hadoop `FileSystem` implementations for
In order to use a Hadoop file system with Flink, make sure that
-- the `flink-conf.yaml` has set the `fs.hdfs.hadoopconf` property set to the Hadoop configuration directory.
-- the Hadoop configuration (in that directory) has an entry for the required file system. Examples for S3 and Alluxio are shown below.
-- the required classes for using the file system are available in the `lib/` folder of the Flink installation (on all machines running Flink). If putting the files into the directory is not possible, Flink is also respecting the `HADOOP_CLASSPATH` environment variable to add Hadoop jar files to the classpath.
+- the `flink-conf.yaml` has set the `fs.hdfs.hadoopconf` property to the Hadoop configuration directory. For automated testing or running from an IDE the directory containing `flink-conf.yaml` can be set by defining the FLINK_CONF_DIR environment variable.
--- End diff --
Add \` around FLINK_CONF_DIR ==> `FLINK_CONF_DIR`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #3054: [Flink 5404] Consolidate and update S3 documentation
Posted by medale <gi...@git.apache.org>.
Github user medale commented on the issue:
https://github.com/apache/flink/pull/3054
Error about RocksDB caused by: java.lang.UnsatisfiedLinkError: Native Library /tmp/librocksdbjni-linux64.so already loaded in another classloader at java.lang.ClassLoader.loadLibrary1(ClassLoader.java:1931). Since I only changed documentation this seems like an unrelated error with the overall build. How do I continue?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---