[GitHub] storm pull request #1778: [STORM-2082][SQL] add sql external module storm-sq...

[vesense <gi...@git.apache.org>]

Github user vesense commented on a diff in the pull request:

    https://github.com/apache/storm/pull/1778#discussion_r93867628
  
    --- Diff: docs/storm-sql-reference.md ---
    @@ -1312,4 +1313,23 @@ You can use below as working reference for `--artifacts` option, and change depe
     
     `org.apache.storm:storm-sql-mongodb:2.0.0-SNAPSHOT,org.apache.storm:storm-mongodb:2.0.0-SNAPSHOT`
     
    -Storing record with preserving fields are not supported for now.
    \ No newline at end of file
    +Storing record with preserving fields are not supported for now.
    +
    +#### HDFS
    +
    +HDFS data source requires below properties to be set:
    +
    +* `hdfs.file.path`: HDFS file path
    +* `hdfs.file.name`: HDFS file name - please refer to [SimpleFileNameFormat]({{page.git-blob-base}}/external/storm-hdfs/src/main/java/org/apache/storm/hdfs/trident/format/SimpleFileNameFormat.java)
    +* `hdfs.rotation.size.kb`: HDFS FileSizeRotationPolicy in KB
    +* `hdfs.rotation.time.seconds`: HDFS TimedRotationPolicy in seconds
    +
    +Please note that `hdfs.rotation.size.kb` and `hdfs.rotation.time.seconds` only one can be used for hdfs rotation.
    +
    +And note that `storm-sql-hdfs` requires users to provide `storm-hdfs`.
    +You can use below as working reference for `--artifacts` option, and change dependencies version if really needed:
    +
    +`org.apache.storm:storm-sql-hdfs:2.0.0-SNAPSHOT,org.apache.storm:storm-hdfs:2.0.0-SNAPSHOT`
    +
    +Also, hdfs configuration files should be provided.
    +You can put the `core-site.xml` and `hdfs-site.xm` into the `conf` directory.
    --- End diff --
    
    Will fix typo.
    Yes the `conf` dir is the one in Storm installation directory. The `conf` dir is added to classpath in supervisor `BasicContainer`. see: https://github.com/apache/storm/blob/2832aeb369d431a2e2f0709175db68f079855387/storm-core/src/jvm/org/apache/storm/daemon/supervisor/BasicContainer.java#L350
    
    Maybe there is a better way to locate the hdfs configuration files. For now users have to put them in each supervisor node.



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---