You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Nitiraj Rathore <ni...@gmail.com> on 2017/01/13 17:57:20 UTC
Review Request 55511: Ambari views : Adding new FileSystem support to
views should work without code changes
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55511/
-----------------------------------------------------------
Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, Renjith Kamath, and Ashwin Rajeev.
Bugs: AMBARI-19528
https://issues.apache.org/jira/browse/AMBARI-19528
Repository: ambari
Description
-------
Before this patch :
Right now when new file-system support has to be added to views, we need to add code to include several more key value pairs into the conf to make it work.
Usually these key value pairs are file-system class names, secret key and access key.
In this patch:
1. load the complete core-site and hdfs-site into the conf for creating filesystem. This will load any secret key, access key configured. It will also load any other necessary parameters.
2. Create a new view setting's parameter per view which will have semi colon separated key=value pairs that needs to be loaded into conf for connecting to the filesystem. This will cover any extra parameters.
3. changes in pom.xml -> included Amazon S3 related jars and excluded conflicting jars.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/view/ClusterImpl.java 10f216e
ambari-server/src/main/java/org/apache/ambari/server/view/RemoteAmbariCluster.java 6a4f04a
ambari-views/src/main/java/org/apache/ambari/view/cluster/Cluster.java 7f7c19b
contrib/views/commons/pom.xml 4b7ee3a
contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/FileOperationService.java bc3d11d
contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/HdfsService.java 91eebcf
contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/UploadService.java 97253ad
contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/UserService.java dc303ce
contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/ViewPropertyHelper.java PRE-CREATION
contrib/views/files/pom.xml ad9af4f
contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/DownloadService.java 96d3541
contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FileBrowserService.java adaa6c9
contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FilePreviewService.java 051e40d
contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/HelpService.java 92af2d5
contrib/views/files/src/main/resources/view.xml 640cee9
contrib/views/hive-next/pom.xml 2e7beab
contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/internal/HdfsApiSupplier.java 1cc1211
contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/resources/files/FileService.java 654bfcc
contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/utils/SharedObjectsFactory.java dd0b715
contrib/views/hive-next/src/main/resources/view.xml 1107cd1
contrib/views/hive20/pom.xml 6fc30f0
contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/internal/HdfsApiSupplier.java e66b9ab
contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/resources/browser/FileService.java 96e9554
contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/resources/files/FileService.java a3623e9
contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/utils/SharedObjectsFactory.java 5bc6070
contrib/views/hive20/src/main/resources/view.xml 315d0a9
contrib/views/jobs/src/main/resources/ui/.gitignore 503a8a9
contrib/views/pig/pom.xml d7d2de9
contrib/views/pig/src/main/java/org/apache/ambari/view/pig/resources/files/FileService.java 509b20e
contrib/views/pig/src/main/java/org/apache/ambari/view/pig/utils/UserLocalObjects.java 8293899
contrib/views/pig/src/main/resources/view.xml da8ffdd
contrib/views/pom.xml 13cfb23
contrib/views/utils/pom.xml c3cdab5
contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java d505e9a
contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsApi.java c7ae952
contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsUtil.java 8bc3ca5
contrib/views/wfmanager/pom.xml 95b988f
contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/FileServices.java ab286b2
contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/HDFSFileUtils.java 327d8fc
contrib/views/wfmanager/src/main/resources/view.xml c5d4a5b
Diff: https://reviews.apache.org/r/55511/diff/
Testing
-------
manual testing done on amazon s3 cluster and normal hdfs cluster.
Thanks,
Nitiraj Rathore
Re: Review Request 55511: Ambari views : Adding new FileSystem
support to views should work without code changes
Posted by Gaurav Nagar <gn...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55511/#review161558
-----------------------------------------------------------
Ship it!
Ship It!
- Gaurav Nagar
On Jan. 13, 2017, 5:57 p.m., Nitiraj Rathore wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55511/
> -----------------------------------------------------------
>
> (Updated Jan. 13, 2017, 5:57 p.m.)
>
>
> Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, Renjith Kamath, and Ashwin Rajeev.
>
>
> Bugs: AMBARI-19528
> https://issues.apache.org/jira/browse/AMBARI-19528
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Before this patch :
> Right now when new file-system support has to be added to views, we need to add code to include several more key value pairs into the conf to make it work.
> Usually these key value pairs are file-system class names, secret key and access key.
>
> In this patch:
> 1. load the complete core-site and hdfs-site into the conf for creating filesystem. This will load any secret key, access key configured. It will also load any other necessary parameters.
> 2. Create a new view setting's parameter per view which will have semi colon separated key=value pairs that needs to be loaded into conf for connecting to the filesystem. This will cover any extra parameters.
> 3. changes in pom.xml -> included Amazon S3 related jars and excluded conflicting jars.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/view/ClusterImpl.java 10f216e
> ambari-server/src/main/java/org/apache/ambari/server/view/RemoteAmbariCluster.java 6a4f04a
> ambari-views/src/main/java/org/apache/ambari/view/cluster/Cluster.java 7f7c19b
> contrib/views/commons/pom.xml 4b7ee3a
> contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/FileOperationService.java bc3d11d
> contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/HdfsService.java 91eebcf
> contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/UploadService.java 97253ad
> contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/UserService.java dc303ce
> contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/ViewPropertyHelper.java PRE-CREATION
> contrib/views/files/pom.xml ad9af4f
> contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/DownloadService.java 96d3541
> contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FileBrowserService.java adaa6c9
> contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FilePreviewService.java 051e40d
> contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/HelpService.java 92af2d5
> contrib/views/files/src/main/resources/view.xml 640cee9
> contrib/views/hive-next/pom.xml 2e7beab
> contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/internal/HdfsApiSupplier.java 1cc1211
> contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/resources/files/FileService.java 654bfcc
> contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/utils/SharedObjectsFactory.java dd0b715
> contrib/views/hive-next/src/main/resources/view.xml 1107cd1
> contrib/views/hive20/pom.xml 6fc30f0
> contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/internal/HdfsApiSupplier.java e66b9ab
> contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/resources/browser/FileService.java 96e9554
> contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/resources/files/FileService.java a3623e9
> contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/utils/SharedObjectsFactory.java 5bc6070
> contrib/views/hive20/src/main/resources/view.xml 315d0a9
> contrib/views/jobs/src/main/resources/ui/.gitignore 503a8a9
> contrib/views/pig/pom.xml d7d2de9
> contrib/views/pig/src/main/java/org/apache/ambari/view/pig/resources/files/FileService.java 509b20e
> contrib/views/pig/src/main/java/org/apache/ambari/view/pig/utils/UserLocalObjects.java 8293899
> contrib/views/pig/src/main/resources/view.xml da8ffdd
> contrib/views/pom.xml 13cfb23
> contrib/views/utils/pom.xml c3cdab5
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java d505e9a
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsApi.java c7ae952
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsUtil.java 8bc3ca5
> contrib/views/wfmanager/pom.xml 95b988f
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/FileServices.java ab286b2
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/HDFSFileUtils.java 327d8fc
> contrib/views/wfmanager/src/main/resources/view.xml c5d4a5b
>
> Diff: https://reviews.apache.org/r/55511/diff/
>
>
> Testing
> -------
>
> manual testing done on amazon s3 cluster and normal hdfs cluster.
>
>
> Thanks,
>
> Nitiraj Rathore
>
>
Re: Review Request 55511: Ambari views : Adding new FileSystem
support to views should work without code changes
Posted by DIPAYAN BHOWMICK <di...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/55511/#review161682
-----------------------------------------------------------
Ship it!
Ship It!
- DIPAYAN BHOWMICK
On Jan. 13, 2017, 5:57 p.m., Nitiraj Rathore wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/55511/
> -----------------------------------------------------------
>
> (Updated Jan. 13, 2017, 5:57 p.m.)
>
>
> Review request for Ambari, DIPAYAN BHOWMICK, Gaurav Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, Renjith Kamath, and Ashwin Rajeev.
>
>
> Bugs: AMBARI-19528
> https://issues.apache.org/jira/browse/AMBARI-19528
>
>
> Repository: ambari
>
>
> Description
> -------
>
> Before this patch :
> Right now when new file-system support has to be added to views, we need to add code to include several more key value pairs into the conf to make it work.
> Usually these key value pairs are file-system class names, secret key and access key.
>
> In this patch:
> 1. load the complete core-site and hdfs-site into the conf for creating filesystem. This will load any secret key, access key configured. It will also load any other necessary parameters.
> 2. Create a new view setting's parameter per view which will have semi colon separated key=value pairs that needs to be loaded into conf for connecting to the filesystem. This will cover any extra parameters.
> 3. changes in pom.xml -> included Amazon S3 related jars and excluded conflicting jars.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/view/ClusterImpl.java 10f216e
> ambari-server/src/main/java/org/apache/ambari/server/view/RemoteAmbariCluster.java 6a4f04a
> ambari-views/src/main/java/org/apache/ambari/view/cluster/Cluster.java 7f7c19b
> contrib/views/commons/pom.xml 4b7ee3a
> contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/FileOperationService.java bc3d11d
> contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/HdfsService.java 91eebcf
> contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/UploadService.java 97253ad
> contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/UserService.java dc303ce
> contrib/views/commons/src/main/java/org/apache/ambari/view/commons/hdfs/ViewPropertyHelper.java PRE-CREATION
> contrib/views/files/pom.xml ad9af4f
> contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/DownloadService.java 96d3541
> contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FileBrowserService.java adaa6c9
> contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/FilePreviewService.java 051e40d
> contrib/views/files/src/main/java/org/apache/ambari/view/filebrowser/HelpService.java 92af2d5
> contrib/views/files/src/main/resources/view.xml 640cee9
> contrib/views/hive-next/pom.xml 2e7beab
> contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/internal/HdfsApiSupplier.java 1cc1211
> contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/resources/files/FileService.java 654bfcc
> contrib/views/hive-next/src/main/java/org/apache/ambari/view/hive2/utils/SharedObjectsFactory.java dd0b715
> contrib/views/hive-next/src/main/resources/view.xml 1107cd1
> contrib/views/hive20/pom.xml 6fc30f0
> contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/internal/HdfsApiSupplier.java e66b9ab
> contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/resources/browser/FileService.java 96e9554
> contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/resources/files/FileService.java a3623e9
> contrib/views/hive20/src/main/java/org/apache/ambari/view/hive20/utils/SharedObjectsFactory.java 5bc6070
> contrib/views/hive20/src/main/resources/view.xml 315d0a9
> contrib/views/jobs/src/main/resources/ui/.gitignore 503a8a9
> contrib/views/pig/pom.xml d7d2de9
> contrib/views/pig/src/main/java/org/apache/ambari/view/pig/resources/files/FileService.java 509b20e
> contrib/views/pig/src/main/java/org/apache/ambari/view/pig/utils/UserLocalObjects.java 8293899
> contrib/views/pig/src/main/resources/view.xml da8ffdd
> contrib/views/pom.xml 13cfb23
> contrib/views/utils/pom.xml c3cdab5
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/ConfigurationBuilder.java d505e9a
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsApi.java c7ae952
> contrib/views/utils/src/main/java/org/apache/ambari/view/utils/hdfs/HdfsUtil.java 8bc3ca5
> contrib/views/wfmanager/pom.xml 95b988f
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/FileServices.java ab286b2
> contrib/views/wfmanager/src/main/java/org/apache/oozie/ambari/view/HDFSFileUtils.java 327d8fc
> contrib/views/wfmanager/src/main/resources/view.xml c5d4a5b
>
> Diff: https://reviews.apache.org/r/55511/diff/
>
>
> Testing
> -------
>
> manual testing done on amazon s3 cluster and normal hdfs cluster.
>
>
> Thanks,
>
> Nitiraj Rathore
>
>