You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ignite.apache.org by vk...@apache.org on 2015/04/01 02:57:11 UTC
incubator-ignite git commit: IGNITE-661 - Removed FTP scanner
Repository: incubator-ignite
Updated Branches:
refs/heads/ignite-sprint-2 [created] b0186fccd
IGNITE-661 - Removed FTP scanner
Project: http://git-wip-us.apache.org/repos/asf/incubator-ignite/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-ignite/commit/b0186fcc
Tree: http://git-wip-us.apache.org/repos/asf/incubator-ignite/tree/b0186fcc
Diff: http://git-wip-us.apache.org/repos/asf/incubator-ignite/diff/b0186fcc
Branch: refs/heads/ignite-sprint-2
Commit: b0186fccdd9d63f0535d5653513569a05beba187
Parents: 5fc2cd0
Author: Valentin Kulichenko <vk...@gridgain.com>
Authored: Tue Mar 31 17:59:12 2015 -0700
Committer: Valentin Kulichenko <vk...@gridgain.com>
Committed: Tue Mar 31 17:59:12 2015 -0700
----------------------------------------------------------------------
modules/urideploy/README.txt | 2 +-
modules/urideploy/licenses/edtftp-license.pdf | Bin 58355 -> 0 bytes
modules/urideploy/licenses/jtidy-license.txt | 50 ----
.../urideploy/licenses/jtidy-mit-license.txt | 50 ++++
modules/urideploy/pom.xml | 6 -
.../spi/deployment/uri/UriDeploymentSpi.java | 73 +-----
.../GridUriDeploymentScannerListener.java | 2 +-
.../ftp/GridUriDeploymentFtpClient.java | 199 ----------------
.../ftp/GridUriDeploymentFtpConfiguration.java | 115 ---------
.../ftp/GridUriDeploymentFtpException.java | 43 ----
.../scanners/ftp/GridUriDeploymentFtpFile.java | 119 ----------
.../ftp/GridUriDeploymentFtpScanner.java | 234 -------------------
.../uri/scanners/ftp/package-info.java | 22 --
...loymentMultiScannersErrorThrottlingTest.java | 3 -
.../GridUriDeploymentMultiScannersSelfTest.java | 3 +-
.../uri/scanners/ftp/GridFtpDeploymentTest.java | 46 ----
.../uri/scanners/ftp/package-info.java | 22 --
17 files changed, 55 insertions(+), 934 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/README.txt
----------------------------------------------------------------------
diff --git a/modules/urideploy/README.txt b/modules/urideploy/README.txt
index a2cdb62..dc3defd 100644
--- a/modules/urideploy/README.txt
+++ b/modules/urideploy/README.txt
@@ -2,7 +2,7 @@ Apache Ignite URI Deploy Module
-------------------------------
Apache Ignite URI Deploy module provides capabilities to deploy tasks from different sources like
-File System, HTTP, FTP, or even Email.
+File System, HTTP, or even Email.
To enable URI Deploy module when starting a standalone node, move 'optional/ignite-urideploy' folder to
'libs' folder before running 'ignite.{sh|bat}' script. The content of the module folder will
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/licenses/edtftp-license.pdf
----------------------------------------------------------------------
diff --git a/modules/urideploy/licenses/edtftp-license.pdf b/modules/urideploy/licenses/edtftp-license.pdf
deleted file mode 100644
index 606d7cd..0000000
Binary files a/modules/urideploy/licenses/edtftp-license.pdf and /dev/null differ
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/licenses/jtidy-license.txt
----------------------------------------------------------------------
diff --git a/modules/urideploy/licenses/jtidy-license.txt b/modules/urideploy/licenses/jtidy-license.txt
deleted file mode 100644
index 3f5714a..0000000
--- a/modules/urideploy/licenses/jtidy-license.txt
+++ /dev/null
@@ -1,50 +0,0 @@
-Java HTML Tidy - JTidy
-HTML parser and pretty printer
-
-Copyright (c) 1998-2000 World Wide Web Consortium (Massachusetts
-Institute of Technology, Institut National de Recherche en
-Informatique et en Automatique, Keio University). All Rights
-Reserved.
-
-Contributing Author(s):
-
- Dave Raggett <ds...@w3.org>
- Andy Quick <ac...@sympatico.ca> (translation to Java)
- Gary L Peskin <ga...@firstech.com> (Java development)
- Sami Lempinen <sa...@lempinen.net> (release management)
- Fabrizio Giustina <fgiust at users.sourceforge.net>
-
-The contributing author(s) would like to thank all those who
-helped with testing, bug fixes, and patience. This wouldn't
-have been possible without all of you.
-
-COPYRIGHT NOTICE:
-
-This software and documentation is provided "as is," and
-the copyright holders and contributing author(s) make no
-representations or warranties, express or implied, including
-but not limited to, warranties of merchantability or fitness
-for any particular purpose or that the use of the software or
-documentation will not infringe any third party patents,
-copyrights, trademarks or other rights.
-
-The copyright holders and contributing author(s) will not be
-liable for any direct, indirect, special or consequential damages
-arising out of any use of the software or documentation, even if
-advised of the possibility of such damage.
-
-Permission is hereby granted to use, copy, modify, and distribute
-this source code, or portions hereof, documentation and executables,
-for any purpose, without fee, subject to the following restrictions:
-
-1. The origin of this source code must not be misrepresented.
-2. Altered versions must be plainly marked as such and must
- not be misrepresented as being the original source.
-3. This Copyright notice may not be removed or altered from any
- source or altered source distribution.
-
-The copyright holders and contributing author(s) specifically
-permit, without fee, and encourage the use of this source code
-as a component for supporting the Hypertext Markup Language in
-commercial products. If you use this source code in a product,
-acknowledgment is not required but would be appreciated.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/licenses/jtidy-mit-license.txt
----------------------------------------------------------------------
diff --git a/modules/urideploy/licenses/jtidy-mit-license.txt b/modules/urideploy/licenses/jtidy-mit-license.txt
new file mode 100644
index 0000000..3f5714a
--- /dev/null
+++ b/modules/urideploy/licenses/jtidy-mit-license.txt
@@ -0,0 +1,50 @@
+Java HTML Tidy - JTidy
+HTML parser and pretty printer
+
+Copyright (c) 1998-2000 World Wide Web Consortium (Massachusetts
+Institute of Technology, Institut National de Recherche en
+Informatique et en Automatique, Keio University). All Rights
+Reserved.
+
+Contributing Author(s):
+
+ Dave Raggett <ds...@w3.org>
+ Andy Quick <ac...@sympatico.ca> (translation to Java)
+ Gary L Peskin <ga...@firstech.com> (Java development)
+ Sami Lempinen <sa...@lempinen.net> (release management)
+ Fabrizio Giustina <fgiust at users.sourceforge.net>
+
+The contributing author(s) would like to thank all those who
+helped with testing, bug fixes, and patience. This wouldn't
+have been possible without all of you.
+
+COPYRIGHT NOTICE:
+
+This software and documentation is provided "as is," and
+the copyright holders and contributing author(s) make no
+representations or warranties, express or implied, including
+but not limited to, warranties of merchantability or fitness
+for any particular purpose or that the use of the software or
+documentation will not infringe any third party patents,
+copyrights, trademarks or other rights.
+
+The copyright holders and contributing author(s) will not be
+liable for any direct, indirect, special or consequential damages
+arising out of any use of the software or documentation, even if
+advised of the possibility of such damage.
+
+Permission is hereby granted to use, copy, modify, and distribute
+this source code, or portions hereof, documentation and executables,
+for any purpose, without fee, subject to the following restrictions:
+
+1. The origin of this source code must not be misrepresented.
+2. Altered versions must be plainly marked as such and must
+ not be misrepresented as being the original source.
+3. This Copyright notice may not be removed or altered from any
+ source or altered source distribution.
+
+The copyright holders and contributing author(s) specifically
+permit, without fee, and encourage the use of this source code
+as a component for supporting the Hypertext Markup Language in
+commercial products. If you use this source code in a product,
+acknowledgment is not required but would be appreciated.
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/pom.xml
----------------------------------------------------------------------
diff --git a/modules/urideploy/pom.xml b/modules/urideploy/pom.xml
index 6e4c50d..71acdb6 100644
--- a/modules/urideploy/pom.xml
+++ b/modules/urideploy/pom.xml
@@ -79,12 +79,6 @@
</dependency>
<dependency>
- <groupId>com.enterprisedt</groupId>
- <artifactId>edtFTPj</artifactId>
- <version>1.5.3</version>
- </dependency>
-
- <dependency>
<groupId>net.sf.jtidy</groupId>
<artifactId>jtidy</artifactId>
<version>r938</version>
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/UriDeploymentSpi.java
----------------------------------------------------------------------
diff --git a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/UriDeploymentSpi.java b/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/UriDeploymentSpi.java
index da9b42e..a301a39 100644
--- a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/UriDeploymentSpi.java
+++ b/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/UriDeploymentSpi.java
@@ -28,7 +28,6 @@ import org.apache.ignite.spi.*;
import org.apache.ignite.spi.deployment.*;
import org.apache.ignite.spi.deployment.uri.scanners.*;
import org.apache.ignite.spi.deployment.uri.scanners.file.*;
-import org.apache.ignite.spi.deployment.uri.scanners.ftp.*;
import org.apache.ignite.spi.deployment.uri.scanners.http.*;
import org.jetbrains.annotations.*;
@@ -39,7 +38,7 @@ import java.util.Map.*;
/**
* Implementation of {@link org.apache.ignite.spi.deployment.DeploymentSpi} which can deploy tasks from
- * different sources like file system folders, FTP, email and HTTP.
+ * different sources like file system folders, email and HTTP.
* There are different ways to deploy tasks in grid and every deploy method
* depends on selected source protocol. This SPI is configured to work
* with a list of URI's. Every URI contains all data about protocol/transport
@@ -140,7 +139,6 @@ import java.util.Map.*;
* <ul>
* <li><a href="#file">file://</a> - File protocol</li>
* <li><a href="#classes">classes://</a> - Custom File protocol.</li>
- * <li><a href="#ftp">ftp://</a> - File transfer protocol</li>
* <li><a href="#http">http://</a> - HTTP protocol</li>
* <li><a href="#http">https://</a> - Secure HTTP protocol</li>
* </ul>
@@ -209,42 +207,7 @@ import java.util.Map.*;
* <blockquote class="snippet">
* {@code classes://freq=5000@localhost/c:/Program files/ignite/deployment}
* </blockquote>
- * <a name="ftp"></a>
- * <h1 class="header">FTP</h1>
- * For FTP protocol SPI will scan and download only GAR files from source
- * directory defined in URI. SPI doesn't scan FTP folders recursively.
- * The following parameters are supported for FTP protocol:
- * <table class="doctable">
- * <tr>
- * <th>Parameter</th>
- * <th>Description</th>
- * <th>Optional</th>
- * <th>Default</th>
- * </tr>
- * <tr>
- * <td>freq</td>
- * <td>FTP location scan frequency in milliseconds.</td>
- * <td>Yes</td>
- * <td>{@code 300000} ms specified in {@link #DFLT_FTP_SCAN_FREQUENCY DFLT_FTP_SCAN_FREQUENCY}.</td>
- * </tr>
- * <tr>
- * <td>username:password</td>
- * <td>
- * FTP username and password specified in standard URI server-based
- * authority format.
- * </td>
- * <td>No</td>
- * <td>---</td>
- * </tr>
- * </table>
- * <h2 class="header">FTP URI Example</h2>
- * Here is an example of an FTP URI that connects identified as
- * {@code username:password} to {@code 'localhost'} on port {@code '21'},
- * with initial path set to {@code 'ignite/deployment'}
- * <blockquote class="snippet">
- * ftp://username:password;freq=10000@localhost:21/ignite/deployment
- * </blockquote>
- * <p>
+ * <a name="http"></a>
* <h2 class="header">HTTP URI Example</h2>
* The following example will scan {@code 'ignite/deployment'} folder with
* on site {@code 'www.mysite.com'} using authentication
@@ -262,7 +225,6 @@ import java.util.Map.*;
* List<String> uris = new ArrayList<String>(5);
*
* uris.add("http://www.site.com/tasks");
- * uris.add("ftp://ftpuser:password;freq=10000@localhost:21/gg-test/deployment");
* uris.add("file://freq=20000@localhost/c:/Program files/gg-deployment");
* uris.add("classes:///c:/Java_Projects/myproject/out");
*
@@ -290,7 +252,6 @@ import java.util.Map.*;
* <property name="uriList">
* <list>
* <value>http://www.site.com/tasks</value>
- * <value>ftp://ftpuser:password;freq=10000@localhost:21/gg-test/deployment</value>
* <value>file://freq=20000@localhost/c:/Program files/gg-deployment</value>
* <value>classes:///c:/Java_Projects/myproject/out</value>
* </list>
@@ -322,9 +283,6 @@ public class UriDeploymentSpi extends IgniteSpiAdapter implements DeploymentSpi,
/** Default scan frequency for {@code file://} and {@code classes://} protocols (value is {@code 5000}). */
public static final int DFLT_DISK_SCAN_FREQUENCY = 5000;
- /** Default scan frequency for {@code ftp://} protocol (value is {@code 300000}). */
- public static final int DFLT_FTP_SCAN_FREQUENCY = 300000;
-
/** Default scan frequency for {@code http://} protocol (value is {@code 300000}). */
public static final int DFLT_HTTP_SCAN_FREQUENCY = 300000;
@@ -617,11 +575,6 @@ public class UriDeploymentSpi extends IgniteSpiAdapter implements DeploymentSpi,
DFLT_HTTP_SCAN_FREQUENCY, filter, lsnr, log);
break;
- case "ftp":
- scanner = new GridUriDeploymentFtpScanner(gridName, uri, file, freq > 0 ? freq :
- DFLT_FTP_SCAN_FREQUENCY, filter, lsnr, log);
- break;
-
default:
throw new IgniteSpiException("Unsupported protocol: " + proto);
}
@@ -946,28 +899,6 @@ public class UriDeploymentSpi extends IgniteSpiAdapter implements DeploymentSpi,
}
/**
- * Gets resource name for a given class name.
- *
- * @param clsName Class name.
- * @param rsrcs Map of resources.
- * @return Resource name.
- */
- private String getResourceName(String clsName, Map<String, String> rsrcs) {
- assert Thread.holdsLock(mux);
-
- String rsrcName = clsName;
-
- for (Entry<String, String> e : rsrcs.entrySet())
- if (e.getValue().equals(clsName) && !e.getKey().equals(clsName)) {
- rsrcName = e.getKey();
-
- break;
- }
-
- return rsrcName;
- }
-
- /**
* Check is class can be reached.
*
* @param ldr Class loader.
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/GridUriDeploymentScannerListener.java
----------------------------------------------------------------------
diff --git a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/GridUriDeploymentScannerListener.java b/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/GridUriDeploymentScannerListener.java
index ff33bd5..905e944 100644
--- a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/GridUriDeploymentScannerListener.java
+++ b/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/GridUriDeploymentScannerListener.java
@@ -22,7 +22,7 @@ import java.util.*;
/**
* Scanner listener interface. Whatever deployment scanner is used
- * (ftp, http, file and so on) following events happens:
+ * (http, file and so on) following events happens:
* <ul>
* <li>{@code onNewOrUpdatedFile} - happens when new file has been found or updated.</li>
* <li>{@code onDeletedFiles} - happens when file(s) has been removed.</li>
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpClient.java
----------------------------------------------------------------------
diff --git a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpClient.java b/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpClient.java
deleted file mode 100644
index e85082e..0000000
--- a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpClient.java
+++ /dev/null
@@ -1,199 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ignite.spi.deployment.uri.scanners.ftp;
-
-import com.enterprisedt.net.ftp.*;
-import org.apache.ignite.*;
-import org.apache.ignite.internal.util.typedef.internal.*;
-
-import java.io.*;
-import java.text.*;
-import java.util.*;
-
-/**
- * URI FTP deployment client.
- */
-class GridUriDeploymentFtpClient {
- /** Timeout in milliseconds on the underlying socket. */
- private static final int TIMEOUT = 60000;
-
- /** */
- private final GridUriDeploymentFtpConfiguration cfg;
-
- /** */
- private final IgniteLogger log;
-
- /** */
- private FTPClient ftp;
-
- /** */
- private boolean isConnected;
-
- /**
- * @param cfg FTP configuration.
- * @param log Logger to use.
- */
- GridUriDeploymentFtpClient(GridUriDeploymentFtpConfiguration cfg, IgniteLogger log) {
- assert cfg != null;
- assert log != null;
-
- this.cfg = cfg;
- this.log = log;
- }
-
-
- /**
- * @param rmtFile Remote file.
- * @param localFile Local file.
- * @throws GridUriDeploymentFtpException Thrown in case of any error.
- */
- void downloadToFile(GridUriDeploymentFtpFile rmtFile, File localFile) throws GridUriDeploymentFtpException {
- assert ftp != null;
- assert rmtFile != null;
- assert localFile != null;
-
- String dirName = rmtFile.getParentDirectory();
-
- if (dirName.length() == 0 || '/' != dirName.charAt(dirName.length() - 1))
- dirName += '/';
-
- String srcPath = dirName + rmtFile.getName();
-
- try {
- BufferedOutputStream out = null;
-
- try {
- out = new BufferedOutputStream(new FileOutputStream(localFile));
-
- ftp.get(out, srcPath);
- }
- finally {
- U.close(out, log);
- }
- }
- catch (IOException | FTPException e) {
- throw new GridUriDeploymentFtpException("Failed to download file [rmtFile=" + srcPath + ", localFile=" +
- localFile + ']', e);
- }
- }
-
- /**
- * @throws GridUriDeploymentFtpException Thrown in case of any error.
- */
- void connect() throws GridUriDeploymentFtpException {
- ftp = new FTPClient();
-
- try {
- ftp.setRemoteHost(cfg.getHost());
- ftp.setRemotePort(cfg.getPort());
-
- // Set socket timeout to avoid an infinite timeout.
- ftp.setTimeout(TIMEOUT);
-
- ftp.connect();
-
- ftp.login(cfg.getUsername(), cfg.getPassword());
-
- // Set up passive binary transfers.
- ftp.setConnectMode(FTPConnectMode.PASV);
- ftp.setType(FTPTransferType.BINARY);
-
- if (!ftp.connected()) {
- ftp.quit();
-
- throw new GridUriDeploymentFtpException("FTP server refused connection [host=" + cfg.getHost() +
- ", port=" + cfg.getPort() + ", username=" + cfg.getUsername() + ']');
- }
- }
- catch (IOException | FTPException e) {
- throw new GridUriDeploymentFtpException("Failed to connect to host [host=" + cfg.getHost() +
- ", port=" + cfg.getPort() + ']', e);
- }
-
- isConnected = true;
- }
-
- /**
- * @throws GridUriDeploymentFtpException Thrown in case of any error.
- */
- void close() throws GridUriDeploymentFtpException {
- if (!isConnected)
- return;
-
- assert ftp != null;
-
- Exception e = null;
-
- try {
- ftp.quit();
- }
- catch (IOException e1) {
- e = e1;
- }
- catch (FTPException e1) {
- e = e1;
- }
- finally{
- if (ftp.connected()) {
- try {
- ftp.quit();
- }
- catch (IOException | FTPException e1) {
- // Don't loose the initial exception.
- if (e == null)
- e = e1;
- }
- }
- }
-
- ftp = null;
-
- isConnected = false;
-
- if (e != null)
- throw new GridUriDeploymentFtpException("Failed to close FTP client.", e);
- }
-
- /**
- * @return List of files.
- * @throws GridUriDeploymentFtpException Thrown in case of any error.
- */
- List<GridUriDeploymentFtpFile> getFiles() throws GridUriDeploymentFtpException {
- try {
- assert cfg.getDirectory() != null;
-
- List<GridUriDeploymentFtpFile> clientFiles = new ArrayList<>();
-
- FTPFile[] files = ftp.dirDetails(cfg.getDirectory());
-
- for (FTPFile file : files) {
- clientFiles.add(new GridUriDeploymentFtpFile(cfg.getDirectory(), file));
- }
-
- return clientFiles;
- }
- catch (IOException | ParseException | FTPException e) {
- throw new GridUriDeploymentFtpException("Failed to get files in directory: " + cfg.getDirectory(), e);
- }
- }
-
- /** {@inheritDoc} */
- @Override public String toString() {
- return S.toString(GridUriDeploymentFtpClient.class, this);
- }
-}
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpConfiguration.java
----------------------------------------------------------------------
diff --git a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpConfiguration.java b/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpConfiguration.java
deleted file mode 100644
index 5d5b62a..0000000
--- a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpConfiguration.java
+++ /dev/null
@@ -1,115 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ignite.spi.deployment.uri.scanners.ftp;
-
-import org.apache.ignite.internal.util.tostring.*;
-import org.apache.ignite.internal.util.typedef.internal.*;
-
-/**
- * URI FTP deployment configuration.
- */
-class GridUriDeploymentFtpConfiguration {
- /** */
- private static final String DFLT_DIR = "/";
-
- /** */
- private String host;
-
- /** */
- private int port = 21;
-
- /** */
- private String username;
-
- /** */
- @GridToStringExclude private String pswd;
-
- /** */
- private String dir = DFLT_DIR;
-
- /**
- * @return Host.
- */
- String getHost() { return host; }
-
- /**
- * @param host FTP host.
- */
- void setHost(String host) {
- assert host != null;
-
- this.host = host;
- }
-
- /**
- * @return Port.
- */
- int getPort() { return port; }
-
- /**
- * @param port FTP port.
- */
- void setPort(int port) {
- assert port > 0;
-
- this.port = port;
- }
-
- /**
- * @return Username.
- */
- String getUsername() { return username; }
-
- /**
- * @param username FTP username.
- */
- void setUsername(String username) {
- assert username != null;
-
- this.username = username;
- }
-
- /**
- * @return Password.
- */
- String getPassword() { return pswd; }
-
- /**
- * @param pswd FTP password.
- */
- void setPassword(String pswd) {
- assert pswd != null;
-
- this.pswd = pswd;
- }
-
- /**
- * @return Directory.
- */
- String getDirectory() { return dir; }
-
- /**
- * @param dir FTP remote directory.
- */
- void setDirectory(String dir) { this.dir = dir == null ? DFLT_DIR : dir; }
-
- /** {@inheritDoc} */
- @Override public String toString() {
- return S.toString(GridUriDeploymentFtpConfiguration.class, this);
- }
-}
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpException.java
----------------------------------------------------------------------
diff --git a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpException.java b/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpException.java
deleted file mode 100644
index cc046a8..0000000
--- a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpException.java
+++ /dev/null
@@ -1,43 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ignite.spi.deployment.uri.scanners.ftp;
-
-import org.apache.ignite.*;
-
-/**
- * An exception occurred during URI FTP deployment.
- */
-class GridUriDeploymentFtpException extends IgniteCheckedException {
- /** */
- private static final long serialVersionUID = 0L;
-
- /**
- * Creates new grid exception with given error message.
- *
- * @param msg Error message.
- */
- GridUriDeploymentFtpException(String msg) { super(msg); }
-
- /**
- * Creates new grid ftp client exception with given error message and optional nested exception.
- *
- * @param msg Error message.
- * @param cause Optional nested exception (can be {@code null}).
- */
- GridUriDeploymentFtpException(String msg, Throwable cause) { super(msg, cause); }
-}
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpFile.java
----------------------------------------------------------------------
diff --git a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpFile.java b/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpFile.java
deleted file mode 100644
index 90e941f..0000000
--- a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpFile.java
+++ /dev/null
@@ -1,119 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ignite.spi.deployment.uri.scanners.ftp;
-
-import com.enterprisedt.net.ftp.*;
-import org.apache.ignite.internal.util.typedef.internal.*;
-
-import java.util.*;
-
-/**
- * Value object which encapsulates {@link FTPFile} and corresponding directory.
- */
-class GridUriDeploymentFtpFile {
- /** */
- private final String dir;
-
- /** */
- private final FTPFile file;
-
- /**
- * @param dir Remote FTP directory.
- * @param file FTP file.
- */
- GridUriDeploymentFtpFile(String dir, FTPFile file) {
- assert dir != null;
- assert file != null;
- assert file.getName() != null;
-
- this.dir = dir;
- this.file = file;
- }
-
- /** {@inheritDoc} */
- @Override public boolean equals(Object obj) {
- if (this == obj)
- return true;
-
- if (!(obj instanceof GridUriDeploymentFtpFile))
- return false;
-
- GridUriDeploymentFtpFile other = (GridUriDeploymentFtpFile)obj;
-
- return dir.equals(other.dir) && file.getName().equals(other.file.getName());
- }
-
- /** {@inheritDoc} */
- @Override public int hashCode() {
- int res = dir.hashCode();
-
- res = 29 * res + file.getName().hashCode();
-
- return res;
- }
-
- /**
- * @return File name.
- */
- String getName() {
- return file.getName();
- }
-
- /**
- * @return Last modification date as calendar.
- */
- Calendar getTimestamp() {
- Date date = file.lastModified();
-
- Calendar cal = null;
-
- if (date != null) {
- cal = Calendar.getInstance();
-
- cal.setTime(date);
- }
-
- return cal;
- }
-
- /**
- * @return {@code True} in case provided file is a directory.
- */
- boolean isDirectory() {
- return file.isDir();
- }
-
- /**
- * @return {@code True} in case provided file is not a directory or link.
- */
- boolean isFile() {
- return !file.isDir() && !file.isLink();
- }
-
- /**
- * @return File parent directory.
- */
- String getParentDirectory() {
- return dir;
- }
-
- /** {@inheritDoc} */
- @Override public String toString() {
- return S.toString(GridUriDeploymentFtpFile.class, this);
- }
-}
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpScanner.java
----------------------------------------------------------------------
diff --git a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpScanner.java b/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpScanner.java
deleted file mode 100644
index dc9044e..0000000
--- a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridUriDeploymentFtpScanner.java
+++ /dev/null
@@ -1,234 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ignite.spi.deployment.uri.scanners.ftp;
-
-import org.apache.ignite.*;
-import org.apache.ignite.internal.util.typedef.internal.*;
-import org.apache.ignite.spi.deployment.uri.scanners.*;
-
-import java.io.*;
-import java.net.*;
-import java.util.*;
-
-/**
- * FTP scanner scans directory for new files. Scanned directory defined in URI.
- * Scanner doesn't search files in subfolders.
- */
-public class GridUriDeploymentFtpScanner extends GridUriDeploymentScanner {
- /** */
- private static final long UNKNOWN_FILE_TSTAMP = -1;
-
- /** */
- private final GridUriDeploymentFtpConfiguration cfg;
-
- /** Cache of found files to check if any of it has been updated. */
- private Map<GridUriDeploymentFtpFile, Long> cache = new HashMap<>();
-
- /**
- * @param gridName Grid instance name.
- * @param uri FTP URI.
- * @param deployDir FTP directory.
- * @param freq Scanner frequency.
- * @param filter Scanner filter.
- * @param lsnr Deployment listener.
- * @param log Logger to use.
- */
- public GridUriDeploymentFtpScanner(
- String gridName,
- URI uri,
- File deployDir,
- long freq,
- FilenameFilter filter,
- GridUriDeploymentScannerListener lsnr,
- IgniteLogger log) {
- super(gridName, uri, deployDir, freq, filter, lsnr, log);
-
- cfg = initializeFtpConfiguration(uri);
- }
-
- /**
- * @param uri FTP URI.
- * @return FTP configuration.
- */
- private GridUriDeploymentFtpConfiguration initializeFtpConfiguration(URI uri) {
- assert "ftp".equals(uri.getScheme());
-
- GridUriDeploymentFtpConfiguration cfg = new GridUriDeploymentFtpConfiguration();
-
- String userInfo = uri.getUserInfo();
- String username = null;
- String pswd = null;
-
- if (userInfo != null) {
- String[] arr = userInfo.split(";");
-
- if (arr != null && arr.length > 0)
- for (String el : arr)
- if (el.startsWith("freq=")) {
- // No-op.
- }
- else if (el.indexOf(':') != -1) {
- int idx = el.indexOf(':');
-
- username = el.substring(0, idx);
- pswd = el.substring(idx + 1);
- }
- else
- username = el;
- }
-
- // Username and password must be defined in URI.
- if (username == null)
- throw new IgniteException("Username has not been provided.");
-
- if (pswd == null)
- throw new IgniteException("Password has not been provided.");
-
- cfg.setHost(uri.getHost());
- cfg.setPort(uri.getPort());
- cfg.setUsername(username);
- cfg.setPassword(pswd);
- cfg.setDirectory(uri.getPath());
-
- return cfg;
- }
-
- /** {@inheritDoc} */
- @Override protected void process() {
- Collection<GridUriDeploymentFtpFile> foundFiles = U.newHashSet(cache.size());
-
- long start = U.currentTimeMillis();
-
- processFtp(foundFiles);
-
- if (getLogger().isDebugEnabled())
- getLogger().debug("FTP scanner time in milliseconds: " + (U.currentTimeMillis() - start));
-
- if (!isFirstScan()) {
- Collection<GridUriDeploymentFtpFile> delFiles = new HashSet<>(cache.keySet());
-
- delFiles.removeAll(foundFiles);
-
- if (!delFiles.isEmpty()) {
- List<String> uris = new ArrayList<>();
-
- for (GridUriDeploymentFtpFile file : delFiles) {
- Long tstamp = cache.get(file);
-
- // Ignore files in cache w/o timestamp.
- if (tstamp != null && tstamp != UNKNOWN_FILE_TSTAMP)
- uris.add(getFileUri(file.getName()));
- }
-
- cache.keySet().removeAll(delFiles);
-
- getListener().onDeletedFiles(uris);
- }
- }
- }
-
- /**
- * @param files File to process.
- */
- @SuppressWarnings({"UnusedCatchParameter"})
- private void processFtp(Collection<GridUriDeploymentFtpFile> files) {
- GridUriDeploymentFtpClient ftp = new GridUriDeploymentFtpClient(cfg, getLogger());
-
- try {
- ftp.connect();
-
- for (GridUriDeploymentFtpFile file : ftp.getFiles()) {
- String fileName = file.getName();
-
- if (getFilter().accept(null, fileName.toLowerCase()) && file.isFile()) {
- files.add(file);
-
- Long lastModified = cache.get(file);
-
- Calendar fileTstamp = file.getTimestamp();
-
- if (fileTstamp == null) {
- if (lastModified == null) {
- // Add new file in cache to avoid print warning every time.
- cache.put(file, UNKNOWN_FILE_TSTAMP);
-
- U.warn(getLogger(), "File with unknown timestamp will be ignored " +
- "(check FTP server configuration): " + file);
- }
- }
- // If file is new or has been modified.
- else if (lastModified == null || lastModified != fileTstamp.getTimeInMillis()) {
- cache.put(file, fileTstamp.getTimeInMillis());
-
- if (getLogger().isDebugEnabled())
- getLogger().debug("Discovered deployment file or directory: " + file);
-
- try {
- File diskFile = createTempFile(fileName, getDeployDirectory());
-
- ftp.downloadToFile(file, diskFile);
-
- String fileUri = getFileUri(fileName);
-
- // Delete file when JVM stopped.
- diskFile.deleteOnExit();
-
- // Deployment SPI apply.
- // NOTE: If SPI listener blocks then FTP connection may be closed by timeout.
- getListener().onNewOrUpdatedFile(diskFile, fileUri, fileTstamp.getTimeInMillis());
- }
- catch (IOException e) {
- U.error(getLogger(), "Failed to download file from FTP server: " + fileName, e);
- }
- }
- }
- }
- }
- catch (GridUriDeploymentFtpException e) {
- if (!isCancelled()) {
- String maskedUri = getUri() != null ? U.hidePassword(getUri().toString()) : null;
-
- if (e.hasCause(ConnectException.class))
- LT.warn(getLogger(), e, "Failed to connect to FTP server (connection refused): " + maskedUri);
-
- else if (e.hasCause(UnknownHostException.class))
- LT.warn(getLogger(), e, "Failed to connect to FTP server (host is unknown): " + maskedUri);
-
- else
- U.error(getLogger(), "Failed to get files from FTP server: " + maskedUri, e);
- }
- }
- finally {
- try {
- ftp.close();
- }
- catch (GridUriDeploymentFtpException e) {
- if (!isCancelled())
- U.error(getLogger(), "Failed to close FTP client.", e);
- }
- }
- }
-
- /** {@inheritDoc} */
- @Override public String toString() {
- return S.toString(GridUriDeploymentFtpScanner.class, this,
- "uri", getUri() != null ? U.hidePassword(getUri().toString()) : null,
- "freq", getFrequency(),
- "deployDir", getDeployDirectory());
- }
-}
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/package-info.java
----------------------------------------------------------------------
diff --git a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/package-info.java b/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/package-info.java
deleted file mode 100644
index ca29005..0000000
--- a/modules/urideploy/src/main/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/package-info.java
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/**
- * <!-- Package description. -->
- * Contains FTP scanner for URI deployer SPI.
- */
-package org.apache.ignite.spi.deployment.uri.scanners.ftp;
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/GridUriDeploymentMultiScannersErrorThrottlingTest.java
----------------------------------------------------------------------
diff --git a/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/GridUriDeploymentMultiScannersErrorThrottlingTest.java b/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/GridUriDeploymentMultiScannersErrorThrottlingTest.java
index 21e5eb5..32fdadc 100644
--- a/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/GridUriDeploymentMultiScannersErrorThrottlingTest.java
+++ b/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/GridUriDeploymentMultiScannersErrorThrottlingTest.java
@@ -44,9 +44,6 @@ public class GridUriDeploymentMultiScannersErrorThrottlingTest extends GridUriDe
public List<String> getUriList() {
List<String> uriList = new ArrayList<>();
- uriList.add("ftp://anonymous:111111;freq=5000@unknown.host:21/pub/gg-test");
- uriList.add("ftp://anonymous:111111;freq=5000@localhost:21/pub/gg-test");
-
uriList.add("http://freq=5000@localhost/tasks");
uriList.add("http://freq=5000@unknownhost.host/tasks");
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/GridUriDeploymentMultiScannersSelfTest.java
----------------------------------------------------------------------
diff --git a/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/GridUriDeploymentMultiScannersSelfTest.java b/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/GridUriDeploymentMultiScannersSelfTest.java
index af54e06..1beee49 100644
--- a/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/GridUriDeploymentMultiScannersSelfTest.java
+++ b/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/GridUriDeploymentMultiScannersSelfTest.java
@@ -42,8 +42,7 @@ public class GridUriDeploymentMultiScannersSelfTest extends GridUriDeploymentAbs
public List<String> getUriList() {
List<String> uriList = new ArrayList<>();
- // Fake URIs.
- uriList.add(GridTestProperties.getProperty("deploy.uri.ftp"));
+ // Fake URI.
uriList.add(GridTestProperties.getProperty("deploy.uri.http"));
// One real URI.
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridFtpDeploymentTest.java
----------------------------------------------------------------------
diff --git a/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridFtpDeploymentTest.java b/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridFtpDeploymentTest.java
deleted file mode 100644
index 5b73889..0000000
--- a/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/GridFtpDeploymentTest.java
+++ /dev/null
@@ -1,46 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.ignite.spi.deployment.uri.scanners.ftp;
-
-import org.apache.ignite.spi.deployment.uri.*;
-import org.apache.ignite.testframework.config.*;
-import org.apache.ignite.testframework.junits.spi.*;
-
-import java.util.*;
-
-/**
- * Test for FTP deployment.
- */
-@GridSpiTest(spi = UriDeploymentSpi.class, group = "Deployment SPI")
-public class GridFtpDeploymentTest extends GridUriDeploymentAbstractSelfTest {
- /**
- * @throws Exception If failed.
- */
- public void testDeployment() throws Exception {
- checkTask("org.apache.ignite.spi.deployment.uri.tasks.GridUriDeploymentTestTask3");
- checkTask("GridUriDeploymentTestWithNameTask3");
- }
-
- /**
- * @return List of deployment sources.
- */
- @GridSpiTestConfig
- public List<String> getUriList() {
- return Collections.singletonList(GridTestProperties.getProperty("deploy.uri.ftp"));
- }
-}
http://git-wip-us.apache.org/repos/asf/incubator-ignite/blob/b0186fcc/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/package-info.java
----------------------------------------------------------------------
diff --git a/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/package-info.java b/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/package-info.java
deleted file mode 100644
index 50fd545..0000000
--- a/modules/urideploy/src/test/java/org/apache/ignite/spi/deployment/uri/scanners/ftp/package-info.java
+++ /dev/null
@@ -1,22 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-/**
- * <!-- Package description. -->
- * Contains internal tests or test related classes and interfaces.
- */
-package org.apache.ignite.spi.deployment.uri.scanners.ftp;
\ No newline at end of file