You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@activemq.apache.org by "Antun Pendo (JIRA)" <ji...@apache.org> on 2009/12/22 19:49:40 UTC
[jira] Created: (AMQ-2548) Downloading Blob messages via FTP fails
for files larger than 64KB
Downloading Blob messages via FTP fails for files larger than 64KB
------------------------------------------------------------------
Key: AMQ-2548
URL: https://issues.apache.org/activemq/browse/AMQ-2548
Project: ActiveMQ
Issue Type: Bug
Affects Versions: 5.3.0
Environment: OS: Windows XP
Reporter: Antun Pendo
Priority: Minor
The following code will only download 64 KB of any uploaded file greater than 64 KB.
The test file (ca. 15 MB) was completly uploaded to the FTP-Server.
{code}
File file = new File(directoryName+fileName);
ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(URI);
Connection connection = factory.createQueueConnection();
connection.start();
ActiveMQSession session = (ActiveMQSession) connection.createSession(
false, Session.AUTO_ACKNOWLEDGE);
Destination destination = session.createQueue("MyQ");
MessageProducer producer = session.createProducer(destination);
MessageConsumer consumer = session.createConsumer(destination);
BlobMessage message = session.createBlobMessage(file);
producer.send(message);
System.out.println("Sent: " + message);
Thread.sleep(1000);
// check message sent
Message msg = consumer.receive();
BlobDownloadStrategy strategy = new FTPBlobDownloadStrategy();
InputStream input = strategy.getInputStream((ActiveMQBlobMessage)msg);
File f=new File(fileName);
OutputStream out=new FileOutputStream(f);
byte buf[]=new byte[1024];
int len;
while((len=input.read(buf))>0){
out.write(buf,0,len);
}
out.close();
input.close();
System.out.println("Received: " + message);
{code}
After examining org.apache.activemq.blob.DefaultBlobUploadStrategy it seemed suspicious that the FTPClient connection was destroyed before the input stream is processed.
{code}
public InputStream getInputStream(ActiveMQBlobMessage message) throws IOException, JMSException {
URL url = message.getURL();
setUserInformation(url.getUserInfo());
String connectUrl = url.getHost();
int port = url.getPort() < 1 ? 21 : url.getPort();
FTPClient ftp = new FTPClient();
try {
ftp.connect(connectUrl, port);
} catch(ConnectException e) {
throw new JMSException("Problem connecting the FTP-server");
}
if(!ftp.login(ftpUser, ftpPass)) {
ftp.quit();
ftp.disconnect();
throw new JMSException("Cant Authentificate to FTP-Server");
}
String path = url.getPath();
String workingDir = path.substring(0, path.lastIndexOf("/"));
String file = path.substring(path.lastIndexOf("/")+1);
ftp.changeWorkingDirectory(workingDir);
ftp.setFileType(FTPClient.BINARY_FILE_TYPE);
InputStream input = ftp.retrieveFileStream(file);
ftp.quit(); // really?
ftp.disconnect(); // really?
return input;
}
{code}
After commenting those two last ftp calls, files larger than 64 KB were downloaded properly, but this should of course not be the final solution.
Any suggestions?
Cheers, Toni
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-2548) Downloading Blob messages via FTP fails
for files larger than 64KB
Posted by "Gary Tully (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gary Tully updated AMQ-2548:
----------------------------
Fix Version/s: 5.3.1
merged to 5.3.1 branch in r905219
> Downloading Blob messages via FTP fails for files larger than 64KB
> ------------------------------------------------------------------
>
> Key: AMQ-2548
> URL: https://issues.apache.org/activemq/browse/AMQ-2548
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.3.0
> Environment: OS: Windows XP
> Reporter: Antun Pendo
> Assignee: Timothy Bish
> Priority: Minor
> Fix For: 5.3.1, 5.4.0
>
>
> The following code will only download 64 KB of any uploaded file greater than 64 KB.
> The test file (ca. 15 MB) was completly uploaded to the FTP-Server.
>
> {code}
> File file = new File(directoryName+fileName);
>
> ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(URI);
> Connection connection = factory.createQueueConnection();
> connection.start();
>
> ActiveMQSession session = (ActiveMQSession) connection.createSession(
> false, Session.AUTO_ACKNOWLEDGE);
> Destination destination = session.createQueue("MyQ");
> MessageProducer producer = session.createProducer(destination);
> MessageConsumer consumer = session.createConsumer(destination);
> BlobMessage message = session.createBlobMessage(file);
>
> producer.send(message);
> System.out.println("Sent: " + message);
>
> Thread.sleep(1000);
>
> // check message sent
> Message msg = consumer.receive();
>
> BlobDownloadStrategy strategy = new FTPBlobDownloadStrategy();
>
> InputStream input = strategy.getInputStream((ActiveMQBlobMessage)msg);
> File f=new File(fileName);
> OutputStream out=new FileOutputStream(f);
> byte buf[]=new byte[1024];
> int len;
>
> while((len=input.read(buf))>0){
> out.write(buf,0,len);
> }
> out.close();
> input.close();
> System.out.println("Received: " + message);
>
> {code}
> After examining org.apache.activemq.blob.FTPBlobDownloadStrategy it seemed suspicious that the FTPClient connection was destroyed before the input stream is processed.
> {code}
> public InputStream getInputStream(ActiveMQBlobMessage message) throws IOException, JMSException {
> URL url = message.getURL();
>
> setUserInformation(url.getUserInfo());
> String connectUrl = url.getHost();
> int port = url.getPort() < 1 ? 21 : url.getPort();
> FTPClient ftp = new FTPClient();
> try {
> ftp.connect(connectUrl, port);
> } catch(ConnectException e) {
> throw new JMSException("Problem connecting the FTP-server");
> }
>
> if(!ftp.login(ftpUser, ftpPass)) {
> ftp.quit();
> ftp.disconnect();
> throw new JMSException("Cant Authentificate to FTP-Server");
> }
> String path = url.getPath();
> String workingDir = path.substring(0, path.lastIndexOf("/"));
> String file = path.substring(path.lastIndexOf("/")+1);
>
> ftp.changeWorkingDirectory(workingDir);
> ftp.setFileType(FTPClient.BINARY_FILE_TYPE);
> InputStream input = ftp.retrieveFileStream(file);
> ftp.quit(); // really?
> ftp.disconnect(); // really?
>
> return input;
> }
> {code}
> After commenting those two last ftp calls, files larger than 64 KB were downloaded properly, but this should of course not be the final solution.
> Any suggestions?
> Cheers, Toni
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (AMQ-2548) Downloading Blob messages via FTP fails
for files larger than 64KB
Posted by "Timothy Bish (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Timothy Bish resolved AMQ-2548.
-------------------------------
Resolution: Fixed
Fix Version/s: 5.4.0
Assignee: Timothy Bish
Updated the test case to reproduce the problem and fixed the getInputStream method so that the FTP connection is not disconnected until the stream is closed.
Fixed in SVN rev 901911
> Downloading Blob messages via FTP fails for files larger than 64KB
> ------------------------------------------------------------------
>
> Key: AMQ-2548
> URL: https://issues.apache.org/activemq/browse/AMQ-2548
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.3.0
> Environment: OS: Windows XP
> Reporter: Antun Pendo
> Assignee: Timothy Bish
> Priority: Minor
> Fix For: 5.4.0
>
>
> The following code will only download 64 KB of any uploaded file greater than 64 KB.
> The test file (ca. 15 MB) was completly uploaded to the FTP-Server.
>
> {code}
> File file = new File(directoryName+fileName);
>
> ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(URI);
> Connection connection = factory.createQueueConnection();
> connection.start();
>
> ActiveMQSession session = (ActiveMQSession) connection.createSession(
> false, Session.AUTO_ACKNOWLEDGE);
> Destination destination = session.createQueue("MyQ");
> MessageProducer producer = session.createProducer(destination);
> MessageConsumer consumer = session.createConsumer(destination);
> BlobMessage message = session.createBlobMessage(file);
>
> producer.send(message);
> System.out.println("Sent: " + message);
>
> Thread.sleep(1000);
>
> // check message sent
> Message msg = consumer.receive();
>
> BlobDownloadStrategy strategy = new FTPBlobDownloadStrategy();
>
> InputStream input = strategy.getInputStream((ActiveMQBlobMessage)msg);
> File f=new File(fileName);
> OutputStream out=new FileOutputStream(f);
> byte buf[]=new byte[1024];
> int len;
>
> while((len=input.read(buf))>0){
> out.write(buf,0,len);
> }
> out.close();
> input.close();
> System.out.println("Received: " + message);
>
> {code}
> After examining org.apache.activemq.blob.FTPBlobDownloadStrategy it seemed suspicious that the FTPClient connection was destroyed before the input stream is processed.
> {code}
> public InputStream getInputStream(ActiveMQBlobMessage message) throws IOException, JMSException {
> URL url = message.getURL();
>
> setUserInformation(url.getUserInfo());
> String connectUrl = url.getHost();
> int port = url.getPort() < 1 ? 21 : url.getPort();
> FTPClient ftp = new FTPClient();
> try {
> ftp.connect(connectUrl, port);
> } catch(ConnectException e) {
> throw new JMSException("Problem connecting the FTP-server");
> }
>
> if(!ftp.login(ftpUser, ftpPass)) {
> ftp.quit();
> ftp.disconnect();
> throw new JMSException("Cant Authentificate to FTP-Server");
> }
> String path = url.getPath();
> String workingDir = path.substring(0, path.lastIndexOf("/"));
> String file = path.substring(path.lastIndexOf("/")+1);
>
> ftp.changeWorkingDirectory(workingDir);
> ftp.setFileType(FTPClient.BINARY_FILE_TYPE);
> InputStream input = ftp.retrieveFileStream(file);
> ftp.quit(); // really?
> ftp.disconnect(); // really?
>
> return input;
> }
> {code}
> After commenting those two last ftp calls, files larger than 64 KB were downloaded properly, but this should of course not be the final solution.
> Any suggestions?
> Cheers, Toni
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (AMQ-2548) Downloading Blob messages via FTP fails
for files larger than 64KB
Posted by "Antun Pendo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/AMQ-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Antun Pendo updated AMQ-2548:
-----------------------------
Description:
The following code will only download 64 KB of any uploaded file greater than 64 KB.
The test file (ca. 15 MB) was completly uploaded to the FTP-Server.
{code}
File file = new File(directoryName+fileName);
ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(URI);
Connection connection = factory.createQueueConnection();
connection.start();
ActiveMQSession session = (ActiveMQSession) connection.createSession(
false, Session.AUTO_ACKNOWLEDGE);
Destination destination = session.createQueue("MyQ");
MessageProducer producer = session.createProducer(destination);
MessageConsumer consumer = session.createConsumer(destination);
BlobMessage message = session.createBlobMessage(file);
producer.send(message);
System.out.println("Sent: " + message);
Thread.sleep(1000);
// check message sent
Message msg = consumer.receive();
BlobDownloadStrategy strategy = new FTPBlobDownloadStrategy();
InputStream input = strategy.getInputStream((ActiveMQBlobMessage)msg);
File f=new File(fileName);
OutputStream out=new FileOutputStream(f);
byte buf[]=new byte[1024];
int len;
while((len=input.read(buf))>0){
out.write(buf,0,len);
}
out.close();
input.close();
System.out.println("Received: " + message);
{code}
After examining org.apache.activemq.blob.FTPBlobDownloadStrategy it seemed suspicious that the FTPClient connection was destroyed before the input stream is processed.
{code}
public InputStream getInputStream(ActiveMQBlobMessage message) throws IOException, JMSException {
URL url = message.getURL();
setUserInformation(url.getUserInfo());
String connectUrl = url.getHost();
int port = url.getPort() < 1 ? 21 : url.getPort();
FTPClient ftp = new FTPClient();
try {
ftp.connect(connectUrl, port);
} catch(ConnectException e) {
throw new JMSException("Problem connecting the FTP-server");
}
if(!ftp.login(ftpUser, ftpPass)) {
ftp.quit();
ftp.disconnect();
throw new JMSException("Cant Authentificate to FTP-Server");
}
String path = url.getPath();
String workingDir = path.substring(0, path.lastIndexOf("/"));
String file = path.substring(path.lastIndexOf("/")+1);
ftp.changeWorkingDirectory(workingDir);
ftp.setFileType(FTPClient.BINARY_FILE_TYPE);
InputStream input = ftp.retrieveFileStream(file);
ftp.quit(); // really?
ftp.disconnect(); // really?
return input;
}
{code}
After commenting those two last ftp calls, files larger than 64 KB were downloaded properly, but this should of course not be the final solution.
Any suggestions?
Cheers, Toni
was:
The following code will only download 64 KB of any uploaded file greater than 64 KB.
The test file (ca. 15 MB) was completly uploaded to the FTP-Server.
{code}
File file = new File(directoryName+fileName);
ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(URI);
Connection connection = factory.createQueueConnection();
connection.start();
ActiveMQSession session = (ActiveMQSession) connection.createSession(
false, Session.AUTO_ACKNOWLEDGE);
Destination destination = session.createQueue("MyQ");
MessageProducer producer = session.createProducer(destination);
MessageConsumer consumer = session.createConsumer(destination);
BlobMessage message = session.createBlobMessage(file);
producer.send(message);
System.out.println("Sent: " + message);
Thread.sleep(1000);
// check message sent
Message msg = consumer.receive();
BlobDownloadStrategy strategy = new FTPBlobDownloadStrategy();
InputStream input = strategy.getInputStream((ActiveMQBlobMessage)msg);
File f=new File(fileName);
OutputStream out=new FileOutputStream(f);
byte buf[]=new byte[1024];
int len;
while((len=input.read(buf))>0){
out.write(buf,0,len);
}
out.close();
input.close();
System.out.println("Received: " + message);
{code}
After examining org.apache.activemq.blob.DefaultBlobUploadStrategy it seemed suspicious that the FTPClient connection was destroyed before the input stream is processed.
{code}
public InputStream getInputStream(ActiveMQBlobMessage message) throws IOException, JMSException {
URL url = message.getURL();
setUserInformation(url.getUserInfo());
String connectUrl = url.getHost();
int port = url.getPort() < 1 ? 21 : url.getPort();
FTPClient ftp = new FTPClient();
try {
ftp.connect(connectUrl, port);
} catch(ConnectException e) {
throw new JMSException("Problem connecting the FTP-server");
}
if(!ftp.login(ftpUser, ftpPass)) {
ftp.quit();
ftp.disconnect();
throw new JMSException("Cant Authentificate to FTP-Server");
}
String path = url.getPath();
String workingDir = path.substring(0, path.lastIndexOf("/"));
String file = path.substring(path.lastIndexOf("/")+1);
ftp.changeWorkingDirectory(workingDir);
ftp.setFileType(FTPClient.BINARY_FILE_TYPE);
InputStream input = ftp.retrieveFileStream(file);
ftp.quit(); // really?
ftp.disconnect(); // really?
return input;
}
{code}
After commenting those two last ftp calls, files larger than 64 KB were downloaded properly, but this should of course not be the final solution.
Any suggestions?
Cheers, Toni
> Downloading Blob messages via FTP fails for files larger than 64KB
> ------------------------------------------------------------------
>
> Key: AMQ-2548
> URL: https://issues.apache.org/activemq/browse/AMQ-2548
> Project: ActiveMQ
> Issue Type: Bug
> Affects Versions: 5.3.0
> Environment: OS: Windows XP
> Reporter: Antun Pendo
> Priority: Minor
>
> The following code will only download 64 KB of any uploaded file greater than 64 KB.
> The test file (ca. 15 MB) was completly uploaded to the FTP-Server.
>
> {code}
> File file = new File(directoryName+fileName);
>
> ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory(URI);
> Connection connection = factory.createQueueConnection();
> connection.start();
>
> ActiveMQSession session = (ActiveMQSession) connection.createSession(
> false, Session.AUTO_ACKNOWLEDGE);
> Destination destination = session.createQueue("MyQ");
> MessageProducer producer = session.createProducer(destination);
> MessageConsumer consumer = session.createConsumer(destination);
> BlobMessage message = session.createBlobMessage(file);
>
> producer.send(message);
> System.out.println("Sent: " + message);
>
> Thread.sleep(1000);
>
> // check message sent
> Message msg = consumer.receive();
>
> BlobDownloadStrategy strategy = new FTPBlobDownloadStrategy();
>
> InputStream input = strategy.getInputStream((ActiveMQBlobMessage)msg);
> File f=new File(fileName);
> OutputStream out=new FileOutputStream(f);
> byte buf[]=new byte[1024];
> int len;
>
> while((len=input.read(buf))>0){
> out.write(buf,0,len);
> }
> out.close();
> input.close();
> System.out.println("Received: " + message);
>
> {code}
> After examining org.apache.activemq.blob.FTPBlobDownloadStrategy it seemed suspicious that the FTPClient connection was destroyed before the input stream is processed.
> {code}
> public InputStream getInputStream(ActiveMQBlobMessage message) throws IOException, JMSException {
> URL url = message.getURL();
>
> setUserInformation(url.getUserInfo());
> String connectUrl = url.getHost();
> int port = url.getPort() < 1 ? 21 : url.getPort();
> FTPClient ftp = new FTPClient();
> try {
> ftp.connect(connectUrl, port);
> } catch(ConnectException e) {
> throw new JMSException("Problem connecting the FTP-server");
> }
>
> if(!ftp.login(ftpUser, ftpPass)) {
> ftp.quit();
> ftp.disconnect();
> throw new JMSException("Cant Authentificate to FTP-Server");
> }
> String path = url.getPath();
> String workingDir = path.substring(0, path.lastIndexOf("/"));
> String file = path.substring(path.lastIndexOf("/")+1);
>
> ftp.changeWorkingDirectory(workingDir);
> ftp.setFileType(FTPClient.BINARY_FILE_TYPE);
> InputStream input = ftp.retrieveFileStream(file);
> ftp.quit(); // really?
> ftp.disconnect(); // really?
>
> return input;
> }
> {code}
> After commenting those two last ftp calls, files larger than 64 KB were downloaded properly, but this should of course not be the final solution.
> Any suggestions?
> Cheers, Toni
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.