Request: Direct access to inputstream from

[Mobinor <to...@mobinor.no>]

As part of embedding Apache Mina FTP server, we need access to the stream
from the ftp connection from within our application. 

 

We have decided to implement our own STORE command, and then catching the
data stream from the DataConnection. 

 

Our issue is that the DataConnection only offers an outputstream(
transferFromClient(OutputStream out)). I have looked at the implementation
of the interface (org.apache.ftpserver.IODataConnection), and see there the
is a private method getDataInputStream. We would really like to get access
to this method from the DataConnection interface, or a similar method to get
direct access to the inputstream.

 

 

Mvh

 

Tore Storødegård 

 

AW: [SPAMCheck] RE: Request: Direct access to inputstream from no---signature

[Steve Ulrich <st...@proemion.com>]

Hi!

> mobic.com [mailto:tore@mobic.com] wrote:
> ByteArrayStream is what we are using now, but I am concerned about the
> memory consumption since we are planning for large files with video
> content.
> It might be that we have to implement an intermediate file storage then
> to
> avoid memory issues.

That's truly a problem.
Storing big binary files in a database is always a bit tricky, because most databases are designed to store huge amounts of small data (numbers and characters), not big binary information.
Maybe there's a chance with using PipedStreams, but I doubt it will end up in a good solution. I think temp files, as you mentioned, will be a better solution and easier to play with.

regards,

Steve

RE: Request: Direct access to inputstream from no---signature

["mobic.com" <to...@mobic.com>]

ByteArrayStream is what we are using now, but I am concerned about the
memory consumption since we are planning for large files with video content.
It might be that we have to implement an intermediate file storage then to
avoid memory issues.

Thanks for the feedback.


Tore

Hi!

> mobic.com [mailto:tore@mobic.com] wrote
>
>
> I must admit that I have not studied the implementation of MINA ftp in
> detail, so I will rather describe our need.
>
> Our intent is to store a file as a blob directly in a database, not
> using
> the file system. A very quick implementation is this one line code:
>
> Hibernate.createBlob(inputstream)
>
> This requires access to the data as an inputstream.

The most simple solution would be to fetch the bytes of the File you get,
put them in an array and the array in a ByteArrayInputStream, so you get
your InputStream, without needing access to the underlying IO
implementation.

> To get access to the stream we have created our own STOR command, and
> we
> then have to convert the outputstream to an inputstream to be able to
> use
> the Hibernate util.

I don't think a simple passthrough will get you what you want, because (As
far as I remember), the Hibernate util try to read all bytes from the
inputstream, it get. So as long as your connection is open, the Hibernate
util won't return, because the IO-InputStream always says "There's more
data, just wait" until connection is closed.

> With the current implementation of the IODataConnection it seams very
> simple
> to get access to the inputstream. If you intent to change the design
> this
> might be another case. If so, please evaluate our need and implement a
> proper solution. Store data directly into a database will be a very
> important usage of an embedded MINA ftp server.

Yes, I will have to do a similar solution in the next time. But If design
have to be broken for my solution, I would first question my solution and
then the design

> Please tell if you have better ways of using MINAftp for storing files
> directly into a database.

HTH

Steve

AW: Request: Direct access to inputstream from no---signature

[Steve Ulrich <st...@proemion.com>]

Hi!

> mobic.com [mailto:tore@mobic.com] wrote
>
>
> I must admit that I have not studied the implementation of MINA ftp in
> detail, so I will rather describe our need.
>
> Our intent is to store a file as a blob directly in a database, not
> using
> the file system. A very quick implementation is this one line code:
>
> Hibernate.createBlob(inputstream)
>
> This requires access to the data as an inputstream.

The most simple solution would be to fetch the bytes of the File you get, put them in an array and the array in a ByteArrayInputStream, so you get your InputStream, without needing access to the underlying IO implementation.

> To get access to the stream we have created our own STOR command, and
> we
> then have to convert the outputstream to an inputstream to be able to
> use
> the Hibernate util.

I don't think a simple passthrough will get you what you want, because (As far as I remember), the Hibernate util try to read all bytes from the inputstream, it get. So as long as your connection is open, the Hibernate util won't return, because the IO-InputStream always says "There's more data, just wait" until connection is closed.

> With the current implementation of the IODataConnection it seams very
> simple
> to get access to the inputstream. If you intent to change the design
> this
> might be another case. If so, please evaluate our need and implement a
> proper solution. Store data directly into a database will be a very
> important usage of an embedded MINA ftp server.

Yes, I will have to do a similar solution in the next time. But If design have to be broken for my solution, I would first question my solution and then the design

> Please tell if you have better ways of using MINAftp for storing files
> directly into a database.

HTH

Steve

RE: Request: Direct access to inputstream from

["mobic.com" <to...@mobic.com>]

Niklas

I have always been lazy, letting a container do most of the jdbc stuff for
me (first EJB, now spring+jpa+annotation), so plain jdbc is not my strength.
But I will tell if I get time to do a full filesystem implementation, which
is currently beyond the scope of our project. It might be that I end up
doing either jdbc or a full Spring implementation to fulfill our needs. I
will let you know.


Tore


On Mon, Aug 25, 2008 at 3:59 PM, mobic.com <to...@mobic.com> wrote:
> A plain jdbc implementation of the file system is a good idea. For our
need
> we also need the application to be notified, I suppose this can be handled
> by a small ftplet.

Would you be able to contribute one to the project? I'll be happy
point you in the right direction but I don't have the time to develop
the entire solution at the moment.

> A spring implementation should also be evaluated. We are using JPA+Spring
> for our implementation, and are currently testing JPA+Spring for the STORE
> command.

I think that for a basic implementation, Spring+JPA is a bit
heavy-weight to include in the code project, but I would be happy to
have one as either an example or as an extension.

/niklas

Re: Request: Direct access to inputstream from

[Niklas Gustavsson <ni...@protocol7.com>]

On Mon, Aug 25, 2008 at 3:59 PM, mobic.com <to...@mobic.com> wrote:
> A plain jdbc implementation of the file system is a good idea. For our need
> we also need the application to be notified, I suppose this can be handled
> by a small ftplet.

Would you be able to contribute one to the project? I'll be happy
point you in the right direction but I don't have the time to develop
the entire solution at the moment.

> A spring implementation should also be evaluated. We are using JPA+Spring
> for our implementation, and are currently testing JPA+Spring for the STORE
> command.

I think that for a basic implementation, Spring+JPA is a bit
heavy-weight to include in the code project, but I would be happy to
have one as either an example or as an extension.

/niklas

RE: Request: Direct access to inputstream from

["mobic.com" <to...@mobic.com>]

A plain jdbc implementation of the file system is a good idea. For our need
we also need the application to be notified, I suppose this can be handled
by a small ftplet.

A spring implementation should also be evaluated. We are using JPA+Spring
for our implementation, and are currently testing JPA+Spring for the STORE
command.


Mvh

Tore Storødegård
tore@mobic.no

Mobinor AS - For deg som trenger bistand til utvikling og drift av
nettbaserte løsninger (www.mobinor.no)
Timebox AS - For deg som trenger hjelp til realisering av ideer og it
baserte prosjekter (www.timebox.no)
Mobilfoto AS - For deg som ønsker å fremkalle bider på mobiltelefon
(www.mobilfoto.no)
Zlip AS - For deg som ønsker å tilby en trådløse sone til venner og kunder
(www.zlip.no)


Alle dyr, bortsett fra mennesket, vet at den eneste meningen med livet er å
nyte det. Samuel Butler


-----Opprinnelig melding-----
Fra: Niklas Gustavsson [mailto:niklas@protocol7.com] 
Sendt: 25. august 2008 13:28
Til: dev@mina.apache.org
Emne: Re: Request: Direct access to inputstream from

On Mon, Aug 25, 2008 at 11:26 AM, mobic.com <to...@mobic.com> wrote:
> I must admit that I have not studied the implementation of MINA ftp in
> detail, so I will rather describe our need.
>
> Our intent is to store a file as a blob directly in a database, not using
> the file system. A very quick implementation is this one line code:
>
> Hibernate.createBlob(inputstream)
>
> This requires access to the data as an inputstream.
>
> To get access to the stream we have created our own STOR command, and we
> then have to convert the outputstream to an inputstream to be able to use
> the Hibernate util.
>
> With the current implementation of the IODataConnection it seams very
simple
> to get access to the inputstream. If you intent to change the design this
> might be another case. If so, please evaluate our need and implement a
> proper solution. Store data directly into a database will be a very
> important usage of an embedded MINA ftp server.
>
> Please tell if you have better ways of using MINAftp for storing files
> directly into a database.

The "normal" way of doing this would be to create a custom file system
implementation. That way, your storage would work with any of the FTP
commands, like listing or any of the storage commands (STOR, STOU,
APPE).

I would like to have a plain JDBC based file system implementation as
part of the project.

/niklas

Re: Request: Direct access to inputstream from

[Niklas Gustavsson <ni...@protocol7.com>]

On Mon, Aug 25, 2008 at 11:26 AM, mobic.com <to...@mobic.com> wrote:
> I must admit that I have not studied the implementation of MINA ftp in
> detail, so I will rather describe our need.
>
> Our intent is to store a file as a blob directly in a database, not using
> the file system. A very quick implementation is this one line code:
>
> Hibernate.createBlob(inputstream)
>
> This requires access to the data as an inputstream.
>
> To get access to the stream we have created our own STOR command, and we
> then have to convert the outputstream to an inputstream to be able to use
> the Hibernate util.
>
> With the current implementation of the IODataConnection it seams very simple
> to get access to the inputstream. If you intent to change the design this
> might be another case. If so, please evaluate our need and implement a
> proper solution. Store data directly into a database will be a very
> important usage of an embedded MINA ftp server.
>
> Please tell if you have better ways of using MINAftp for storing files
> directly into a database.

The "normal" way of doing this would be to create a custom file system
implementation. That way, your storage would work with any of the FTP
commands, like listing or any of the storage commands (STOR, STOU,
APPE).

I would like to have a plain JDBC based file system implementation as
part of the project.

/niklas

RE: Request: Direct access to inputstream from

["mobic.com" <to...@mobic.com>]

On Sun, Aug 24, 2008 at 12:41 PM, Mobinor <to...@mobinor.no> wrote:
> As part of embedding Apache Mina FTP server, we need access to the stream
> from the ftp connection from within our application.
>
> We have decided to implement our own STORE command, and then catching the
> data stream from the DataConnection.
>
> Our issue is that the DataConnection only offers an outputstream(
> transferFromClient(OutputStream out)). I have looked at the implementation
> of the interface (org.apache.ftpserver.IODataConnection), and see there
the
> is a private method getDataInputStream. We would really like to get access
> to this method from the DataConnection interface, or a similar method to
get
> direct access to the inputstream.

Yes, the reasoning behind the design of the interface is that I would
like to do a MINA based implementation of the data connection. That
would mean that we would not have an input stream at all.

That means that even if we made getDataInputStream (or similar)
public, you would be dependent on the internal implementation, would
that be sufficient for you? Be aware that if we do get around to
making a MINA based data connection and it turns out to work better
than the current implementation, we're likely to drop
IoDataConnection.

Could you also describe why you need direct access to the input stream
rather than providing an output stream for the data connection to read
from?

/niklas


Thanks Niklas

I must admit that I have not studied the implementation of MINA ftp in
detail, so I will rather describe our need.

Our intent is to store a file as a blob directly in a database, not using
the file system. A very quick implementation is this one line code:

Hibernate.createBlob(inputstream)

This requires access to the data as an inputstream.

To get access to the stream we have created our own STOR command, and we
then have to convert the outputstream to an inputstream to be able to use
the Hibernate util.

With the current implementation of the IODataConnection it seams very simple
to get access to the inputstream. If you intent to change the design this
might be another case. If so, please evaluate our need and implement a
proper solution. Store data directly into a database will be a very
important usage of an embedded MINA ftp server.

Please tell if you have better ways of using MINAftp for storing files
directly into a database.



Tore

Re: Request: Direct access to inputstream from

[Niklas Gustavsson <ni...@protocol7.com>]

On Sun, Aug 24, 2008 at 12:41 PM, Mobinor <to...@mobinor.no> wrote:
> As part of embedding Apache Mina FTP server, we need access to the stream
> from the ftp connection from within our application.
>
> We have decided to implement our own STORE command, and then catching the
> data stream from the DataConnection.
>
> Our issue is that the DataConnection only offers an outputstream(
> transferFromClient(OutputStream out)). I have looked at the implementation
> of the interface (org.apache.ftpserver.IODataConnection), and see there the
> is a private method getDataInputStream. We would really like to get access
> to this method from the DataConnection interface, or a similar method to get
> direct access to the inputstream.

Yes, the reasoning behind the design of the interface is that I would
like to do a MINA based implementation of the data connection. That
would mean that we would not have an input stream at all.

That means that even if we made getDataInputStream (or similar)
public, you would be dependent on the internal implementation, would
that be sufficient for you? Be aware that if we do get around to
making a MINA based data connection and it turns out to work better
than the current implementation, we're likely to drop
IoDataConnection.

Could you also describe why you need direct access to the input stream
rather than providing an output stream for the data connection to read
from?

/niklas