Re: Producing and consuming binary data

[Dan Haywood <da...@haywood-associates.co.uk>]

On 21 August 2012 21:04, Giedrius Graževičius <me...@inightmare.org> wrote:

> Hello,
>
> Hi Giedrius

... and thanks, by the way, for your patch on the Wicket viewer (ISIS-239)
- as you've probably seen, I've applied it.




> Often there is a need to
>
> 1) produce some kind of a document (receipt, report, etc.) when a user
> invokes an action
> 2) store and retrieve files together with other business data
>
> As I understand, currently, Apache Isis does not support any of the
> use cases described above. Issue list contains an item ISIS-172 that
> talks about building documents, but does not mention a delivery
> method.
>

I see these as rather separate from each other, so will discuss them
separately.


(1) ISIS-172 addresses this, using an approach similar to the way we do
this on the big NO project in Ireland.  The requirement there is to allow
"the business" to develop communications in Word, but then use some sort of
mail merge capability to inject in values from the domain object model.

The JIRA issue describes the use of an XML input file; note also that some
information is multi-valued, eg the "Payments" element which would
correspond to a table.  This probably points to a more sophisticated API
than a simple mail merge (the latter of which can generally only merge in
single scalar values such as a name or date).

The Irish project is implemented in .NET, and Microsoft provide a rather
good .NET API for manipulating OpenXml (.docx) files.  When I went looking
just now I couldn't find anything similar in Java-land, so I've just
updated the JIRA to suggest that OpenDocument format (ODF / Apache
OpenOffice) be considered instead.

In terms of how a domain object would use this service, let me again
explain how the Irish project works: we have a CommunicationTemplate domain
object, which holds the .docx (or .odf) as a base 64 encoded string.  The
CommunicationTemplate acts as a factory for Communication objects,
requiring that the appropriate XML be provided to it.  The template then
calls the DocumentBuilderService and gets back a mail-merged version; this
is stored in the resultant Communication (again base 64 encoded).

This sequence diagram [1] demonstrates this idea.  (And I've updated the
JIRA with all the above so we don't lose it).


(2) one option for the storing of generated docs is to store them in domain
objects that represent them.  This is what CommunicationTemplate and
Communication, described above, do.  It does mean that the database grows
pretty large of course, but RDBMS can handle these sorts of things
perfectly well, and it saves having lots of complexity elsewhere.

Using base 64 encoded strings also means that there's no new datatypes to
be implemented.  There is an argument, perhaps, for Isis to support a Blob
type as well though, to be a little more space-efficient.

I suspect you were considering some sort of "federated" object store,
whereby the domain object holds some sort of URI link to a generated doc,
that is itself served up via a file server?  Of course, such a thing could
be designed, but it would take more work to ensure that the viewers knew
how to dereference such a URI, and it would certainly add complexity to the
backend object store impls.



>
> Are there any plans to add ability for services to generate binary
> data and for domain entities to store such data?


As I noted above, there is an argument, perhaps, for Isis to support a Blob
type as well though, to be a little more space-efficient.  By all means
raise a JIRA for this if you think it's warranted.



> Or would it be
> possible to contribute such functionality to the project?
>
> Contributions always welcome, of course!  But before heading off on your
own, I suggest we continue the discussion on this thread a little while,
make sure we've come to a shared understanding as to what would be worth
implementing, and how to do it.

Many thanks,
Dan



>
> Giedrius Graževičius
>


[1]
http://www.websequencediagrams.com/cgi-bin/cdraw?lz=dGl0bGUgY3JlYXRlIENvbW11bmljYXRpb24KCkN1c3RvbWVyLT4rAA0NVGVtcGxhdGVSZXBvc2l0b3J5OiBsb29rdXAodAAUB0NvZGUpCgAXHy0tPi0AXAg6IABkDAAMCmJ1aWxkIHhtbABtIjoAgUAHKHhtbCkKAGcWLT4rRG9jdW1lbnRCdWlsZGVyU2VydmljZTogbWFpbE1lcmdlKHRoaXMub2RmAIFgCCwgAE0FACMWAIFABQB5Fm1lcmdlZE9kZgB0GQCCZA06IGluc3RhbnRpYXRlKAA0CQCBMhkAgi8PCg&s=qsd

Re: Producing and consuming binary data

[Kevin Meyer - KMZ <ke...@kmz.co.za>]

Just to help mailing list cross-referencing - this thread continues 
here[1] as "Regarding ISIS-254".

[1] http://isis.markmail.org/message/qmuw25gd4rfapud7

On 2 Sep 2012 at 13:08, Dan Haywood wrote:

> On 2 September 2012 12:44, Giedrius Grazevicius <me...@inightmare.org> wrote:
> >
> > Currently looking into the binary support. I've added simple byte[]
> > support in a Wicket viewer (treats the property as a file that can be
> > uploaded). Changes can be seen here:
> > https://github.com/INightmare/apache-isis/commits/ISIS-254
> >
> > Thanks for taking this forward.  Had a quick review of those commits,
> looks like you're progressing fine.  (Nice to see for me, since I've given
> you zero guidance around the code base ;-)
> 
> 
> 
> > Now to figure out a way to make Wicket treat such property a bit
> > differently than other values (that is prevent it from accessing it,
> > unless user specifically requested it).
> >
> 
> The normal way to do this would be to define a facet for this (if there
> isn't a suitable one), and a corresponding facet factory, then registered
> into ProgrammingModelFacetsJava5.
> 
> 
> 
> >
> > Also:
> >
> > When trying to edit newly created object in Wicket viewer with
> > DataNucleus as ObjectStore, the following exception occurs:
> > http://pastebin.com/4tCiCqiZ
> > Am I doing something wrong or is this a known problem?
> >
> 
> I think this might be a semi-known problem.  Jeroen and I were fixing a
> problem to do with Wicket and JDO on Friday evening, whereby a newly
> persisted object was then persisted twice if we immediately invoked an
> action on it.  The problem was that Wicket was serializing the state of the
> object into the page prior to the transaction actually executing (causing
> the adapter's state to move to transient to persistent).  My suspicion is
> that this is the same problem.
> 
> It's on the list to look at then...
> 
> Cheers
> Dan
> 

Re: Producing and consuming binary data

[Dan Haywood <da...@haywood-associates.co.uk>]

On 2 September 2012 12:44, Giedrius Graževičius <me...@inightmare.org> wrote:

>
> Currently looking into the binary support. I've added simple byte[]
> support in a Wicket viewer (treats the property as a file that can be
> uploaded). Changes can be seen here:
> https://github.com/INightmare/apache-isis/commits/ISIS-254
>
> Thanks for taking this forward.  Had a quick review of those commits,
looks like you're progressing fine.  (Nice to see for me, since I've given
you zero guidance around the code base ;-)



> Now to figure out a way to make Wicket treat such property a bit
> differently than other values (that is prevent it from accessing it,
> unless user specifically requested it).
>

The normal way to do this would be to define a facet for this (if there
isn't a suitable one), and a corresponding facet factory, then registered
into ProgrammingModelFacetsJava5.



>
> Also:
>
> When trying to edit newly created object in Wicket viewer with
> DataNucleus as ObjectStore, the following exception occurs:
> http://pastebin.com/4tCiCqiZ
> Am I doing something wrong or is this a known problem?
>

I think this might be a semi-known problem.  Jeroen and I were fixing a
problem to do with Wicket and JDO on Friday evening, whereby a newly
persisted object was then persisted twice if we immediately invoked an
action on it.  The problem was that Wicket was serializing the state of the
object into the page prior to the transaction actually executing (causing
the adapter's state to move to transient to persistent).  My suspicion is
that this is the same problem.

It's on the list to look at then...

Cheers
Dan

Re: Producing and consuming binary data

[Giedrius Graževičius <me...@inightmare.org>]

Than you, Dan.

Currently looking into the binary support. I've added simple byte[]
support in a Wicket viewer (treats the property as a file that can be
uploaded). Changes can be seen here:
https://github.com/INightmare/apache-isis/commits/ISIS-254

Now to figure out a way to make Wicket treat such property a bit
differently than other values (that is prevent it from accessing it,
unless user specifically requested it).

Also:

When trying to edit newly created object in Wicket viewer with
DataNucleus as ObjectStore, the following exception occurs:
http://pastebin.com/4tCiCqiZ
Am I doing something wrong or is this a known problem?



On Thu, Aug 30, 2012 at 2:12 PM, Dan Haywood
<da...@haywood-associates.co.uk> wrote:
> Hi Giedrius,
>
> just coming back to you on this.
>
> I've raised ISIS-252 for the blob support; meanwhile ISIS-172 is already
> open for a document PDF service (as previously discussed).
>
> An Apache FOP-based service is definitely also a possibility; if you decide
> to raise a ticket on this, then please assign to the "Domain: Services"
> component.
>
> Thx
> Dan
>
>
> On 22 August 2012 18:49, Giedrius Graževičius <me...@inightmare.org> wrote:
>
>> Hi Dan.
>>
>> thank you for your answers.
>>
>> > (2) one option for the storing of generated docs is to store them in
>> domain
>> > objects that represent them.  This is what CommunicationTemplate and
>> > Communication, described above, do.  It does mean that the database grows
>> > pretty large of course, but RDBMS can handle these sorts of things
>> > perfectly well, and it saves having lots of complexity elsewhere.
>>
>> This is exactly of what I had in mind. I'm a big fan of storing all
>> the data in one place (usually meaning some kind of DBMS). The BLOB
>> type would be perfect. It's just that in such cases if it's an
>> attribute of a domain object it should be lazily loaded. I'm not sure
>> if this is already covered in some way in Isis storage infrastructure
>> (as all properties and one-to-one relations are likely to be fetched
>> eagerly).
>>
>> > Contributions always welcome, of course!  But before heading off on your
>> > own, I suggest we continue the discussion on this thread a little while,
>> > make sure we've come to a shared understanding as to what would be worth
>> > implementing, and how to do it.
>>
>> This was the intent of my e-mail.
>>
>> What I envision that these blob typed attributes would be handled as
>> files by the viewers. That is the file upload box would be presented
>> and if there is some data present it would be possible to download it
>> as a file. The implementation for this seems to be pretty
>> straightforward, though all the views need to be updated to support
>> this (an ability to disable file reupload is a must, as some systems,
>> like document/record management, don't allow modifying the data after
>> it has been created).
>>
>> There would also be some type of an action that would return produced
>> documents directly to the user. For example user wants report for
>> April, he enters the date and gets the report as a direct download.
>> (In my simple POC implementation I apply a Facet to all the actions
>> starting with "produce" and expect those actions to return an
>> InputStream, though a more efficient solution would be to pass an
>> OutputStream to directly write the data).
>>
>> So from what I see
>> 1) all object sores need to be updated to support the lazy loading of blobs
>> 2) a way of presenting those types should be added to the viewers
>> 3) a way of allowing to directly return binary content from an action
>> would be added to the viewers (This is optional as files can be stored
>> in domain objects. It's fine in most cases. However, my argument pro
>> direct downloads is that not all the documents need to be stored, also
>> if a document (report, template) can be generated quickly it's much
>> more convenient for the user to just get it directly. It also
>> simplifies the case of external integrations as instead of two calls:
>> generate and retrieve, you only have to do one generate-and-retrieve).
>>
>> Despite the effort associated I consider this a must-have feature, as
>> most enterprise systems I have seen/worked on deal with documents in
>> one way or another.
>>
>> As for an out of the box document generation solution, I was thinking
>> about Apache FOP, as solution like Jasper Reports is probably not
>> "free enough" for Apache projects. They also have ODF support in their
>> wish list, maybe someone is already working on this as ODF Toolkit
>> (and the whole OpenOffice) is now an Apache project.
>>
>> Giedrius
>>

Re: Producing and consuming binary data

[Giedrius Graževičius <me...@inightmare.org>]

Than you, Dan.

Currently looking into the binary support. I've added simple byte[]
support in a Wicket viewer (treats the property as a file that can be
uploaded). Changes can be seen here:
https://github.com/INightmare/apache-isis/commits/ISIS-254

Now to figure out a way to make Wicket treat such property a bit
differently than other values (that is prevent it from accessing it,
unless user specifically requested it).

Also:

When trying to edit newly created object in Wicket viewer with
DataNucleus as ObjectStore, the following exception occurs:
http://pastebin.com/4tCiCqiZ
Am I doing something wrong or is this a known problem?



On Thu, Aug 30, 2012 at 2:12 PM, Dan Haywood
<da...@haywood-associates.co.uk> wrote:
> Hi Giedrius,
>
> just coming back to you on this.
>
> I've raised ISIS-252 for the blob support; meanwhile ISIS-172 is already
> open for a document PDF service (as previously discussed).
>
> An Apache FOP-based service is definitely also a possibility; if you decide
> to raise a ticket on this, then please assign to the "Domain: Services"
> component.
>
> Thx
> Dan
>
>
> On 22 August 2012 18:49, Giedrius Graževičius <me...@inightmare.org> wrote:
>
>> Hi Dan.
>>
>> thank you for your answers.
>>
>> > (2) one option for the storing of generated docs is to store them in
>> domain
>> > objects that represent them.  This is what CommunicationTemplate and
>> > Communication, described above, do.  It does mean that the database grows
>> > pretty large of course, but RDBMS can handle these sorts of things
>> > perfectly well, and it saves having lots of complexity elsewhere.
>>
>> This is exactly of what I had in mind. I'm a big fan of storing all
>> the data in one place (usually meaning some kind of DBMS). The BLOB
>> type would be perfect. It's just that in such cases if it's an
>> attribute of a domain object it should be lazily loaded. I'm not sure
>> if this is already covered in some way in Isis storage infrastructure
>> (as all properties and one-to-one relations are likely to be fetched
>> eagerly).
>>
>> > Contributions always welcome, of course!  But before heading off on your
>> > own, I suggest we continue the discussion on this thread a little while,
>> > make sure we've come to a shared understanding as to what would be worth
>> > implementing, and how to do it.
>>
>> This was the intent of my e-mail.
>>
>> What I envision that these blob typed attributes would be handled as
>> files by the viewers. That is the file upload box would be presented
>> and if there is some data present it would be possible to download it
>> as a file. The implementation for this seems to be pretty
>> straightforward, though all the views need to be updated to support
>> this (an ability to disable file reupload is a must, as some systems,
>> like document/record management, don't allow modifying the data after
>> it has been created).
>>
>> There would also be some type of an action that would return produced
>> documents directly to the user. For example user wants report for
>> April, he enters the date and gets the report as a direct download.
>> (In my simple POC implementation I apply a Facet to all the actions
>> starting with "produce" and expect those actions to return an
>> InputStream, though a more efficient solution would be to pass an
>> OutputStream to directly write the data).
>>
>> So from what I see
>> 1) all object sores need to be updated to support the lazy loading of blobs
>> 2) a way of presenting those types should be added to the viewers
>> 3) a way of allowing to directly return binary content from an action
>> would be added to the viewers (This is optional as files can be stored
>> in domain objects. It's fine in most cases. However, my argument pro
>> direct downloads is that not all the documents need to be stored, also
>> if a document (report, template) can be generated quickly it's much
>> more convenient for the user to just get it directly. It also
>> simplifies the case of external integrations as instead of two calls:
>> generate and retrieve, you only have to do one generate-and-retrieve).
>>
>> Despite the effort associated I consider this a must-have feature, as
>> most enterprise systems I have seen/worked on deal with documents in
>> one way or another.
>>
>> As for an out of the box document generation solution, I was thinking
>> about Apache FOP, as solution like Jasper Reports is probably not
>> "free enough" for Apache projects. They also have ODF support in their
>> wish list, maybe someone is already working on this as ODF Toolkit
>> (and the whole OpenOffice) is now an Apache project.
>>
>> Giedrius
>>

Re: Producing and consuming binary data

[Dan Haywood <da...@haywood-associates.co.uk>]

Hi Giedrius,

just coming back to you on this.

I've raised ISIS-252 for the blob support; meanwhile ISIS-172 is already
open for a document PDF service (as previously discussed).

An Apache FOP-based service is definitely also a possibility; if you decide
to raise a ticket on this, then please assign to the "Domain: Services"
component.

Thx
Dan


On 22 August 2012 18:49, Giedrius Graževičius <me...@inightmare.org> wrote:

> Hi Dan.
>
> thank you for your answers.
>
> > (2) one option for the storing of generated docs is to store them in
> domain
> > objects that represent them.  This is what CommunicationTemplate and
> > Communication, described above, do.  It does mean that the database grows
> > pretty large of course, but RDBMS can handle these sorts of things
> > perfectly well, and it saves having lots of complexity elsewhere.
>
> This is exactly of what I had in mind. I'm a big fan of storing all
> the data in one place (usually meaning some kind of DBMS). The BLOB
> type would be perfect. It's just that in such cases if it's an
> attribute of a domain object it should be lazily loaded. I'm not sure
> if this is already covered in some way in Isis storage infrastructure
> (as all properties and one-to-one relations are likely to be fetched
> eagerly).
>
> > Contributions always welcome, of course!  But before heading off on your
> > own, I suggest we continue the discussion on this thread a little while,
> > make sure we've come to a shared understanding as to what would be worth
> > implementing, and how to do it.
>
> This was the intent of my e-mail.
>
> What I envision that these blob typed attributes would be handled as
> files by the viewers. That is the file upload box would be presented
> and if there is some data present it would be possible to download it
> as a file. The implementation for this seems to be pretty
> straightforward, though all the views need to be updated to support
> this (an ability to disable file reupload is a must, as some systems,
> like document/record management, don't allow modifying the data after
> it has been created).
>
> There would also be some type of an action that would return produced
> documents directly to the user. For example user wants report for
> April, he enters the date and gets the report as a direct download.
> (In my simple POC implementation I apply a Facet to all the actions
> starting with "produce" and expect those actions to return an
> InputStream, though a more efficient solution would be to pass an
> OutputStream to directly write the data).
>
> So from what I see
> 1) all object sores need to be updated to support the lazy loading of blobs
> 2) a way of presenting those types should be added to the viewers
> 3) a way of allowing to directly return binary content from an action
> would be added to the viewers (This is optional as files can be stored
> in domain objects. It's fine in most cases. However, my argument pro
> direct downloads is that not all the documents need to be stored, also
> if a document (report, template) can be generated quickly it's much
> more convenient for the user to just get it directly. It also
> simplifies the case of external integrations as instead of two calls:
> generate and retrieve, you only have to do one generate-and-retrieve).
>
> Despite the effort associated I consider this a must-have feature, as
> most enterprise systems I have seen/worked on deal with documents in
> one way or another.
>
> As for an out of the box document generation solution, I was thinking
> about Apache FOP, as solution like Jasper Reports is probably not
> "free enough" for Apache projects. They also have ODF support in their
> wish list, maybe someone is already working on this as ODF Toolkit
> (and the whole OpenOffice) is now an Apache project.
>
> Giedrius
>

Re: Producing and consuming binary data

[Dan Haywood <da...@haywood-associates.co.uk>]

Hi Giedrius,

just coming back to you on this.

I've raised ISIS-252 for the blob support; meanwhile ISIS-172 is already
open for a document PDF service (as previously discussed).

An Apache FOP-based service is definitely also a possibility; if you decide
to raise a ticket on this, then please assign to the "Domain: Services"
component.

Thx
Dan


On 22 August 2012 18:49, Giedrius Graževičius <me...@inightmare.org> wrote:

> Hi Dan.
>
> thank you for your answers.
>
> > (2) one option for the storing of generated docs is to store them in
> domain
> > objects that represent them.  This is what CommunicationTemplate and
> > Communication, described above, do.  It does mean that the database grows
> > pretty large of course, but RDBMS can handle these sorts of things
> > perfectly well, and it saves having lots of complexity elsewhere.
>
> This is exactly of what I had in mind. I'm a big fan of storing all
> the data in one place (usually meaning some kind of DBMS). The BLOB
> type would be perfect. It's just that in such cases if it's an
> attribute of a domain object it should be lazily loaded. I'm not sure
> if this is already covered in some way in Isis storage infrastructure
> (as all properties and one-to-one relations are likely to be fetched
> eagerly).
>
> > Contributions always welcome, of course!  But before heading off on your
> > own, I suggest we continue the discussion on this thread a little while,
> > make sure we've come to a shared understanding as to what would be worth
> > implementing, and how to do it.
>
> This was the intent of my e-mail.
>
> What I envision that these blob typed attributes would be handled as
> files by the viewers. That is the file upload box would be presented
> and if there is some data present it would be possible to download it
> as a file. The implementation for this seems to be pretty
> straightforward, though all the views need to be updated to support
> this (an ability to disable file reupload is a must, as some systems,
> like document/record management, don't allow modifying the data after
> it has been created).
>
> There would also be some type of an action that would return produced
> documents directly to the user. For example user wants report for
> April, he enters the date and gets the report as a direct download.
> (In my simple POC implementation I apply a Facet to all the actions
> starting with "produce" and expect those actions to return an
> InputStream, though a more efficient solution would be to pass an
> OutputStream to directly write the data).
>
> So from what I see
> 1) all object sores need to be updated to support the lazy loading of blobs
> 2) a way of presenting those types should be added to the viewers
> 3) a way of allowing to directly return binary content from an action
> would be added to the viewers (This is optional as files can be stored
> in domain objects. It's fine in most cases. However, my argument pro
> direct downloads is that not all the documents need to be stored, also
> if a document (report, template) can be generated quickly it's much
> more convenient for the user to just get it directly. It also
> simplifies the case of external integrations as instead of two calls:
> generate and retrieve, you only have to do one generate-and-retrieve).
>
> Despite the effort associated I consider this a must-have feature, as
> most enterprise systems I have seen/worked on deal with documents in
> one way or another.
>
> As for an out of the box document generation solution, I was thinking
> about Apache FOP, as solution like Jasper Reports is probably not
> "free enough" for Apache projects. They also have ODF support in their
> wish list, maybe someone is already working on this as ODF Toolkit
> (and the whole OpenOffice) is now an Apache project.
>
> Giedrius
>

Re: Producing and consuming binary data

[Giedrius Graževičius <me...@inightmare.org>]

Hi Dan.

thank you for your answers.

> (2) one option for the storing of generated docs is to store them in domain
> objects that represent them.  This is what CommunicationTemplate and
> Communication, described above, do.  It does mean that the database grows
> pretty large of course, but RDBMS can handle these sorts of things
> perfectly well, and it saves having lots of complexity elsewhere.

This is exactly of what I had in mind. I'm a big fan of storing all
the data in one place (usually meaning some kind of DBMS). The BLOB
type would be perfect. It's just that in such cases if it's an
attribute of a domain object it should be lazily loaded. I'm not sure
if this is already covered in some way in Isis storage infrastructure
(as all properties and one-to-one relations are likely to be fetched
eagerly).

> Contributions always welcome, of course!  But before heading off on your
> own, I suggest we continue the discussion on this thread a little while,
> make sure we've come to a shared understanding as to what would be worth
> implementing, and how to do it.

This was the intent of my e-mail.

What I envision that these blob typed attributes would be handled as
files by the viewers. That is the file upload box would be presented
and if there is some data present it would be possible to download it
as a file. The implementation for this seems to be pretty
straightforward, though all the views need to be updated to support
this (an ability to disable file reupload is a must, as some systems,
like document/record management, don't allow modifying the data after
it has been created).

There would also be some type of an action that would return produced
documents directly to the user. For example user wants report for
April, he enters the date and gets the report as a direct download.
(In my simple POC implementation I apply a Facet to all the actions
starting with "produce" and expect those actions to return an
InputStream, though a more efficient solution would be to pass an
OutputStream to directly write the data).

So from what I see
1) all object sores need to be updated to support the lazy loading of blobs
2) a way of presenting those types should be added to the viewers
3) a way of allowing to directly return binary content from an action
would be added to the viewers (This is optional as files can be stored
in domain objects. It's fine in most cases. However, my argument pro
direct downloads is that not all the documents need to be stored, also
if a document (report, template) can be generated quickly it's much
more convenient for the user to just get it directly. It also
simplifies the case of external integrations as instead of two calls:
generate and retrieve, you only have to do one generate-and-retrieve).

Despite the effort associated I consider this a must-have feature, as
most enterprise systems I have seen/worked on deal with documents in
one way or another.

As for an out of the box document generation solution, I was thinking
about Apache FOP, as solution like Jasper Reports is probably not
"free enough" for Apache projects. They also have ODF support in their
wish list, maybe someone is already working on this as ODF Toolkit
(and the whole OpenOffice) is now an Apache project.

Giedrius

Re: Producing and consuming binary data

[Giedrius Graževičius <me...@inightmare.org>]

Hi Dan.

thank you for your answers.

> (2) one option for the storing of generated docs is to store them in domain
> objects that represent them.  This is what CommunicationTemplate and
> Communication, described above, do.  It does mean that the database grows
> pretty large of course, but RDBMS can handle these sorts of things
> perfectly well, and it saves having lots of complexity elsewhere.

This is exactly of what I had in mind. I'm a big fan of storing all
the data in one place (usually meaning some kind of DBMS). The BLOB
type would be perfect. It's just that in such cases if it's an
attribute of a domain object it should be lazily loaded. I'm not sure
if this is already covered in some way in Isis storage infrastructure
(as all properties and one-to-one relations are likely to be fetched
eagerly).

> Contributions always welcome, of course!  But before heading off on your
> own, I suggest we continue the discussion on this thread a little while,
> make sure we've come to a shared understanding as to what would be worth
> implementing, and how to do it.

This was the intent of my e-mail.

What I envision that these blob typed attributes would be handled as
files by the viewers. That is the file upload box would be presented
and if there is some data present it would be possible to download it
as a file. The implementation for this seems to be pretty
straightforward, though all the views need to be updated to support
this (an ability to disable file reupload is a must, as some systems,
like document/record management, don't allow modifying the data after
it has been created).

There would also be some type of an action that would return produced
documents directly to the user. For example user wants report for
April, he enters the date and gets the report as a direct download.
(In my simple POC implementation I apply a Facet to all the actions
starting with "produce" and expect those actions to return an
InputStream, though a more efficient solution would be to pass an
OutputStream to directly write the data).

So from what I see
1) all object sores need to be updated to support the lazy loading of blobs
2) a way of presenting those types should be added to the viewers
3) a way of allowing to directly return binary content from an action
would be added to the viewers (This is optional as files can be stored
in domain objects. It's fine in most cases. However, my argument pro
direct downloads is that not all the documents need to be stored, also
if a document (report, template) can be generated quickly it's much
more convenient for the user to just get it directly. It also
simplifies the case of external integrations as instead of two calls:
generate and retrieve, you only have to do one generate-and-retrieve).

Despite the effort associated I consider this a must-have feature, as
most enterprise systems I have seen/worked on deal with documents in
one way or another.

As for an out of the box document generation solution, I was thinking
about Apache FOP, as solution like Jasper Reports is probably not
"free enough" for Apache projects. They also have ODF support in their
wish list, maybe someone is already working on this as ODF Toolkit
(and the whole OpenOffice) is now an Apache project.

Giedrius