You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cayenne.apache.org by Andrus Adamchik <an...@objectstyle.org> on 2015/09/22 21:57:42 UTC
Re: Change audit framework
Here is the design of the audit framework for everyone's review:
https://issues.apache.org/jira/browse/CAY-2030
The way it will work from the user perspective is this:
// bootstrap the filter by adding an extra Cayenne-provided module to
// runtime. Optionally add your own module that overrides context provider
// (e.g. to store request IP address and user name).
ServerRuntime r = ServerRuntimeBuilder.builder().addModule(new WhateverModuleNameWeUse()).build();
// register listener
r.getDataDomain().addListener(new MyListener());
// listener code
class MyListener {
@PostCommit
void processAsChangeset(Map<ObjectId, ObjectChangeSet> map) {..}
// optionally can take a changeset serialized to JSON
@PostCommit
void processAsJSON(Map<ObjectId, String> map) {..}
}
Andrus
Re: Change audit framework
Posted by Andrus Adamchik <an...@objectstyle.org>.
> On Sep 23, 2015, at 9:42 AM, Andrus Adamchik <an...@objectstyle.org> wrote:
>
>> Right now, if you modify an object, listeners can be fired on objects at the other end of its relations, even if no FK or other changes will be made in the database to those other objects. Do you anticipate the change set in the audit will include these kinds of related records, or only ones where a database attribute change is made?
>
> We are tracking changes at the object level, so IIRC the mechanism I am porting to Cayenne tracks all graph operations, including e.g. to-many relationship additions/subtractions that do not cause any SQL updates directly.
BTW, looks like to-many relationship reporting still has some issues. Hopefully easy to fix. Looking into this now.
Andrus
Re: Change audit framework
Posted by Andrus Adamchik <an...@objectstyle.org>.
> On Sep 23, 2015, at 12:58 AM, Aristedes Maniatis <ar...@maniatis.org> wrote:
> Why would you perform this serialisation for the user and not just supply a piece of sample code to put into the listener. Yes, json is the flavour of the day, but some people will want separate bits of json for every object, others will want to try and assemble the whole commit tree into one json string, others will want to do something else. Why would Cayenne itself have an opinion on this format?
>
> There are very few other places where Cayenne specifies the data format, Hessian being the only one I can think of.
I used a JAXRS-like approach of a pluggable serializer that produces a desired "media type" (e.g. [1]). I.e. you simply declare in your method signature that you want a String, and the framework provides a strategy for converting your object into that String, thus simplifying the listener code. JSON was supposed to be a default format, and would closely mirror the object structure. But yeah, serialization does feel out of place like you said. So perhaps we'll leave it out.
> My other question is about what objects will be found in the change set.
First, here is a structure of the change set object for a single DataObject (this is based on my internal code, that will be reworked for Cayenne inclusion) :
public class A2ObjectChangeSet {
private ObjectId preCommitId;
private ObjectId postCommitId;
private Map<String, A2PropertyChange> changes;
private Map<String, Object> snapshot;
// DELETE, INSERT, UPDATE
private AuditableOperation op;
// any extra properties that are common to the entire transaction,
// like request IP, or user name
private Map<String, Object> transactionContext;
}
So it has all the info on the object - a snapshot of properties before the object got changed (e.g. this way you might inspect a a previous state of a deleted object after commit), a list of changes and any extra metadata associated with transaction.
> Right now, if you modify an object, listeners can be fired on objects at the other end of its relations, even if no FK or other changes will be made in the database to those other objects. Do you anticipate the change set in the audit will include these kinds of related records, or only ones where a database attribute change is made?
We are tracking changes at the object level, so IIRC the mechanism I am porting to Cayenne tracks all graph operations, including e.g. to-many relationship additions/subtractions that do not cause any SQL updates directly.
Andrus
[1] https://jersey.java.net/documentation/latest/message-body-workers.html
Re: Change audit framework
Posted by Aristedes Maniatis <ar...@maniatis.org>.
On 23/09/2015 5:57am, Andrus Adamchik wrote:
> // optionally can take a changeset serialized to JSON
> @PostCommit
> void processAsJSON(Map<ObjectId, String> map) {..}
Why would you perform this serialisation for the user and not just supply a piece of sample code to put into the listener. Yes, json is the flavour of the day, but some people will want separate bits of json for every object, others will want to try and assemble the whole commit tree into one json string, others will want to do something else. Why would Cayenne itself have an opinion on this format?
There are very few other places where Cayenne specifies the data format, Hessian being the only one I can think of.
My other question is about what objects will be found in the change set. Right now, if you modify an object, listeners can be fired on objects at the other end of its relations, even if no FK or other changes will be made in the database to those other objects. Do you anticipate the change set in the audit will include these kinds of related records, or only ones where a database attribute change is made?
This is a very nice feature for Cayenne
Ari
--
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
Re: Change audit framework
Posted by Andrus Adamchik <an...@objectstyle.org>.
I don't have an example handy, but IIRC added object is [null, "id"], and removed is ["id", null]. I.e. we don't dump the entire rel state.
Andrus
> On Sep 23, 2015, at 11:28 AM, Hugi Thordarson <hu...@karlmenn.is> wrote:
>
> Ah, of course… How do you handle a change to a to-many relationship in that scheme, do oldValue and newValue contain the entire object lists pre- and post-modification?
>
> - hugi
>
>
>
>> On 23. sep. 2015, at 08:09, Andrus Adamchik <an...@objectstyle.org> wrote:
>>
>> Just a pair of values:
>>
>> public class A2PropertyChange {
>>
>> private Object oldValue;
>> private Object newValue;
>>
>> ...
>> }
>>
>> and as I've shown earlier, my system would emit a stream of JSON messages like this:
>>
>> {"ts":1427090346831,"by":"someuser","clientIP":"10.1.1.1","serverIP":"127.0.0.1","op":"UPDATE","id":"MyEntity:954619","changes":{"email":["root@example.com","root@example.org"]}}
>> {"ts":1441285381823,"by":"someotheruser","clientIP":"10.1.1.2","serverIP":"127.0.0.1","op":"DELETE","id":"MyOtherEntity:3279984:17","snapshot":{"favorite":true,"sequence":0}}
>>
>> We'd send them to Apache Kafka, so that other internal apps can process them sequentially at their leisure.
>>
>> Andrus
>>
>>> On Sep 23, 2015, at 10:55 AM, Hugi Thordarson <hu...@karlmenn.is> wrote:
>>>
>>> This looks really great! Would you mind sharing how the class A2PropertyChange looks?
>>>
>>> Cheers,
>>> - hugi
>>>
>>>
>>>
>>>> On 22. sep. 2015, at 19:57, Andrus Adamchik <an...@objectstyle.org> wrote:
>>>>
>>>> Here is the design of the audit framework for everyone's review:
>>>>
>>>> https://issues.apache.org/jira/browse/CAY-2030
>>>>
>>>> The way it will work from the user perspective is this:
>>>>
>>>> // bootstrap the filter by adding an extra Cayenne-provided module to
>>>> // runtime. Optionally add your own module that overrides context provider
>>>> // (e.g. to store request IP address and user name).
>>>> ServerRuntime r = ServerRuntimeBuilder.builder().addModule(new WhateverModuleNameWeUse()).build();
>>>>
>>>> // register listener
>>>> r.getDataDomain().addListener(new MyListener());
>>>>
>>>>
>>>> // listener code
>>>> class MyListener {
>>>>
>>>>
>>>> @PostCommit
>>>> void processAsChangeset(Map<ObjectId, ObjectChangeSet> map) {..}
>>>>
>>>> // optionally can take a changeset serialized to JSON
>>>> @PostCommit
>>>> void processAsJSON(Map<ObjectId, String> map) {..}
>>>> }
>>>>
>>>> Andrus
>>>
>>
>
Re: Change audit framework
Posted by Hugi Thordarson <hu...@karlmenn.is>.
Ah, of course… How do you handle a change to a to-many relationship in that scheme, do oldValue and newValue contain the entire object lists pre- and post-modification?
- hugi
> On 23. sep. 2015, at 08:09, Andrus Adamchik <an...@objectstyle.org> wrote:
>
> Just a pair of values:
>
> public class A2PropertyChange {
>
> private Object oldValue;
> private Object newValue;
>
> ...
> }
>
> and as I've shown earlier, my system would emit a stream of JSON messages like this:
>
> {"ts":1427090346831,"by":"someuser","clientIP":"10.1.1.1","serverIP":"127.0.0.1","op":"UPDATE","id":"MyEntity:954619","changes":{"email":["root@example.com","root@example.org"]}}
> {"ts":1441285381823,"by":"someotheruser","clientIP":"10.1.1.2","serverIP":"127.0.0.1","op":"DELETE","id":"MyOtherEntity:3279984:17","snapshot":{"favorite":true,"sequence":0}}
>
> We'd send them to Apache Kafka, so that other internal apps can process them sequentially at their leisure.
>
> Andrus
>
>> On Sep 23, 2015, at 10:55 AM, Hugi Thordarson <hu...@karlmenn.is> wrote:
>>
>> This looks really great! Would you mind sharing how the class A2PropertyChange looks?
>>
>> Cheers,
>> - hugi
>>
>>
>>
>>> On 22. sep. 2015, at 19:57, Andrus Adamchik <an...@objectstyle.org> wrote:
>>>
>>> Here is the design of the audit framework for everyone's review:
>>>
>>> https://issues.apache.org/jira/browse/CAY-2030
>>>
>>> The way it will work from the user perspective is this:
>>>
>>> // bootstrap the filter by adding an extra Cayenne-provided module to
>>> // runtime. Optionally add your own module that overrides context provider
>>> // (e.g. to store request IP address and user name).
>>> ServerRuntime r = ServerRuntimeBuilder.builder().addModule(new WhateverModuleNameWeUse()).build();
>>>
>>> // register listener
>>> r.getDataDomain().addListener(new MyListener());
>>>
>>>
>>> // listener code
>>> class MyListener {
>>>
>>>
>>> @PostCommit
>>> void processAsChangeset(Map<ObjectId, ObjectChangeSet> map) {..}
>>>
>>> // optionally can take a changeset serialized to JSON
>>> @PostCommit
>>> void processAsJSON(Map<ObjectId, String> map) {..}
>>> }
>>>
>>> Andrus
>>
>
Re: Change audit framework
Posted by Andrus Adamchik <an...@objectstyle.org>.
Correct, that was a typo :)
> On Sep 29, 2015, at 5:18 PM, Michael Gentry <mg...@masslight.net> wrote:
>
> Is L1.class supposed to be L.class?
>
> mrg
>
>
> On Tue, Sep 29, 2015 at 5:46 AM, Andrus Adamchik <an...@objectstyle.org>
> wrote:
>
>> I just sent a pull request with the first cut of the implementation. My
>> plan is to try this new implementation with my audit system and if it works
>> well, apply it to master (hopefully by then CAY-2028 will also be ready).
>> The implementation I ended up with does not have JSON serialization. Just
>> pure objects. Here is how it works:
>>
>> 1. Write a listener (currently requires an interface, but if needed we can
>> easily switch to annotations) :
>>
>> public class L implements PostCommitListener {
>>
>> @Override
>> public void onPostCommit(ObjectContext originatingContext, ChangeMap
>> changes) {
>> // do something with your changes
>> }
>> }
>>
>> 2. Configure Cayenne stack with "PostCommit" module:
>>
>> Module m = PostCommitModuleBuilder.builder().listener(L1.class);
>> ServerRuntime r = ServerRuntimeBuilder
>> .builder()
>> .addConfig("cayenne-lifecycle.xml")
>> .addModule(m)
>> .build();
>>
>>
>> PostCommitModuleBuilder supports a few non-default options, such as
>> excluding entities, entity properties, and hiding "confidential" property
>> values, such as passwords using the existing @Auditable annotation. Or you
>> can write a custom PostCommitEntityFactory to support your own annotations.
>>
>> TODO: there is not support for @AuditableChild yet. Need to think how to
>> better handle this one.
>>
>> Comments are welcomed.
>>
>> Andrus
Re: Change audit framework
Posted by Michael Gentry <mg...@masslight.net>.
Is L1.class supposed to be L.class?
mrg
On Tue, Sep 29, 2015 at 5:46 AM, Andrus Adamchik <an...@objectstyle.org>
wrote:
> I just sent a pull request with the first cut of the implementation. My
> plan is to try this new implementation with my audit system and if it works
> well, apply it to master (hopefully by then CAY-2028 will also be ready).
> The implementation I ended up with does not have JSON serialization. Just
> pure objects. Here is how it works:
>
> 1. Write a listener (currently requires an interface, but if needed we can
> easily switch to annotations) :
>
> public class L implements PostCommitListener {
>
> @Override
> public void onPostCommit(ObjectContext originatingContext, ChangeMap
> changes) {
> // do something with your changes
> }
> }
>
> 2. Configure Cayenne stack with "PostCommit" module:
>
> Module m = PostCommitModuleBuilder.builder().listener(L1.class);
> ServerRuntime r = ServerRuntimeBuilder
> .builder()
> .addConfig("cayenne-lifecycle.xml")
> .addModule(m)
> .build();
>
>
> PostCommitModuleBuilder supports a few non-default options, such as
> excluding entities, entity properties, and hiding "confidential" property
> values, such as passwords using the existing @Auditable annotation. Or you
> can write a custom PostCommitEntityFactory to support your own annotations.
>
> TODO: there is not support for @AuditableChild yet. Need to think how to
> better handle this one.
>
> Comments are welcomed.
>
> Andrus
Re: Change audit framework
Posted by Andrus Adamchik <an...@objectstyle.org>.
Thanks for the feedback :) I am glad that after a few years of iterations we finally nailed it.
Andrus
> On Feb 28, 2016, at 2:52 PM, Hugi Thordarson <hu...@karlmenn.is> wrote:
>
> Hi all.
>
> I’d just like to chime in to mention how ridiculously useful this feature has turned out to be. I’ve been using it since day one for audit logging, but I’ve since used it in many other places to react to changes in the object graph. Really, really nice.
>
> Cheers,
> - hugi
>
>
>
>
>> On 9. okt. 2015, at 19:49, Andrus Adamchik <an...@objectstyle.org> wrote:
>>
>> Just committed an updated version that supports to-many relationships. Altered the model of individual object changes to separately track changes to attributes, to-one and to-many relationships:
>>
>> https://github.com/apache/cayenne/blob/master/cayenne-lifecycle/src/main/java/org/apache/cayenne/lifecycle/changemap/ObjectChange.java
>>
>> Eventually we should probably reuse this model for internal Cayenne change tracking. It seems universal enough, will allow ObjectContext users to inspect changes prior to commit (which is doable now, but not easy), and will avoid multiple copies of "diffs".
>>
>> Andrus
>>
>>
>>> On Sep 29, 2015, at 5:46 AM, Andrus Adamchik <an...@objectstyle.org> wrote:
>>>
>>> I just sent a pull request with the first cut of the implementation. My plan is to try this new implementation with my audit system and if it works well, apply it to master (hopefully by then CAY-2028 will also be ready). The implementation I ended up with does not have JSON serialization. Just pure objects. Here is how it works:
>>>
>>> 1. Write a listener (currently requires an interface, but if needed we can easily switch to annotations) :
>>>
>>> public class L implements PostCommitListener {
>>>
>>> @Override
>>> public void onPostCommit(ObjectContext originatingContext, ChangeMap changes) {
>>> // do something with your changes
>>> }
>>> }
>>>
>>> 2. Configure Cayenne stack with "PostCommit" module:
>>>
>>> Module m = PostCommitModuleBuilder.builder().listener(L1.class);
>>> ServerRuntime r = ServerRuntimeBuilder
>>> .builder()
>>> .addConfig("cayenne-lifecycle.xml")
>>> .addModule(m)
>>> .build();
>>>
>>>
>>> PostCommitModuleBuilder supports a few non-default options, such as excluding entities, entity properties, and hiding "confidential" property values, such as passwords using the existing @Auditable annotation. Or you can write a custom PostCommitEntityFactory to support your own annotations.
>>>
>>> TODO: there is not support for @AuditableChild yet. Need to think how to better handle this one.
>>>
>>> Comments are welcomed.
>>>
>>> Andrus
>>
>
Re: Change audit framework
Posted by Hugi Thordarson <hu...@karlmenn.is>.
Hi all.
I’d just like to chime in to mention how ridiculously useful this feature has turned out to be. I’ve been using it since day one for audit logging, but I’ve since used it in many other places to react to changes in the object graph. Really, really nice.
Cheers,
- hugi
> On 9. okt. 2015, at 19:49, Andrus Adamchik <an...@objectstyle.org> wrote:
>
> Just committed an updated version that supports to-many relationships. Altered the model of individual object changes to separately track changes to attributes, to-one and to-many relationships:
>
> https://github.com/apache/cayenne/blob/master/cayenne-lifecycle/src/main/java/org/apache/cayenne/lifecycle/changemap/ObjectChange.java
>
> Eventually we should probably reuse this model for internal Cayenne change tracking. It seems universal enough, will allow ObjectContext users to inspect changes prior to commit (which is doable now, but not easy), and will avoid multiple copies of "diffs".
>
> Andrus
>
>
>> On Sep 29, 2015, at 5:46 AM, Andrus Adamchik <an...@objectstyle.org> wrote:
>>
>> I just sent a pull request with the first cut of the implementation. My plan is to try this new implementation with my audit system and if it works well, apply it to master (hopefully by then CAY-2028 will also be ready). The implementation I ended up with does not have JSON serialization. Just pure objects. Here is how it works:
>>
>> 1. Write a listener (currently requires an interface, but if needed we can easily switch to annotations) :
>>
>> public class L implements PostCommitListener {
>>
>> @Override
>> public void onPostCommit(ObjectContext originatingContext, ChangeMap changes) {
>> // do something with your changes
>> }
>> }
>>
>> 2. Configure Cayenne stack with "PostCommit" module:
>>
>> Module m = PostCommitModuleBuilder.builder().listener(L1.class);
>> ServerRuntime r = ServerRuntimeBuilder
>> .builder()
>> .addConfig("cayenne-lifecycle.xml")
>> .addModule(m)
>> .build();
>>
>>
>> PostCommitModuleBuilder supports a few non-default options, such as excluding entities, entity properties, and hiding "confidential" property values, such as passwords using the existing @Auditable annotation. Or you can write a custom PostCommitEntityFactory to support your own annotations.
>>
>> TODO: there is not support for @AuditableChild yet. Need to think how to better handle this one.
>>
>> Comments are welcomed.
>>
>> Andrus
>
Re: Change audit framework
Posted by Andrus Adamchik <an...@objectstyle.org>.
Just committed an updated version that supports to-many relationships. Altered the model of individual object changes to separately track changes to attributes, to-one and to-many relationships:
https://github.com/apache/cayenne/blob/master/cayenne-lifecycle/src/main/java/org/apache/cayenne/lifecycle/changemap/ObjectChange.java
Eventually we should probably reuse this model for internal Cayenne change tracking. It seems universal enough, will allow ObjectContext users to inspect changes prior to commit (which is doable now, but not easy), and will avoid multiple copies of "diffs".
Andrus
> On Sep 29, 2015, at 5:46 AM, Andrus Adamchik <an...@objectstyle.org> wrote:
>
> I just sent a pull request with the first cut of the implementation. My plan is to try this new implementation with my audit system and if it works well, apply it to master (hopefully by then CAY-2028 will also be ready). The implementation I ended up with does not have JSON serialization. Just pure objects. Here is how it works:
>
> 1. Write a listener (currently requires an interface, but if needed we can easily switch to annotations) :
>
> public class L implements PostCommitListener {
>
> @Override
> public void onPostCommit(ObjectContext originatingContext, ChangeMap changes) {
> // do something with your changes
> }
> }
>
> 2. Configure Cayenne stack with "PostCommit" module:
>
> Module m = PostCommitModuleBuilder.builder().listener(L1.class);
> ServerRuntime r = ServerRuntimeBuilder
> .builder()
> .addConfig("cayenne-lifecycle.xml")
> .addModule(m)
> .build();
>
>
> PostCommitModuleBuilder supports a few non-default options, such as excluding entities, entity properties, and hiding "confidential" property values, such as passwords using the existing @Auditable annotation. Or you can write a custom PostCommitEntityFactory to support your own annotations.
>
> TODO: there is not support for @AuditableChild yet. Need to think how to better handle this one.
>
> Comments are welcomed.
>
> Andrus
Re: Change audit framework
Posted by Hugi Thordarson <hu...@karlmenn.is>.
Looks great, can't wait to try it out :)
- hugi
> On 29. sep. 2015, at 09:46, Andrus Adamchik <an...@objectstyle.org> wrote:
>
> I just sent a pull request with the first cut of the implementation. My plan is to try this new implementation with my audit system and if it works well, apply it to master (hopefully by then CAY-2028 will also be ready). The implementation I ended up with does not have JSON serialization. Just pure objects. Here is how it works:
>
> 1. Write a listener (currently requires an interface, but if needed we can easily switch to annotations) :
>
> public class L implements PostCommitListener {
>
> @Override
> public void onPostCommit(ObjectContext originatingContext, ChangeMap changes) {
> // do something with your changes
> }
> }
>
> 2. Configure Cayenne stack with "PostCommit" module:
>
> Module m = PostCommitModuleBuilder.builder().listener(L1.class);
> ServerRuntime r = ServerRuntimeBuilder
> .builder()
> .addConfig("cayenne-lifecycle.xml")
> .addModule(m)
> .build();
>
>
> PostCommitModuleBuilder supports a few non-default options, such as excluding entities, entity properties, and hiding "confidential" property values, such as passwords using the existing @Auditable annotation. Or you can write a custom PostCommitEntityFactory to support your own annotations.
>
> TODO: there is not support for @AuditableChild yet. Need to think how to better handle this one.
>
> Comments are welcomed.
>
> Andrus
Re: Change audit framework
Posted by Andrus Adamchik <an...@objectstyle.org>.
I just sent a pull request with the first cut of the implementation. My plan is to try this new implementation with my audit system and if it works well, apply it to master (hopefully by then CAY-2028 will also be ready). The implementation I ended up with does not have JSON serialization. Just pure objects. Here is how it works:
1. Write a listener (currently requires an interface, but if needed we can easily switch to annotations) :
public class L implements PostCommitListener {
@Override
public void onPostCommit(ObjectContext originatingContext, ChangeMap changes) {
// do something with your changes
}
}
2. Configure Cayenne stack with "PostCommit" module:
Module m = PostCommitModuleBuilder.builder().listener(L1.class);
ServerRuntime r = ServerRuntimeBuilder
.builder()
.addConfig("cayenne-lifecycle.xml")
.addModule(m)
.build();
PostCommitModuleBuilder supports a few non-default options, such as excluding entities, entity properties, and hiding "confidential" property values, such as passwords using the existing @Auditable annotation. Or you can write a custom PostCommitEntityFactory to support your own annotations.
TODO: there is not support for @AuditableChild yet. Need to think how to better handle this one.
Comments are welcomed.
Andrus
Re: Change audit framework
Posted by Andrus Adamchik <an...@objectstyle.org>.
> On Sep 23, 2015, at 1:20 PM, Aristedes Maniatis <ar...@maniatis.org> wrote:
>
> * fail fast and don't stop the main Cayenne processing thread. That is, if the message broker/other database fails to return quickly, don't prevent the commit returning control to the application. Possibly even try to attempt the storage in a separate thread.
>
> * if the audit message fails (say your message broker is dead), then raise an error and try to roll back the transaction. I'm guessing it is too late for Cayenne to help with that.
I think multi-threading and error handling concerns are outside of Cayenne concerns. We'll dispatch these events synchronously, letting the receiver do anything fancy beyond that on their own.
E.g. in my code, Kafka access is abstracted as a "log queue service", swappable via DI to sending output to STDOUT.
Andrus
Re: Change audit framework
Posted by Aristedes Maniatis <ar...@maniatis.org>.
On 23/09/2015 6:09pm, Andrus Adamchik wrote:
> We'd send them to Apache Kafka, so that other internal apps can process them sequentially at their leisure.
Some random thoughts that this raised in my head...
I guess many people are pushing this to:
1. A database in a json field, so that works except that you might want one json object per commit
2. A database with a real schema (eg. some sort of polymorphic joins to all other tables in your main schema). This could give you an in-application real view of changes, per record.
3. elasticsearch (it has a restful json http interface and also a native Java client)
4. One of the thousands of noSQL key-value databases out there
5. A log file to disk
6. syslog server?
Some message broker like ActiveMQ, JMS, Kafka, RabbitMQ, log4jv2 etc could be in the middle, but that's just really a buffer to get the data to somewhere else, like one of the options above.
People might have very different needs around failure modes too...
* fail fast and don't stop the main Cayenne processing thread. That is, if the message broker/other database fails to return quickly, don't prevent the commit returning control to the application. Possibly even try to attempt the storage in a separate thread.
* if the audit message fails (say your message broker is dead), then raise an error and try to roll back the transaction. I'm guessing it is too late for Cayenne to help with that.
Ari
--
-------------------------->
Aristedes Maniatis
GPG fingerprint CBFB 84B4 738D 4E87 5E5C 5EFA EF6A 7D2E 3E49 102A
Re: Change audit framework
Posted by Andrus Adamchik <an...@objectstyle.org>.
Just a pair of values:
public class A2PropertyChange {
private Object oldValue;
private Object newValue;
...
}
and as I've shown earlier, my system would emit a stream of JSON messages like this:
{"ts":1427090346831,"by":"someuser","clientIP":"10.1.1.1","serverIP":"127.0.0.1","op":"UPDATE","id":"MyEntity:954619","changes":{"email":["root@example.com","root@example.org"]}}
{"ts":1441285381823,"by":"someotheruser","clientIP":"10.1.1.2","serverIP":"127.0.0.1","op":"DELETE","id":"MyOtherEntity:3279984:17","snapshot":{"favorite":true,"sequence":0}}
We'd send them to Apache Kafka, so that other internal apps can process them sequentially at their leisure.
Andrus
> On Sep 23, 2015, at 10:55 AM, Hugi Thordarson <hu...@karlmenn.is> wrote:
>
> This looks really great! Would you mind sharing how the class A2PropertyChange looks?
>
> Cheers,
> - hugi
>
>
>
>> On 22. sep. 2015, at 19:57, Andrus Adamchik <an...@objectstyle.org> wrote:
>>
>> Here is the design of the audit framework for everyone's review:
>>
>> https://issues.apache.org/jira/browse/CAY-2030
>>
>> The way it will work from the user perspective is this:
>>
>> // bootstrap the filter by adding an extra Cayenne-provided module to
>> // runtime. Optionally add your own module that overrides context provider
>> // (e.g. to store request IP address and user name).
>> ServerRuntime r = ServerRuntimeBuilder.builder().addModule(new WhateverModuleNameWeUse()).build();
>>
>> // register listener
>> r.getDataDomain().addListener(new MyListener());
>>
>>
>> // listener code
>> class MyListener {
>>
>>
>> @PostCommit
>> void processAsChangeset(Map<ObjectId, ObjectChangeSet> map) {..}
>>
>> // optionally can take a changeset serialized to JSON
>> @PostCommit
>> void processAsJSON(Map<ObjectId, String> map) {..}
>> }
>>
>> Andrus
>
Re: Change audit framework
Posted by Hugi Thordarson <hu...@karlmenn.is>.
This looks really great! Would you mind sharing how the class A2PropertyChange looks?
Cheers,
- hugi
> On 22. sep. 2015, at 19:57, Andrus Adamchik <an...@objectstyle.org> wrote:
>
> Here is the design of the audit framework for everyone's review:
>
> https://issues.apache.org/jira/browse/CAY-2030
>
> The way it will work from the user perspective is this:
>
> // bootstrap the filter by adding an extra Cayenne-provided module to
> // runtime. Optionally add your own module that overrides context provider
> // (e.g. to store request IP address and user name).
> ServerRuntime r = ServerRuntimeBuilder.builder().addModule(new WhateverModuleNameWeUse()).build();
>
> // register listener
> r.getDataDomain().addListener(new MyListener());
>
>
> // listener code
> class MyListener {
>
>
> @PostCommit
> void processAsChangeset(Map<ObjectId, ObjectChangeSet> map) {..}
>
> // optionally can take a changeset serialized to JSON
> @PostCommit
> void processAsJSON(Map<ObjectId, String> map) {..}
> }
>
> Andrus