You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Ron Anderson <bi...@yahoo.com> on 2013/02/13 20:01:02 UTC
MongoDB Jackson Date Mapping Option
The mongodb component is using the Jackson library for object mapping and has
mapping of dates defaulted to Unix timestamps rather than ISODate. This
causes some problems with using the Aggregation Framework and makes queries
more difficult to view dates.
There is an option to set this default behavior using the following:
objectMapper.configure(SerializationConfig.Feature.WRITE_DATES_AS_TIMESTAMPS,
false);
Is there a way to get access to the Jackson ObjectMapper to set this option
during set up of the camel context etc?
thanks,
Ron
--
View this message in context: http://camel.465427.n5.nabble.com/MongoDB-Jackson-Date-Mapping-Option-tp5727548.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: MongoDB Jackson Date Mapping Option
Posted by gramanero <gr...@gmail.com>.
I am wondering if there is a good option for getting ISODates or any of the
other built-in MongoDB types to work within the context of Spring. For
example, given a very simple document:
/{ "time" : ISODate("2016-02-19T17:37:57.673+0000") }/
How might one get the ISODate to properly work with camel-mongo? I have been
round and round with variations with no success. It appears according to the
camel-mongo docs that I can send in a string and that string will be
converted to a DBObject, but that does not appear to work with built-in
types unless I am missing something obvious.
/<convertBodyTo type="String" />
<to
uri="mongodb:mongoBean?database=PMA&collection=Elmah&writeResultAsHeader=true&operation=insert"
/>/
*Error received:*
/org.apache.camel.RuntimeCamelException:
org.apache.camel.component.mongodb.CamelMongoDbException: MongoDB operation
= insert, Body is not conversible to type DBObject nor List<DBObject>/
I have also attempted to convert to a string and then unmarshall using the
Jackson parser, but the parser chokes on the ISODate as I suspect that it
really does not understand the mongo built-in types.
We do not code in Java and really do everything via Spring DSL. I can likely
manipulate the JSON prior to sending on the exchange to camel-mongo, but
still not sure what that manipulation might look like.
I attempted to change it to this based on my very loose understanding that
this was a possible solution:
/{ "time" : { "$date" : "2016-02-19T17:37:57.673+0000" }}/
But that still did not work as I got the same error as mentioned above. I
could try to get the Object --> DBObject conversion to work, but honestly I
am not sure how to get the JSON string into the form of an Object within
Spring before passing on the exchange to camel-mongo.
Any advice / suggestions are VERY much appreciated.
Thank you!
--
View this message in context: http://camel.465427.n5.nabble.com/MongoDB-Jackson-Date-Mapping-Option-tp5727548p5778438.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: MongoDB Jackson Date Mapping Option
Posted by Ron Anderson <bi...@yahoo.com>.
Thanks so much for your help - For anyone else trying to solve this I'll show
what I ended up doing that worked to map a Java date to a Bson date or
ISODate in MongoDB.
Created a custom serializer class:
public class BsonDateSerializer extends JsonSerializer<Date>{
@Override
public void serialize(Date value, JsonGenerator jgen, SerializerProvider
provider) throws IOException, JsonProcessingException {
jgen.writeStartObject();
serializeContents(value, jgen, provider);
jgen.writeEndObject();
}
private void serializeContents(Date value, JsonGenerator jgen,
SerializerProvider provider) throws IOException {
jgen.writeFieldName("$date");
SimpleDateFormat formatter = new
SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'");
String formattedDate = formatter.format(value);
jgen.writeString(formattedDate);
}
}
Annotated my pojo to use custom serializer:
@JsonSerialize(using = BsonDateSerializer.class)
public Date getStartTime() {
return startTime;
}
And the Mongo route was already working but repeated here:
from("activemq:queue:incomingRecords")
.routeId("record_processor")
.marshal(jackson)
.convertBodyTo(String.class)
.to("mongodb:mongoConn?database=usage&collection=lteusage&operation=insert")
.end();
--
View this message in context: http://camel.465427.n5.nabble.com/MongoDB-Jackson-Date-Mapping-Option-tp5727548p5727671.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: MongoDB Jackson Date Mapping Option
Posted by Raul Kripalani <ra...@evosent.com>.
Hi Ron,
This is a very good question. It's more a MongoDB-specific question rather
than a Camel question, but here's what I know of the topic.
Take into account that the JSON spec doesn't define a standard
date/dateTime format (unlike XSD), hence there's no universal way to
recognise a JSON value as a date. Moreover, MongoDB extends JSON with a
number of custom types, amongst which is the ISODate type.
It looks like the MongoDB JSON.parse(s) method interprets keys with the
special operator "$date" as dates. See [1]. You could modify your JSON data
model classes and add a few @Annotations so that Jackson already spits out
a $date field.
If you don't want to pollute your data model with MongoDB specific quirks,
you can post-process the "normal" JSON in a number of ways.
1) Manipulate at the BasicDBObject level, e.g.:
.process(new Processor() {
public void process(Exchange exchange) {
BasicDBObject bdbo = (BasicDBObject)
exchange.getIn().getBody(DBObject.class);
// do the DBO manipulation, turn the date into an ISODate
bdbo.put("date", new Date(...));
}
});
2) Massage at the JSON level, by turning the Jackson-generated JSON into a
HashMap using ObjectMapper, and adding an intermediate key "$date", for
MongoDB's JSON parser to recognise the field as a date.
[1] https://jira.mongodb.org/browse/JAVA-565.
*Raúl Kripalani*
Apache Camel Committer
Enterprise Architect, Program Manager, Open Source Integration specialist
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk <http://twitter.com/raulvk>
On Thu, Feb 14, 2013 at 11:26 PM, Ron Anderson <bi...@yahoo.com> wrote:
> Well it almost solved it. It cured my exception and is now persisting the
> date in the database but it is storing it as a BSON string so still not
> recognized as an ISODate or BSON date type so no date functions yet work.
>
> Any way for the JSON date representation to get converted to BSON date
> type?
>
> thanks again,
>
> Ron
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/MongoDB-Jackson-Date-Mapping-Option-tp5727548p5727633.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Re: MongoDB Jackson Date Mapping Option
Posted by Ron Anderson <bi...@yahoo.com>.
Well it almost solved it. It cured my exception and is now persisting the
date in the database but it is storing it as a BSON string so still not
recognized as an ISODate or BSON date type so no date functions yet work.
Any way for the JSON date representation to get converted to BSON date type?
thanks again,
Ron
--
View this message in context: http://camel.465427.n5.nabble.com/MongoDB-Jackson-Date-Mapping-Option-tp5727548p5727633.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: MongoDB Jackson Date Mapping Option
Posted by Ron Anderson <bi...@yahoo.com>.
That solved it. Dates are now showing up as ISODates.
Thanks so much for your help.
--
View this message in context: http://camel.465427.n5.nabble.com/MongoDB-Jackson-Date-Mapping-Option-tp5727548p5727628.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: MongoDB Jackson Date Mapping Option
Posted by Raul Kripalani <ra...@evosent.com>.
Try adding a .convertBodyTo(String.class) just after the .marshal(jackson)
and before the MongoDB endpoint.
Regards,
*Raúl Kripalani*
Apache Camel Committer
Enterprise Architect, Program Manager, Open Source Integration specialist
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk <http://twitter.com/raulvk>
On Thu, Feb 14, 2013 at 6:01 PM, Ron Anderson <bi...@yahoo.com> wrote:
> Hi Raul,
>
> Thanks for your helpful suggestion. I'm having an issue though that I
> can't
> seem to figure out and wondering if you have any ideas.
>
> Trying your suggestion I've set up the DataFormat as:
> ObjectMapper objectMapper = new ObjectMapper();
>
> objectMapper.disable(SerializationConfig.Feature.WRITE_DATES_AS_TIMESTAMPS);
> JacksonDataFormat jackson = new JacksonDataFormat(objectMapper, null);
>
> And applied it to the marshaller as:
>
> from("activemq:queue:incomingRecords")
> .routeId("record_processor")
> .marshal(jackson)
>
> .to("mongodb:mongoConn?database=usage&collection=lteusage&operation=insert")
> .end();
>
> But I'm getting the following exception that does not occur when I remove
> the marshaller and just let mongodb component do the marshalling.
>
> Hoping you might have some ideas as to what might be causing this. My pojo
> is very simple Strings, Integers and Date.
>
> Caused by: org.apache.camel.NoTypeConversionAvailableException: No type
> converter available to convert from type: java.lang.Byte to the required
> type: com.mongodb.DBObject with value 110
>
> at
>
> org.apache.camel.impl.converter.BaseTypeConverterRegistry.mandatoryConvertTo(BaseTypeConverterRegistry.java:169)
> at
>
> org.apache.camel.impl.converter.BaseTypeConverterRegistry.mandatoryConvertTo(BaseTypeConverterRegistry.java:142)
> at
>
> org.apache.camel.component.mongodb.MongoDbProducer.attemptConvertToList(MongoDbProducer.java:419)
> ... 49 more
> 2013-02-14 09:51:56,531 WARN ~ Conversion has fallen back to generic
> Object
> -> DBObject, but unable to convert type
> org.apache.camel.component.jms.JmsMessage. Returning null.
>
> org.apache.camel.component.mongodb.CamelMongoDbException: MongoDB operation
> = insert, Assuming List variant of MongoDB insert operation, but List
> contains non-DBObject items
> org.apache.camel.component.mongodb.CamelMongoDbException: MongoDB operation
> = insert, Assuming List variant of MongoDB insert operation, but List
> contains non-DBObject items
> at
>
> org.apache.camel.component.mongodb.MongoDbProducer.attemptConvertToList(MongoDbProducer.java:422)
> at
>
> org.apache.camel.component.mongodb.MongoDbProducer.doInsert(MongoDbProducer.java:239)
> at
>
> org.apache.camel.component.mongodb.MongoDbProducer.invokeOperation(MongoDbProducer.java:102)
> at
>
> org.apache.camel.component.mongodb.MongoDbProducer.process(MongoDbProducer.java:70)
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/MongoDB-Jackson-Date-Mapping-Option-tp5727548p5727625.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Re: MongoDB Jackson Date Mapping Option
Posted by Ron Anderson <bi...@yahoo.com>.
Hi Raul,
Thanks for your helpful suggestion. I'm having an issue though that I can't
seem to figure out and wondering if you have any ideas.
Trying your suggestion I've set up the DataFormat as:
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.disable(SerializationConfig.Feature.WRITE_DATES_AS_TIMESTAMPS);
JacksonDataFormat jackson = new JacksonDataFormat(objectMapper, null);
And applied it to the marshaller as:
from("activemq:queue:incomingRecords")
.routeId("record_processor")
.marshal(jackson)
.to("mongodb:mongoConn?database=usage&collection=lteusage&operation=insert")
.end();
But I'm getting the following exception that does not occur when I remove
the marshaller and just let mongodb component do the marshalling.
Hoping you might have some ideas as to what might be causing this. My pojo
is very simple Strings, Integers and Date.
Caused by: org.apache.camel.NoTypeConversionAvailableException: No type
converter available to convert from type: java.lang.Byte to the required
type: com.mongodb.DBObject with value 110
at
org.apache.camel.impl.converter.BaseTypeConverterRegistry.mandatoryConvertTo(BaseTypeConverterRegistry.java:169)
at
org.apache.camel.impl.converter.BaseTypeConverterRegistry.mandatoryConvertTo(BaseTypeConverterRegistry.java:142)
at
org.apache.camel.component.mongodb.MongoDbProducer.attemptConvertToList(MongoDbProducer.java:419)
... 49 more
2013-02-14 09:51:56,531 WARN ~ Conversion has fallen back to generic Object
-> DBObject, but unable to convert type
org.apache.camel.component.jms.JmsMessage. Returning null.
org.apache.camel.component.mongodb.CamelMongoDbException: MongoDB operation
= insert, Assuming List variant of MongoDB insert operation, but List
contains non-DBObject items
org.apache.camel.component.mongodb.CamelMongoDbException: MongoDB operation
= insert, Assuming List variant of MongoDB insert operation, but List
contains non-DBObject items
at
org.apache.camel.component.mongodb.MongoDbProducer.attemptConvertToList(MongoDbProducer.java:422)
at
org.apache.camel.component.mongodb.MongoDbProducer.doInsert(MongoDbProducer.java:239)
at
org.apache.camel.component.mongodb.MongoDbProducer.invokeOperation(MongoDbProducer.java:102)
at
org.apache.camel.component.mongodb.MongoDbProducer.process(MongoDbProducer.java:70)
--
View this message in context: http://camel.465427.n5.nabble.com/MongoDB-Jackson-Date-Mapping-Option-tp5727548p5727625.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: MongoDB Jackson Date Mapping Option
Posted by Raul Kripalani <ra...@evosent.com>.
Hi Ron,
Not at the MongoDB component itself. We just provide this Object => JSON
conversion for convenience, not as a fully-fledged, flexible transformation
path.
That said, Camel provides a camel-jackson data format which is indeed more
flexible and supports injecting a custom ObjectMapper with your
configuration. So before sending to your MongoDB endpoint, you could ask
Camel to perform the JSON marshalling using the marshal() DSL. For example:
JacksonDataFormat jackson = new JacksonDataFormat(objectMapper, null); //
use 'null' here as we won't be performing any unmarshalling with this data
format, otherwise indicate the unmarshal type, or HashMap.class otherwise
from("direct:foo").marshal(jackson).to("mongodb:...");
The MongoDB endpoint will detect the input to already be JSON and will
bypass the Object => JSON convenience conversion.
Hope that helps.
P.S.: Sorry for not replying to your private message earlier - today has
been hectic!
Regards,
*Raúl Kripalani*
Apache Camel Committer
Enterprise Architect, Program Manager, Open Source Integration specialist
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
http://blog.raulkr.net | twitter: @raulvk <http://twitter.com/raulvk>
On Wed, Feb 13, 2013 at 7:01 PM, Ron Anderson <bi...@yahoo.com> wrote:
> The mongodb component is using the Jackson library for object mapping and
> has
> mapping of dates defaulted to Unix timestamps rather than ISODate. This
> causes some problems with using the Aggregation Framework and makes queries
> more difficult to view dates.
>
> There is an option to set this default behavior using the following:
>
>
> objectMapper.configure(SerializationConfig.Feature.WRITE_DATES_AS_TIMESTAMPS,
> false);
>
> Is there a way to get access to the Jackson ObjectMapper to set this option
> during set up of the camel context etc?
>
> thanks,
>
> Ron
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/MongoDB-Jackson-Date-Mapping-Option-tp5727548.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>