You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by Royamit <am...@gmail.com> on 2016/02/26 09:03:17 UTC
How to insert Date String as date in MongoDB
Hi all,
I am trying out a sample in which I have used Apache camel component for
MongoDB.
Below is my route:
//Below line queries MongoDB to get the result
from("jetty:http://localhost:8181/dataByQuery")
.to("mongodb:myDb?database=DataDB&collection=data&operation=findOneByQuery");
//Below line inserts data to mongoDB
from("jetty:http://localhost:8181/dataInsert")
//.convertBodyTo(ListJacksonDataFormat.class)
//.convertBodyTo(String.class)
//.marshal(jackson)
//.convertBodyTo(String.class)
.convertBodyTo(DBObject.class)
.to("mongodb:myDb?database=DataDB&collection=data&operation=insert");
Now I am invoking the insert Operation i.e.http://localhost:8181/dataInsert
URL from saopUI and passing
Body as:
{
"DataID": "19dd8a2e-45cb-40cc-ba2c-15d5a81733f8",
"DataDescription": "Data High",
"DataPriority": "High",
"CreatedDateTimeUTC": "2015-01-12T16:39:42.132Z"
}
The above payload gets inserted into MongoDB but the CreatedDateTimeUTC gets
inserted as String instead of Date.
I have tried MongoDB sample through java code and am able to insert Date as
ISODate. But when I am trying to insert data using camel . I am not able to
do so.
So my question is how can I insert CreatedDateTimeUTC as Date Object in
MongoDB.
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-insert-Date-String-as-date-in-MongoDB-tp5778310.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to insert Date String as date in MongoDB
Posted by gramanero <gr...@gmail.com>.
I did a little more playing around. Discovered and verified a few different
things. First, I recall reading somewhere that the type mapping is being
cached (I think I read that). So when I played around with the JSON payload
being sent in I started to get mixed results and what was previously not
throwing any errors was now throwing a conversion error. I reloaded the
bundle, went back to the JSON that we agreed upon, and no more errors. So I
suspect that caching of the type conversion is happening somewhere (just a
hunch based on my observations though).
More importantly, when I changed the JSON to this:
/{ "time" : { "$date" : "2016-02-19T17:37:57.673Z" } }/
replacing the /+0000/ offset with /Z/, the document actually persisted
correctly as a DateTime type. Very cool. Now I need to figure out how to
manipulate the JSON in Spring DSL from /ISODate(...)/ (which is how the
callers will be sending the payload in to the route) to /{ "$date" : "...Z"
}/ so that I can put this piece to rest.
Does any of that make sense to you?
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-insert-Date-String-as-date-in-MongoDB-tp5778310p5779103.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to insert Date String as date in MongoDB
Posted by gramanero <gr...@gmail.com>.
Thanks for confirming. I tested the logic and while there are no errors, the
value of the /time /property is being persisted as /null /in Mongo. I am
going to take a look at the converter classes to see if I can figure out how
that is possible. Any thoughts?
Again, thanks for the help on this!
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-insert-Date-String-as-date-in-MongoDB-tp5778310p5779102.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to insert Date String as date in MongoDB
Posted by Raul Kripalani <ra...@apache.org>.
Yeah. Converting the body to a String is a good way to guarantee that the
conversion logic we're interested in will kick in.
Just for reference, have a look at the MongoDbConverters class.
Cheers,
Raúl.
On 14 Mar 2016 18:52, "gramanero" <gr...@gmail.com> wrote:
> Yes I can try that. To make sure I am on the same page is this what you are
> asking for?
>
> *Route:*
> /<convertBodyTo type="String" />
> <to
>
> uri="mongodb:mongoBean?database=PMA&collection=Elmah&writeResultAsHeader=true&operation=insert"
> />/
>
> *JSON to send:*
> /{ "time" : { "$date" : "2016-02-19T17:37:57.673+0000" } }/
>
>
> Or do think we should not perform the convertBodyTo string?
>
> Thank you for the quick response! I seriously appreciate the attention.
>
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/How-to-insert-Date-String-as-date-in-MongoDB-tp5778310p5779043.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Re: How to insert Date String as date in MongoDB
Posted by gramanero <gr...@gmail.com>.
Yes I can try that. To make sure I am on the same page is this what you are
asking for?
*Route:*
/<convertBodyTo type="String" />
<to
uri="mongodb:mongoBean?database=PMA&collection=Elmah&writeResultAsHeader=true&operation=insert"
/>/
*JSON to send:*
/{ "time" : { "$date" : "2016-02-19T17:37:57.673+0000" } }/
Or do think we should not perform the convertBodyTo string?
Thank you for the quick response! I seriously appreciate the attention.
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-insert-Date-String-as-date-in-MongoDB-tp5778310p5779043.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to insert Date String as date in MongoDB
Posted by Raul Kripalani <ra...@apache.org>.
Jackson is not MongoDB-specific and therefore doesn't recognise $date types.
The camel-mongodb component uses MongoDB's JSON class to parse Strings,
which does recognise Mongo-specific types:
http://api.mongodb.org/java/2.6.5/com/mongodb/util/JSON.html.
Can you try sending the JSON string directly to the endpoint, with no
Jackson marshal/unmarshalling?
Cheers,
*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
Messaging Engineer
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
Blog: raul.io | twitter: @raulvk <https://twitter.com/raulvk>
On Fri, Mar 4, 2016 at 9:43 PM, gramanero <gr...@gmail.com> wrote:
> Thank you for the response and information. I believe I tried what is
> recommended in the link that you provided. Below is a copy from the Logs
> via
> the Karaf console. I hope the formatting is not too terrible. I can try and
> provide it another way if necessary.
>
> Pretty much the same result as I was seeing before as I believe I already
> tried this approach. Here is the document that I used: /{ "time" : {
> "$date"
> : "2016-02-19T17:37:57.673+0000" } }/
>
> Here is what is going on in the route (the reason for the tracing is that I
> wanted to see what type the body was being converted to and from...I was
> toying around with different conversions to see if I could come up with
> some
> magic incantation that would work):
> / <convertBodyTo type="String" />
> <log message="[TRACE] Before unmarshall - ${body}"/>
> <unmarshal ref="jsonMarshaller" />
> <log message="[TRACE] After unmarshall - ${body}"/>
> <to
>
> uri="mongodb:mongoBean?database=PMA&collection=Elmah&writeResultAsHeader=true&operation=insert"
> />/
>
> And the jsonMarshaller dataformat definition is this:
> / <dataFormats>
> <json id="jsonMarshaller" library="Jackson"
> disableFeatures="FAIL_ON_UNKNOWN_PROPERTIES,WRITE_DATES_AS_TIMESTAMPS"/>
> </dataFormats>
> /
>
> And the Mongo bean definition:
> / <bean id="mongoBean" class="com.mongodb.Mongo">
> <constructor-arg index="0">
> <bean class="com.mongodb.MongoURI">
> <constructor-arg index="0"
>
> value="mongodb://DEVVLMONGORPLLOGS00.oecapplications.dev:27018/?replicaSet=rsLOGS&connectTimeoutMS=30000&wtimeoutMS=30000&w=1&journal=true"
> />
> </bean>
> </constructor-arg>
> </bean>/
>
>
> Hope this information is helpful. I certainly appreciate someone looking
> into this. We are not Java developers here, but at the moment it feels like
> my next course of action is to write some Java code to customize the
> serializer.
>
> *From the Logs available via the Karaf Console:*
> *3/4/2016, 4:29:33 PM WARN* /Execution of JMS message listener failed.
> Caused by: [org.apache.camel.RuntimeCamelException -
> org.apache.camel.component.mongodb.CamelMongoDbException: MongoDB operation
> = insert, Body is not conversible to type DBObject nor List<DBObject>]
> camel-core org.apache.camel.RuntimeCamelException:
> org.apache.camel.component.mongodb.CamelMongoDbException: MongoDB operation
> = insert, Body is not conversible to type DBObject nor List<DBObject>/
>
> *3/4/2016, 4:29:33 PM ERROR* /Failed delivery for (MessageId:
> ID:devvlapifuse01-42307-1456773862280-9:8:3:1:1 on ExchangeId:
> ID-devvlapifuse01-49834-1456773865165-1-9). Exhausted after delivery
> attempt: 1 caught:
> org.apache.camel.component.mongodb.CamelMongoDbException:
> MongoDB operation = insert, Body is not conversible to type DBObject nor
> List<DBObject> Message History
>
> ---------------------------------------------------------------------------------------------------------------------------------------
> RouteId ProcessorId Processor Elapsed (ms) [oec.common.logger.]
> [oec.common.logger.] [activemq://queue:oec.common.logger.elmah.request ] [
> 2] [oec.common.logger.] [validate4 ]
> [validate[simple{${in.header.MongoDatabase}}] ] [ 0] [oec.common.logger.]
> [validate5 ] [validate[simple{${in.header.MongoCollection}}] ] [ 0]
> [oec.common.logger.] [log2 ] [log ] [ 0] [oec.common.logger.]
> [convertBodyTo1 ] [convertBodyTo[String] ] [ 0] [oec.common.logger.] [log3
> ]
> [log ] [ 1] [oec.common.logger.] [log4 ] [log ] [ 0] [oec.common.logger.]
> [to1 ]
>
> [mongodb:mongoBean?database=PMA&collection=Elmah&writeResultAsHeader=true&opera]
> [ 1] Exchange
>
> ---------------------------------------------------------------------------------------------------------------------------------------
> Exchange[ Id ID-devvlapifuse01-49834-1456773865165-1-9 ExchangePattern
> InOnly Headers {Accept=*/*, Accept-Encoding=gzip, deflate,
> Accept-Language=en-US,en;q=0.8, AppKey=NfQFDRu7Ogq4xF4hZpMGV0CxdgQCnDPW,
> breadcrumbId=ID-devvlapifuse01-49834-1456773865165-1-8,
> CamelRedelivered=false, CamelRedeliveryCounter=0,
> CamelServletContextPath=/private/common/logger/elmah, Content-Length=59,
> Content-Type=application/json, JMSCorrelationID=null,
> JMSCorrelationIDAsBytes=null, JMSDeliveryMode=2,
> JMSDestination=queue://oec.common.logger.elmah.request, JMSExpiration=0,
> JMSMessageID=ID:devvlapifuse01-42307-1456773862280-9:8:3:1:1,
> JMSPriority=4,
> JMSRedelivered=false, JMSReplyTo=null, JMSTimestamp=1457126973022,
> JMSType=null, JMSXGroupID=null, JMSXUserID=null, MongoCollection=Elmah,
> MongoDatabase=PMA,
> Origin=chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop,
> Postman-Token=fc4ab341-5fbb-fb41-d0ad-c30cc7c3c30a, TestType=2} BodyType
> String Body { "time" : { "$date" : "2016-02-19T17:37:57.673+0000" } }
> ] Stacktrace
>
> ---------------------------------------------------------------------------------------------------------------------------------------
> camel-core org.apache.camel.component.mongodb.CamelMongoDbException:
> MongoDB
> operation = insert, Body is not conversible to type DBObject nor
> List<DBObject>/
>
>
> *3/4/2016, 4:29:33 PM WARN* /Conversion has fallen back to generic
> Object ->
> DBObject, but unable to convert type org.apache.camel.impl.DefaultMessage.
> Returning null. camel-mongodb/
>
>
> *3/4/2016, 4:29:33 PM INFO* /[TRACE] After unmarshall - { "time" : {
> "$date"
> : "2016-02-19T17:37:57.673+0000" } }
> camel-core /
>
>
> *3/4/2016, 4:29:33 PM INFO* /[TRACE] Before unmarshall - { "time" : {
> "$date" : "2016-02-19T17:37:57.673+0000" } }
> camel-core /
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/How-to-insert-Date-String-as-date-in-MongoDB-tp5778310p5778652.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Re: How to insert Date String as date in MongoDB
Posted by gramanero <gr...@gmail.com>.
Any thoughts on my last post as to how we can use ISODates with camel-mongo?
Thanks!
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-insert-Date-String-as-date-in-MongoDB-tp5778310p5779035.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to insert Date String as date in MongoDB
Posted by gramanero <gr...@gmail.com>.
Thank you for the response and information. I believe I tried what is
recommended in the link that you provided. Below is a copy from the Logs via
the Karaf console. I hope the formatting is not too terrible. I can try and
provide it another way if necessary.
Pretty much the same result as I was seeing before as I believe I already
tried this approach. Here is the document that I used: /{ "time" : { "$date"
: "2016-02-19T17:37:57.673+0000" } }/
Here is what is going on in the route (the reason for the tracing is that I
wanted to see what type the body was being converted to and from...I was
toying around with different conversions to see if I could come up with some
magic incantation that would work):
/ <convertBodyTo type="String" />
<log message="[TRACE] Before unmarshall - ${body}"/>
<unmarshal ref="jsonMarshaller" />
<log message="[TRACE] After unmarshall - ${body}"/>
<to
uri="mongodb:mongoBean?database=PMA&collection=Elmah&writeResultAsHeader=true&operation=insert"
/>/
And the jsonMarshaller dataformat definition is this:
/ <dataFormats>
<json id="jsonMarshaller" library="Jackson"
disableFeatures="FAIL_ON_UNKNOWN_PROPERTIES,WRITE_DATES_AS_TIMESTAMPS"/>
</dataFormats>
/
And the Mongo bean definition:
/ <bean id="mongoBean" class="com.mongodb.Mongo">
<constructor-arg index="0">
<bean class="com.mongodb.MongoURI">
<constructor-arg index="0"
value="mongodb://DEVVLMONGORPLLOGS00.oecapplications.dev:27018/?replicaSet=rsLOGS&connectTimeoutMS=30000&wtimeoutMS=30000&w=1&journal=true"
/>
</bean>
</constructor-arg>
</bean>/
Hope this information is helpful. I certainly appreciate someone looking
into this. We are not Java developers here, but at the moment it feels like
my next course of action is to write some Java code to customize the
serializer.
*From the Logs available via the Karaf Console:*
*3/4/2016, 4:29:33 PM WARN* /Execution of JMS message listener failed.
Caused by: [org.apache.camel.RuntimeCamelException -
org.apache.camel.component.mongodb.CamelMongoDbException: MongoDB operation
= insert, Body is not conversible to type DBObject nor List<DBObject>]
camel-core org.apache.camel.RuntimeCamelException:
org.apache.camel.component.mongodb.CamelMongoDbException: MongoDB operation
= insert, Body is not conversible to type DBObject nor List<DBObject>/
*3/4/2016, 4:29:33 PM ERROR* /Failed delivery for (MessageId:
ID:devvlapifuse01-42307-1456773862280-9:8:3:1:1 on ExchangeId:
ID-devvlapifuse01-49834-1456773865165-1-9). Exhausted after delivery
attempt: 1 caught: org.apache.camel.component.mongodb.CamelMongoDbException:
MongoDB operation = insert, Body is not conversible to type DBObject nor
List<DBObject> Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId ProcessorId Processor Elapsed (ms) [oec.common.logger.]
[oec.common.logger.] [activemq://queue:oec.common.logger.elmah.request ] [
2] [oec.common.logger.] [validate4 ]
[validate[simple{${in.header.MongoDatabase}}] ] [ 0] [oec.common.logger.]
[validate5 ] [validate[simple{${in.header.MongoCollection}}] ] [ 0]
[oec.common.logger.] [log2 ] [log ] [ 0] [oec.common.logger.]
[convertBodyTo1 ] [convertBodyTo[String] ] [ 0] [oec.common.logger.] [log3 ]
[log ] [ 1] [oec.common.logger.] [log4 ] [log ] [ 0] [oec.common.logger.]
[to1 ]
[mongodb:mongoBean?database=PMA&collection=Elmah&writeResultAsHeader=true&opera]
[ 1] Exchange
---------------------------------------------------------------------------------------------------------------------------------------
Exchange[ Id ID-devvlapifuse01-49834-1456773865165-1-9 ExchangePattern
InOnly Headers {Accept=*/*, Accept-Encoding=gzip, deflate,
Accept-Language=en-US,en;q=0.8, AppKey=NfQFDRu7Ogq4xF4hZpMGV0CxdgQCnDPW,
breadcrumbId=ID-devvlapifuse01-49834-1456773865165-1-8,
CamelRedelivered=false, CamelRedeliveryCounter=0,
CamelServletContextPath=/private/common/logger/elmah, Content-Length=59,
Content-Type=application/json, JMSCorrelationID=null,
JMSCorrelationIDAsBytes=null, JMSDeliveryMode=2,
JMSDestination=queue://oec.common.logger.elmah.request, JMSExpiration=0,
JMSMessageID=ID:devvlapifuse01-42307-1456773862280-9:8:3:1:1, JMSPriority=4,
JMSRedelivered=false, JMSReplyTo=null, JMSTimestamp=1457126973022,
JMSType=null, JMSXGroupID=null, JMSXUserID=null, MongoCollection=Elmah,
MongoDatabase=PMA,
Origin=chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop,
Postman-Token=fc4ab341-5fbb-fb41-d0ad-c30cc7c3c30a, TestType=2} BodyType
String Body { "time" : { "$date" : "2016-02-19T17:37:57.673+0000" } }
] Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
camel-core org.apache.camel.component.mongodb.CamelMongoDbException: MongoDB
operation = insert, Body is not conversible to type DBObject nor
List<DBObject>/
*3/4/2016, 4:29:33 PM WARN* /Conversion has fallen back to generic Object ->
DBObject, but unable to convert type org.apache.camel.impl.DefaultMessage.
Returning null. camel-mongodb/
*3/4/2016, 4:29:33 PM INFO* /[TRACE] After unmarshall - { "time" : { "$date"
: "2016-02-19T17:37:57.673+0000" } }
camel-core /
*3/4/2016, 4:29:33 PM INFO* /[TRACE] Before unmarshall - { "time" : {
"$date" : "2016-02-19T17:37:57.673+0000" } }
camel-core /
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-insert-Date-String-as-date-in-MongoDB-tp5778310p5778652.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to insert Date String as date in MongoDB
Posted by Raul Kripalani <ra...@apache.org>.
MongoDB uses a special representation of JSON they call BSON.
According to their docs, timestamps should be represented with the
$timestamp token, along with 't' and 'i' portions for them to be
interpreted correctly by the drivers:
https://docs.mongodb.org/manual/reference/mongodb-extended-json/#timestamp.
It requires turning the date into an epoch.
For dates:
https://docs.mongodb.org/manual/reference/mongodb-extended-json/#data_date.
Could you guys try that?
*Raúl Kripalani*
PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and
Messaging Engineer
http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani
Blog: raul.io | twitter: @raulvk <https://twitter.com/raulvk>
On Thu, Mar 3, 2016 at 3:47 PM, gramanero <gr...@gmail.com> wrote:
> I would be interested in knowing the answer on how to do this as well. I
> posted in a different thread and the datatype I am trying to do this with
> is
> the ISODate built-in mongo type. I am using Spring DSL as well. Thanks!
>
>
>
> --
> View this message in context:
> http://camel.465427.n5.nabble.com/How-to-insert-Date-String-as-date-in-MongoDB-tp5778310p5778555.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
Re: How to insert Date String as date in MongoDB
Posted by gramanero <gr...@gmail.com>.
I would be interested in knowing the answer on how to do this as well. I
posted in a different thread and the datatype I am trying to do this with is
the ISODate built-in mongo type. I am using Spring DSL as well. Thanks!
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-insert-Date-String-as-date-in-MongoDB-tp5778310p5778555.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to insert Date String as date in MongoDB
Posted by yogu13 <yo...@gmail.com>.
Apologies for delayed response! can u try having it as Date instead of
String?
Regards,
-Yogesh
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-insert-Date-String-as-date-in-MongoDB-tp5778310p5778457.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to insert Date String as date in MongoDB
Posted by Royamit <am...@gmail.com>.
It is of type string
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-insert-Date-String-as-date-in-MongoDB-tp5778310p5778407.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to insert Date String as date in MongoDB
Posted by yogu13 <yo...@gmail.com>.
Hi,
What datatype is CreatedDateTimeUTC after it is converted using
.convertBodyTo(DBObject.class)
Regards,
-Yogesh
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-insert-Date-String-as-date-in-MongoDB-tp5778310p5778385.html
Sent from the Camel - Users mailing list archive at Nabble.com.