You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by "richie.riviere@gmail.com" <ri...@gmail.com> on 2013/10/15 12:51:25 UTC
How to write a type converter?
Hi all (sorry in advance for the long post),
I have been tweaking the camel-example-etl project to what I require for my
project. My requirements are very similar. I need to parse an xml document
and then insert the parsed entities as records into the database.
The structure of my xml is different to the example. Here is what my xml
looks like.
<WEB-ITM-EXT-REC>
<ACTVTY-CODE>A</ACTVTY-CODE>
<WEB-ITM-EXT-DATA>
<ITEM-NO>93501250080</ITEM-NO>
<PROMOTION-DETAILS>
<PROMOTION-ID></PROMOTION-ID>
<PROMOTION-START-DATE>0</PROMOTION-START-DATE>
<PROMOTION-END-DATE>0</PROMOTION-END-DATE>
<PROMOTION-PRICE>.00</PROMOTION-PRICE>
</PROMOTION-DETAILS>
<NORMAL-SELL-PRICES>
<SELL-PRICE-EFFECTIVE-DATE-1>0</SELL-PRICE-EFFECTIVE-DATE-1>
<SELL-PRICE-1>.00</SELL-PRICE-1>
<SELL-PRICE-EFFECTIVE-DATE-2>0</SELL-PRICE-EFFECTIVE-DATE-2>
<SELL-PRICE-2>.00</SELL-PRICE-2>
<SELL-PRICE-EFFECTIVE-DATE-3>0</SELL-PRICE-EFFECTIVE-DATE-3>
<SELL-PRICE-3>189.95</SELL-PRICE-3>
</NORMAL-SELL-PRICES>
<OUT-OF-STOCK-IND>Y</OUT-OF-STOCK-IND>
</WEB-ITM-EXT-DATA>
</WEB-ITM-EXT-REC>
So far I have succesfully managed to get jaxb to parse my xml using four
beans (hooray!).
1. ItemDocuments (holds a list of ItemDocument)
2. ItemDocument
3. PromotionDocument
4. SellPriceDocument
So after parsing the document I end up with a single ItemDocuments class
holding an ArrayList of ItemDocument which in turn has a PromotionDocument
and a SellPriceDocument.
Now I need to insert my List of ItemDocument into the database as records. I
have three entity beans (ItemEntity, PromotionEntity, SellPriceEntity).
I am confused about how to change the Type Converter to do this for me. In
the original camel-example-etl project there was not a collection of parsed
records, it was only parsing 1 record at a time and only using 1 entity.
So I have a few questions...
1. How do I re-write my route below to convert to the three separate entity
beans? i.e. One parsed record for me should lead to three db inserts into my
three entities.
public class ItemEtl extends SpringRouteBuilder {
public void configure() throws Exception {
System.err.println("CONVERT STARTED");
from("file:src/data?noop=true")
.convertBodyTo(ItemDocuments.class)
.to("jpa:org.apache.camel.example.etl.ItemEntity");
System.err.println("CONVERT FINISHED");
}
}
2. How do I write my Type Converter to handle my ItemDocuments and my list
of ItemDocument. In t he original example there is just a single record not
a list of them. I tried something like below but very unsure about it.
@Converter
public class ItemsTransformer {
private static final transient Logger LOG =
LoggerFactory.getLogger(ItemsTransformer.class);
@Converter
public List<ItemEntity> toItem(ItemDocuments docs, Exchange exchange)
throws Exception {
JpaTemplate template =
exchange.getIn().getHeader("CamelJpaTemplate", JpaTemplate.class);
List<ItemEntity> itemEntities = new ArrayList<ItemEntity>();
List<ItemDocument> itemDocumentList = docs.getItemDocumentList();
for (ItemDocument doc : itemDocumentList) {
String itemNo = doc.getItemNo();
ItemEntity item = findItemByItemNumber(template, itemNo);
item.setItemNo(itemNo);
item.setClassNo(item.getClassNo());
item.setDescription(item.getDescription());
item.setGstCode(item.getGstCode());
LOG.debug("Created object item: " + itemNo);
itemEntities.add(item);
}
return itemEntities;
}
protected ItemEntity findItemByItemNumber(JpaTemplate template, String
itemNo) throws Exception {
List<ItemEntity> list = CastUtils.cast(template.find("select x from
item"
+ " x where
x.item_no = ?1", itemNo));
if (list.isEmpty()) {
ItemEntity answer = new ItemEntity();
answer.setItemNo(itemNo);
return answer;
} else {
return list.get(0);
}
}
}
Of course it does not work. My error is ...
No type converter available to convert from type:
org.apache.camel.example.etl.ItemDocuments to the required type:
org.apache.camel.example.etl.ItemEntity with value
org.apache.camel.example.etl.ItemDocuments@535a4838
I would appreciate any help that anyone could give me.
thank you
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-write-a-type-converter-tp5741592.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: JPA type converter for multiple entities?
Posted by "richie.riviere@gmail.com" <ri...@gmail.com>.
okay thanks. My main concern is that I am defining my route properly. At the
moment I am only defining my route to the item entity and I was thinking
maybe because I had not specified the other entities it was using null or
something like that.
So it seems like from the response my problem is not my route but more a
problem with how I have configured JPA. I will take a look at my JPA
definitions and post my resolution... assuming I find it.
thanks again for your help.
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-write-a-type-converter-tp5741592p5741711.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: JPA type converter for multiple entities?
Posted by gquintana <ge...@gmail.com>.
Your error message is telling "Incorrect integer value:
'\xAC\xED\x00\x05sr\x00,org.apache.camel.example.etl.PromotionEntity$\x0C\xF5\xF1\x08\x0B\xA2\x81\x02\x00\x05L\x00\x02idt\x00\x10'
for column 'ITEM_PROMOTION_ID'". It seems to that you are trying to insert a
Java String into a SQL Number column. Either the PromotionEntity has a
column mapping problem or your database schema is wrong.
Gérald
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-write-a-type-converter-tp5741592p5741660.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: JPA type converter for multiple entities?
Posted by "richie.riviere@gmail.com" <ri...@gmail.com>.
Just bumping this question as I'm struggling with a solution. If I am trying
to persist to three separate entities should my routes specify all three
entities?
Or is it enough for only the first entity to be specified in the route and
then to rely on the relationships defined to the other two beans to allow
for inserts into the other tables?
thanks again
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-write-a-type-converter-tp5741592p5741645.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to write a type converter?
Posted by "richie.riviere@gmail.com" <ri...@gmail.com>.
I liked your original idea of doing a split so I had a look around and found
that chapter 8 in the Camel in Action book had a good example.
To split the ItemDocuments what I did was this....
public void configure() throws Exception {
System.err.println("CONVERT STARTED");
from("file:src/data?noop=true")
.convertBodyTo(ItemDocuments.class)
.split().method(ItemDocumentService.class,
"splitItemsDocuments")
.log("Split line ${body}")
.to("jpa:org.apache.camel.example.etl.ItemEntity");
System.err.println("CONVERT FINISHED");
}
public class ItemDocumentService {
public List<ItemDocument> splitItemsDocuments(ItemDocuments docs) {
return docs.getItemDocumentList();
}
}
That has worked. It has split the itemDocuments into ItemDocument objects
and I am now entering my TypeConverter code.
I have another point of confusion now though. In my type converter if only
only persist my item object the item records get inserted into the database
(success!). However, if I try to persist the promotion and sell price
documents with the item document my application starts to spit errors.
Caused by: <openjpa-2.2.1-r422266:1396819 fatal general error>
org.apache.openjpa.persistence.PersistenceException: Incorrect integer
value:
'\xAC\xED\x00\x05sr\x00,org.apache.camel.example.etl.PromotionEntity$\x0C\xF5\xF1\x08\x0B\xA2\x81\x02\x00\x05L\x00\x02idt\x00\x10'
for column 'ITEM_PROMOTION_ID' at row 1 {prepstmnt 1042322838 INSERT INTO
item (id, ATTRIBUTE_1, ATTRIBUTE_3, ATTRIBUTE_2, BRAND_LOGO_FILE_NAME,
BRAND_NAME, CLASS_NO, DEFAULT_MARGIN, DESCRIPTION, EXTENDED_DESCRIPTION,
EXTENDED_DESCRIPTION_2, GST_CODE, IMAGE_FILE_NAME, ITEM_NO,
OUT_OF_STOCK_IND, PACK_QTY, ITEM_PROMOTION_ID, ITEM_SELL_PRICE_ID,
SELLING_UNIT, SIZE_APPLICABLE, STOCK_AVAILABLE, SPPLR_NO, VOLUME,
WEB_AGE_GROUP, WEB_COLOR_DESCRIPTION, WEB_DESCRIPTION, WEB_SIZE_DESCRIPTION,
WEIGHT) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?)} [code=1366, state=HY000]
FailedObject: org.apache.camel.example.etl.ItemEntity@2d8bfa3f
at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4958)
at
org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4918)
at
org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:136)
at
org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:78)
at
org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:143)
at
org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:79)
at
org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInternal(PreparedStatementManagerImpl.java:99)
at
org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(PreparedStatementManagerImpl.java:87)
at
org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:550)
at
org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:106)
at
org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(BatchingConstraintUpdateManager.java:59)
at
org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:105)
at
org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:78)
at
org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:735)
at
org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:131)
... 101 more
What I was thinking was that I could have two things wrong here.
1. In my route I have only specified
.to("jpa:org.apache.camel.example.etl.ItemEntity"); Do I need to specify the
other entities too? Or because there is a relationship between the entities
will JPA be smart enough to figure it out.
2. Perhaps I have the JPA entities set up incorrectly. In my JPA item entity
i have the relationships set up like this...
@OneToOne(cascade = CascadeType.ALL)
@JoinColumn(name="ITEM_PROMOTION_ID")
private PromotionEntity promotion;
@OneToOne(cascade = CascadeType.ALL)
@JoinColumn(name="ITEM_SELL_PRICE_ID")
private SellPriceEntity sellPrice;
I'm thinking it is the 1st one. My route is not defined properly. Do you
guys have any thoughts?
thanks
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-write-a-type-converter-tp5741592p5741628.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to write a type converter?
Posted by brenuart <be...@itma.lu>.
Hmmm... sorry, but I don't think my suggestion will work: ItemDocuments is
not a List<ItemEntity> - so the split as I wrote will fail.
Just wondering if the approach shouldn't be:
1/ convert from XML to ItemsDocuments
2/ split the ItemDocuments into ItemDocument instances with a Splitter
configured with a POJO of your own
3/ you now have as many Exchanges as you have ItemDocument (because of the
split), each with an ItemDocument in the body.
4/ the last step is to convert/transform an ItemDocument into an ItemEntity.
This can be done with a converter as you did, or using a custom POJO
processor
5/ finally, the exchange should now holds something your JPA destination
could accept.
(also learning Camel... so don't take everything I said for granted ;-)
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-write-a-type-converter-tp5741592p5741625.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to write a type converter?
Posted by "richie.riviere@gmail.com" <ri...@gmail.com>.
thanks. This is exactly the type of suggestion I was looking for. Since I'm
new to Camel I'm still learning the concepts and different patterns.
I'm going to try that right now. Will let you know how I go.
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-write-a-type-converter-tp5741592p5741624.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: How to write a type converter?
Posted by brenuart <be...@itma.lu>.
Just my two cents (quite new to Camel), but why don't use a Splitter to split
the List into its individual items?
Example:
from("file:src/data?noop=true")
.convertBodyTo(ItemDocuments.class)
.split().body()
.to("jpa:org.apache.camel.example.etl.ItemEntity");
--
View this message in context: http://camel.465427.n5.nabble.com/How-to-write-a-type-converter-tp5741592p5741623.html
Sent from the Camel - Users mailing list archive at Nabble.com.