You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@camel.apache.org by "Julien Faissolle (JIRA)" <ji...@apache.org> on 2009/04/02 16:43:34 UTC
[jira] Created: (CAMEL-1513) camel-csv : mutliple messages lead to
repeated values
camel-csv : mutliple messages lead to repeated values
-----------------------------------------------------
Key: CAMEL-1513
URL: https://issues.apache.org/activemq/browse/CAMEL-1513
Project: Apache Camel
Issue Type: Bug
Components: camel-core
Affects Versions: 2.0-M1
Reporter: Julien Faissolle
Priority: Minor
I have this config :
{code:xml}
<route>
<from uri="direct:msgIn"/>
<marshal><csv /></marshal>
<to uri="file://msgs?fileName=messages.csv" />
</route>
{code}
I send Map objects with this :
{code:java}
ProducerTemplate template = context.createProducerTemplate();
Map msg1 = new HashMap();
msg1.put("A", 1);
msg1.put("B", 1);
Map msg2 = new HashMap();
msg2.put("A", 2);
msg2.put("B", 2);
template.sendBody("direct:msgIn", msg1);
template.sendBody("direct:msgIn", msg2);
{code}
This produces the following result :
{code}
1,1
2,2,2,2
{code}
instead of
{code}
1,1
2,2
{code}
The more messages are pumped into the CSV marshaller, the more the values are repeated. This is because the marshal method in CsvDataFormat keeps adding columns to the config even if they are already present :
{code:java|title=CsvDataFormat.java}
........
CSVConfig conf = getConfig();
// lets add fields
Set set = map.keySet();
for (Object value : set) {
if (value != null) {
String text = value.toString();
CSVField field = new CSVField(text);
conf.addField(field);
}
}
CSVWriter writer = new CSVWriter(conf);
.......
{code}
I think the marshal method should perform something like
{code:java}
if (config == null) {
config = createConfig();
// lets add fields
Set set = map.keySet();
for (Object value : set) {
if (value != null) {
..............
}
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-1513) camel-csv : mutliple messages lead to
repeated values
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen updated CAMEL-1513:
-------------------------------
Fix Version/s: 2.0.0
> camel-csv : mutliple messages lead to repeated values
> -----------------------------------------------------
>
> Key: CAMEL-1513
> URL: https://issues.apache.org/activemq/browse/CAMEL-1513
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Julien Faissolle
> Assignee: Claus Ibsen
> Priority: Minor
> Fix For: 2.0.0
>
>
> I have this config :
> {code:xml}
> <route>
> <from uri="direct:msgIn"/>
> <marshal><csv /></marshal>
> <to uri="file://msgs?fileName=messages.csv" />
> </route>
> {code}
> I send Map objects with this :
> {code:java}
> ProducerTemplate template = context.createProducerTemplate();
> Map msg1 = new HashMap();
> msg1.put("A", 1);
> msg1.put("B", 1);
> Map msg2 = new HashMap();
> msg2.put("A", 2);
> msg2.put("B", 2);
> template.sendBody("direct:msgIn", msg1);
> template.sendBody("direct:msgIn", msg2);
> {code}
> This produces the following result :
> {code}
> 1,1
> 2,2,2,2
> {code}
> instead of
> {code}
> 1,1
> 2,2
> {code}
> The more messages are pumped into the CSV marshaller, the more the values are repeated. This is because the marshal method in CsvDataFormat keeps adding columns to the config even if they are already present :
> {code:java|title=CsvDataFormat.java}
> ........
> CSVConfig conf = getConfig();
> // lets add fields
> Set set = map.keySet();
> for (Object value : set) {
> if (value != null) {
> String text = value.toString();
> CSVField field = new CSVField(text);
> conf.addField(field);
> }
> }
> CSVWriter writer = new CSVWriter(conf);
> .......
> {code}
> I think the marshal method should perform something like
> {code:java}
> if (config == null) {
> config = createConfig();
> // lets add fields
> Set set = map.keySet();
> for (Object value : set) {
> if (value != null) {
> ..............
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CAMEL-1513) camel-csv : mutliple messages lead to
repeated values
Posted by "Julien Faissolle (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julien Faissolle updated CAMEL-1513:
------------------------------------
Attachment: camel-csv.patch
Here is a patch for this issue along with unit tests. The behavior is now by default to add new fields only if they have not already been set by previous messages. Thus, the field order stays consistent. The field addition is synchronized to avoid concurrency problems. The new property _autogenColumns_ can be set to false in order to disable the automatic generation of fields. Disabling automatic generation is especially useful in the case of a custom CSVConfig.
> camel-csv : mutliple messages lead to repeated values
> -----------------------------------------------------
>
> Key: CAMEL-1513
> URL: https://issues.apache.org/activemq/browse/CAMEL-1513
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Julien Faissolle
> Assignee: Claus Ibsen
> Priority: Minor
> Fix For: 2.0.0
>
> Attachments: camel-csv.patch
>
>
> I have this config :
> {code:xml}
> <route>
> <from uri="direct:msgIn"/>
> <marshal><csv /></marshal>
> <to uri="file://msgs?fileName=messages.csv" />
> </route>
> {code}
> I send Map objects with this :
> {code:java}
> ProducerTemplate template = context.createProducerTemplate();
> Map msg1 = new HashMap();
> msg1.put("A", 1);
> msg1.put("B", 1);
> Map msg2 = new HashMap();
> msg2.put("A", 2);
> msg2.put("B", 2);
> template.sendBody("direct:msgIn", msg1);
> template.sendBody("direct:msgIn", msg2);
> {code}
> This produces the following result :
> {code}
> 1,1
> 2,2,2,2
> {code}
> instead of
> {code}
> 1,1
> 2,2
> {code}
> The more messages are pumped into the CSV marshaller, the more the values are repeated. This is because the marshal method in CsvDataFormat keeps adding columns to the config even if they are already present :
> {code:java|title=CsvDataFormat.java}
> ........
> CSVConfig conf = getConfig();
> // lets add fields
> Set set = map.keySet();
> for (Object value : set) {
> if (value != null) {
> String text = value.toString();
> CSVField field = new CSVField(text);
> conf.addField(field);
> }
> }
> CSVWriter writer = new CSVWriter(conf);
> .......
> {code}
> I think the marshal method should perform something like
> {code:java}
> if (config == null) {
> config = createConfig();
> // lets add fields
> Set set = map.keySet();
> for (Object value : set) {
> if (value != null) {
> ..............
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-1513) camel-csv : mutliple messages lead
to repeated values
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=51026#action_51026 ]
Claus Ibsen commented on CAMEL-1513:
------------------------------------
Julien, well spotted
Do you care to try to create a patch?
http://camel.apache.org/support.html
And please add an unit test demonstrating this bug is fixed.
The problem is indeed the fact that CSVConfig is not created on each operation.
So the CSVConfig should *not* be a shared instance. It should be created on each invocation.
> camel-csv : mutliple messages lead to repeated values
> -----------------------------------------------------
>
> Key: CAMEL-1513
> URL: https://issues.apache.org/activemq/browse/CAMEL-1513
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Julien Faissolle
> Assignee: Claus Ibsen
> Priority: Minor
> Fix For: 2.0.0
>
>
> I have this config :
> {code:xml}
> <route>
> <from uri="direct:msgIn"/>
> <marshal><csv /></marshal>
> <to uri="file://msgs?fileName=messages.csv" />
> </route>
> {code}
> I send Map objects with this :
> {code:java}
> ProducerTemplate template = context.createProducerTemplate();
> Map msg1 = new HashMap();
> msg1.put("A", 1);
> msg1.put("B", 1);
> Map msg2 = new HashMap();
> msg2.put("A", 2);
> msg2.put("B", 2);
> template.sendBody("direct:msgIn", msg1);
> template.sendBody("direct:msgIn", msg2);
> {code}
> This produces the following result :
> {code}
> 1,1
> 2,2,2,2
> {code}
> instead of
> {code}
> 1,1
> 2,2
> {code}
> The more messages are pumped into the CSV marshaller, the more the values are repeated. This is because the marshal method in CsvDataFormat keeps adding columns to the config even if they are already present :
> {code:java|title=CsvDataFormat.java}
> ........
> CSVConfig conf = getConfig();
> // lets add fields
> Set set = map.keySet();
> for (Object value : set) {
> if (value != null) {
> String text = value.toString();
> CSVField field = new CSVField(text);
> conf.addField(field);
> }
> }
> CSVWriter writer = new CSVWriter(conf);
> .......
> {code}
> I think the marshal method should perform something like
> {code:java}
> if (config == null) {
> config = createConfig();
> // lets add fields
> Set set = map.keySet();
> for (Object value : set) {
> if (value != null) {
> ..............
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CAMEL-1513) camel-csv : mutliple messages lead
to repeated values
Posted by "Julien Faissolle (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=51036#action_51036 ]
Julien Faissolle commented on CAMEL-1513:
-----------------------------------------
Ok, I'll work on it. But I don't think the CSVConfig should be automatically created on each invocation because it would forbid providing one's own CSVConfig instance through setConfig(). Note that manually assigning a CSVConfig doesn't work as it should because of the current bug.
> camel-csv : mutliple messages lead to repeated values
> -----------------------------------------------------
>
> Key: CAMEL-1513
> URL: https://issues.apache.org/activemq/browse/CAMEL-1513
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Julien Faissolle
> Assignee: Claus Ibsen
> Priority: Minor
> Fix For: 2.0.0
>
>
> I have this config :
> {code:xml}
> <route>
> <from uri="direct:msgIn"/>
> <marshal><csv /></marshal>
> <to uri="file://msgs?fileName=messages.csv" />
> </route>
> {code}
> I send Map objects with this :
> {code:java}
> ProducerTemplate template = context.createProducerTemplate();
> Map msg1 = new HashMap();
> msg1.put("A", 1);
> msg1.put("B", 1);
> Map msg2 = new HashMap();
> msg2.put("A", 2);
> msg2.put("B", 2);
> template.sendBody("direct:msgIn", msg1);
> template.sendBody("direct:msgIn", msg2);
> {code}
> This produces the following result :
> {code}
> 1,1
> 2,2,2,2
> {code}
> instead of
> {code}
> 1,1
> 2,2
> {code}
> The more messages are pumped into the CSV marshaller, the more the values are repeated. This is because the marshal method in CsvDataFormat keeps adding columns to the config even if they are already present :
> {code:java|title=CsvDataFormat.java}
> ........
> CSVConfig conf = getConfig();
> // lets add fields
> Set set = map.keySet();
> for (Object value : set) {
> if (value != null) {
> String text = value.toString();
> CSVField field = new CSVField(text);
> conf.addField(field);
> }
> }
> CSVWriter writer = new CSVWriter(conf);
> .......
> {code}
> I think the marshal method should perform something like
> {code:java}
> if (config == null) {
> config = createConfig();
> // lets add fields
> Set set = map.keySet();
> for (Object value : set) {
> if (value != null) {
> ..............
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (CAMEL-1513) camel-csv : mutliple messages lead to
repeated values
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen reassigned CAMEL-1513:
----------------------------------
Assignee: Claus Ibsen
> camel-csv : mutliple messages lead to repeated values
> -----------------------------------------------------
>
> Key: CAMEL-1513
> URL: https://issues.apache.org/activemq/browse/CAMEL-1513
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Julien Faissolle
> Assignee: Claus Ibsen
> Priority: Minor
>
> I have this config :
> {code:xml}
> <route>
> <from uri="direct:msgIn"/>
> <marshal><csv /></marshal>
> <to uri="file://msgs?fileName=messages.csv" />
> </route>
> {code}
> I send Map objects with this :
> {code:java}
> ProducerTemplate template = context.createProducerTemplate();
> Map msg1 = new HashMap();
> msg1.put("A", 1);
> msg1.put("B", 1);
> Map msg2 = new HashMap();
> msg2.put("A", 2);
> msg2.put("B", 2);
> template.sendBody("direct:msgIn", msg1);
> template.sendBody("direct:msgIn", msg2);
> {code}
> This produces the following result :
> {code}
> 1,1
> 2,2,2,2
> {code}
> instead of
> {code}
> 1,1
> 2,2
> {code}
> The more messages are pumped into the CSV marshaller, the more the values are repeated. This is because the marshal method in CsvDataFormat keeps adding columns to the config even if they are already present :
> {code:java|title=CsvDataFormat.java}
> ........
> CSVConfig conf = getConfig();
> // lets add fields
> Set set = map.keySet();
> for (Object value : set) {
> if (value != null) {
> String text = value.toString();
> CSVField field = new CSVField(text);
> conf.addField(field);
> }
> }
> CSVWriter writer = new CSVWriter(conf);
> .......
> {code}
> I think the marshal method should perform something like
> {code:java}
> if (config == null) {
> config = createConfig();
> // lets add fields
> Set set = map.keySet();
> for (Object value : set) {
> if (value != null) {
> ..............
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CAMEL-1513) camel-csv : mutliple messages lead to
repeated values
Posted by "Claus Ibsen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/activemq/browse/CAMEL-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Claus Ibsen resolved CAMEL-1513.
--------------------------------
Resolution: Fixed
Fix Version/s: 1.6.1
Applied patch with thanks to Julien
trunk: 763551
1.x: 763553
> camel-csv : mutliple messages lead to repeated values
> -----------------------------------------------------
>
> Key: CAMEL-1513
> URL: https://issues.apache.org/activemq/browse/CAMEL-1513
> Project: Apache Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.0-M1
> Reporter: Julien Faissolle
> Assignee: Claus Ibsen
> Priority: Minor
> Fix For: 2.0.0, 1.6.1
>
> Attachments: camel-csv.patch
>
>
> I have this config :
> {code:xml}
> <route>
> <from uri="direct:msgIn"/>
> <marshal><csv /></marshal>
> <to uri="file://msgs?fileName=messages.csv" />
> </route>
> {code}
> I send Map objects with this :
> {code:java}
> ProducerTemplate template = context.createProducerTemplate();
> Map msg1 = new HashMap();
> msg1.put("A", 1);
> msg1.put("B", 1);
> Map msg2 = new HashMap();
> msg2.put("A", 2);
> msg2.put("B", 2);
> template.sendBody("direct:msgIn", msg1);
> template.sendBody("direct:msgIn", msg2);
> {code}
> This produces the following result :
> {code}
> 1,1
> 2,2,2,2
> {code}
> instead of
> {code}
> 1,1
> 2,2
> {code}
> The more messages are pumped into the CSV marshaller, the more the values are repeated. This is because the marshal method in CsvDataFormat keeps adding columns to the config even if they are already present :
> {code:java|title=CsvDataFormat.java}
> ........
> CSVConfig conf = getConfig();
> // lets add fields
> Set set = map.keySet();
> for (Object value : set) {
> if (value != null) {
> String text = value.toString();
> CSVField field = new CSVField(text);
> conf.addField(field);
> }
> }
> CSVWriter writer = new CSVWriter(conf);
> .......
> {code}
> I think the marshal method should perform something like
> {code:java}
> if (config == null) {
> config = createConfig();
> // lets add fields
> Set set = map.keySet();
> for (Object value : set) {
> if (value != null) {
> ..............
> }
> {code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.