You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@camel.apache.org by "Claus Ibsen (Jira)" <ji...@apache.org> on 2023/02/27 16:05:00 UTC
[jira] [Commented] (CAMEL-19094) Tokenizer ignores includeTokens
[ https://issues.apache.org/jira/browse/CAMEL-19094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17694077#comment-17694077 ]
Claus Ibsen commented on CAMEL-19094:
-------------------------------------
include tokens is only an option when you use tokenizer in pair mode (start and end tokens).
In this use case of yours then its not supported. It would also be a bit confusing as the split will then output the token as well
So you will have
{code}
H
el
lo Cam
el
from java
{code}
But maybe there is a valid use-case for include token(s) in non-pair mode?
> Tokenizer ignores includeTokens
> -------------------------------
>
> Key: CAMEL-19094
> URL: https://issues.apache.org/jira/browse/CAMEL-19094
> Project: Camel
> Issue Type: Bug
> Affects Versions: 3.20.1
> Reporter: Pasquale Congiusti
> Priority: Major
>
> Hi folks,
> maybe I am using it in a wrong way, but it seems to me that the Tokenizer is not working as it should when we try to use IncludeTokens option. I'm testing the following:
> {code}
> TokenizeLanguage lan = new TokenizeLanguage();
> lan.setIncludeTokens(true);
> lan.setToken("el");
> from("timer:java?period={{time:1000}}").routeId("java")
> .setBody()
> .simple("Hello Camel from ${routeId}")
> //.split(body().tokenize("el"))
> .split(lan.createExpression())
> .log("${body}");
> {code}
> But I get
> {code}
> 2023-02-27 15:13:27.462 INFO 596799 --- [ - timer://java] Test.java:22 : H
> 2023-02-27 15:13:27.464 INFO 596799 --- [ - timer://java] Test.java:22 : lo Cam
> 2023-02-27 15:13:27.464 INFO 596799 --- [ - timer://java] Test.java:22 : from java
> 2023-02-27 15:13:28.428 INFO 596799 --- [ - timer://java] Test.java:22 : H
> 2023-02-27 15:13:28.430 INFO 596799 --- [ - timer://java] Test.java:22 : lo Cam
> 2023-02-27 15:13:28.431 INFO 596799 --- [ - timer://java] Test.java:22 : from java
> 2023-02-27 15:13:29.428 INFO 596799 --- [ - timer://java] Test.java:22 : H
> 2023-02-27 15:13:29.430 INFO 596799 --- [ - timer://java] Test.java:22 : lo Cam
> 2023-02-27 15:13:29.431 INFO 596799 --- [ - timer://java] Test.java:22 : from java
> {code}
> I'd expect there to get "el" token as well.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)