You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@camel.apache.org by lsantos <le...@globo.com> on 2016/11/07 15:00:44 UTC
Cache api call result
Hi there,
I am working on a command-line camel application that polls a directory for
a file, reads it, process each line, modifies a piece of information on the
line and rewrites it in new file.
I have to make 2 http calls as part of the msg enrichment process, the first
call is as authorization call that gets a json web token that is sent in the
headers of the second http call.
My route dsl looks something like this:
from("{{consumerEndpoint}}")
.validate(body().regex(FILE_HEADER_PATTERN))
.split(body().tokenize())
.choice()
.when(body().regex(RECORD_LINE_PATTERN))
.process(pluckOutTokenRecordProcessor())
.setHeader(Exchange.HTTP_METHOD, constant("POST"))
// call that I want its result cached for reuse
.enrich("{{firstEnrichEndpoint}}",
getFirstAggregationStrategy()) .setHeader(Exchange.HTTP_METHOD,
constant("GET"))
.enrich().simple("{{sndEnrichEndpoint}}").aggregationStrategy(getSndAggregationStrategy())
.process(recordUpdaterProcessor())
.end() //end choice
.aggregate(header(FILE_NAME_ONLY),
stringBodyAggregator())
.convertBodyTo(String.class)
.to("{{transferEndpoint}}")
.end();
It is all good and working, however, the file that gets processed in
production has millions of lines and I do not want to have to make an extra
authorization http call for each record that is processed, as the json web
token is valid for 20 minutes before it expires which is less than what
takes to process one entire file.
I would like to keep the jwt auth token cached in memory for let's say 15
minutes and refresh it periodically.
I was wondering how is the best approach to achieve this goal. I know that
camel ships with a timer component, but I never worked with it.
--
View this message in context: http://camel.465427.n5.nabble.com/Cache-api-call-result-tp5789849.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Cache api call result
Posted by lsantos <le...@globo.com>.
Thanks, will try that.
On 07/11/2016 12:25, taariq [via Camel] wrote:
> Hi,
> You can use camel-ehcache, something like this example from the old
> camel-cache;
>
> from("direct:start")
> // Prepare headers
> .setHeader(CacheConstants.CACHE_OPERATION,
> constant(CacheConstants.CACHE_OPERATION_GET))
> .setHeader(CacheConstants.CACHE_KEY, constant("Ralph_Waldo_Emerson")).
> .to("cache://TestCache1").
> // Check if entry was not found
> .choice().when(header(CacheConstants.CACHE_ELEMENT_WAS_FOUND).isNull()).
> // If not found, get the payload and put it to cache
> .to("cxf:bean:someHeavyweightOperation").
> .setHeader(CacheConstants.CACHE_OPERATION,
> constant(CacheConstants.CACHE_OPERATION_ADD))
> .setHeader(CacheConstants.CACHE_KEY,
> constant("Ralph_Waldo_Emerson"))
> .to("cache://TestCache1")
> .end()
> .to("direct:nextPhase");
>
> http://camel.apache.org/ehcache.html [1]
>
> Try modify the route to encapsulate your firstEnrichEndpoint.
>
> Cheers
>
> On 07 Nov 2016 17:06, "lsantos" <[hidden email]> wrote:
>
>> Hi there,
>>
>> I am working on a command-line camel application that polls a directory for
>> a file, reads it, process each line, modifies a piece of information on the
>> line and rewrites it in new file.
>>
>> I have to make 2 http calls as part of the msg enrichment process, the
>> first
>> call is as authorization call that gets a json web token that is sent in
>> the
>> headers of the second http call.
>>
>> My route dsl looks something like this:
>>
>> from("{{consumerEndpoint}}")
>> .validate(body().regex(FILE_HEADER_PATTERN))
>> .split(body().tokenize())
>> .choice()
>> .when(body().regex(RECORD_LINE_PATTERN))
>> .process(pluckOutTokenRecordProcessor())
>> .setHeader(Exchange.HTTP_METHOD, constant("POST"))
>> // call that I want its result cached for reuse
>> .enrich("{{firstEnrichEndpoint}}",
>> getFirstAggregationStrategy()) .setHeader(Exchange.HTTP_METHOD,
>> constant("GET"))
>>
>> .enrich().simple("{{sndEnrichEndpoint}}").aggregationStrategy(
>> getSndAggregationStrategy())
>> .process(recordUpdaterProcessor())
>> .end() //end choice
>> .aggregate(header(FILE_NAME_ONLY),
>> stringBodyAggregator())
>> .convertBodyTo(String.class)
>> .to("{{transferEndpoint}}")
>> .end();
>>
>>
>> It is all good and working, however, the file that gets processed in
>> production has millions of lines and I do not want to have to make an extra
>> authorization http call for each record that is processed, as the json web
>> token is valid for 20 minutes before it expires which is less than what
>> takes to process one entire file.
>>
>> I would like to keep the jwt auth token cached in memory for let's say 15
>> minutes and refresh it periodically.
>>
>> I was wondering how is the best approach to achieve this goal. I know that
>> camel ships with a timer component, but I never worked with it.
>>
>>
>>
>>
>>
>> --
>> View this message in context: http://camel.465427.n5.nabble [2].
>> com/Cache-api-call-result-tp5789849.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>
> -------------------------
>
> If you reply to this email, your message will be added to the discussion below: http://camel.465427.n5.nabble.com/Cache-api-call-result-tp5789849p5789864.html [3]
> To unsubscribe from Cache api call result, click here [4].
> NAML [5]
Links:
------
[1] http://camel.apache.org/ehcache.html
[2] http://camel.465427.n5.nabble
[3]
http://camel.465427.n5.nabble.com/Cache-api-call-result-tp5789849p5789864.html
[4]
http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5789849&code=bGVhbmRyby5idWVub0BnbG9iby5jb218NTc4OTg0OXw4MDMyMzU2MzQ=
[5]
http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
--
View this message in context: http://camel.465427.n5.nabble.com/Cache-api-call-result-tp5789849p5789867.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Re: Cache api call result
Posted by Taariq Levack <ta...@gmail.com>.
Hi,
You can use camel-ehcache, something like this example from the old
camel-cache;
from("direct:start")
// Prepare headers
.setHeader(CacheConstants.CACHE_OPERATION,
constant(CacheConstants.CACHE_OPERATION_GET))
.setHeader(CacheConstants.CACHE_KEY, constant("Ralph_Waldo_Emerson")).
.to("cache://TestCache1").
// Check if entry was not found
.choice().when(header(CacheConstants.CACHE_ELEMENT_WAS_FOUND).isNull()).
// If not found, get the payload and put it to cache
.to("cxf:bean:someHeavyweightOperation").
.setHeader(CacheConstants.CACHE_OPERATION,
constant(CacheConstants.CACHE_OPERATION_ADD))
.setHeader(CacheConstants.CACHE_KEY,
constant("Ralph_Waldo_Emerson"))
.to("cache://TestCache1")
.end()
.to("direct:nextPhase");
http://camel.apache.org/ehcache.html
Try modify the route to encapsulate your firstEnrichEndpoint.
Cheers
On 07 Nov 2016 17:06, "lsantos" <le...@globo.com> wrote:
> Hi there,
>
> I am working on a command-line camel application that polls a directory for
> a file, reads it, process each line, modifies a piece of information on the
> line and rewrites it in new file.
>
> I have to make 2 http calls as part of the msg enrichment process, the
> first
> call is as authorization call that gets a json web token that is sent in
> the
> headers of the second http call.
>
> My route dsl looks something like this:
>
> from("{{consumerEndpoint}}")
> .validate(body().regex(FILE_HEADER_PATTERN))
> .split(body().tokenize())
> .choice()
> .when(body().regex(RECORD_LINE_PATTERN))
> .process(pluckOutTokenRecordProcessor())
> .setHeader(Exchange.HTTP_METHOD, constant("POST"))
> // call that I want its result cached for reuse
> .enrich("{{firstEnrichEndpoint}}",
> getFirstAggregationStrategy()) .setHeader(Exchange.HTTP_METHOD,
> constant("GET"))
>
> .enrich().simple("{{sndEnrichEndpoint}}").aggregationStrategy(
> getSndAggregationStrategy())
> .process(recordUpdaterProcessor())
> .end() //end choice
> .aggregate(header(FILE_NAME_ONLY),
> stringBodyAggregator())
> .convertBodyTo(String.class)
> .to("{{transferEndpoint}}")
> .end();
>
>
> It is all good and working, however, the file that gets processed in
> production has millions of lines and I do not want to have to make an extra
> authorization http call for each record that is processed, as the json web
> token is valid for 20 minutes before it expires which is less than what
> takes to process one entire file.
>
> I would like to keep the jwt auth token cached in memory for let's say 15
> minutes and refresh it periodically.
>
> I was wondering how is the best approach to achieve this goal. I know that
> camel ships with a timer component, but I never worked with it.
>
>
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.
> com/Cache-api-call-result-tp5789849.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>