You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by Miguel Ángel Fernández Fernández <mi...@gmail.com> on 2015/08/22 02:29:16 UTC
Spring / Trident (NotSerializableException: org.springframework)
I'm developing a project to testing the feasibility of combine Spring with
Trident (of course, I know https://github.com/granthenke/storm-spring )
In the beginning, I have no problem. As you can see (
https://github.com/miguelangelprogramacion/spring4-storm-cassandra), I
created a simple trident topology with an spout that emits continuous
values parsed from a file.
In the next level, I wanted to insert some information in cassandra. To
that end, I was inspired by https://github.com/hmsonline/storm-cassandra-cql,
but in my code
I'm trying to include the Spring CassandraTemplates.
The problem is that Spring framework objects are not serializable, and once
the topology is deployed, Storm seems to lost the Spring Objects (
java.io.NotSerializableException:
org.springframework.data.cassandra.core.CassandraTemplate)
Im triying to extend some Spring classes with serializable wrappers, but it
not seems to be a good idea.
Does anybody have a clue for this problem?
Re: Spring / Trident (NotSerializableException: org.springframework)
Posted by Miguel Ángel Fernández Fernández <mi...@gmail.com>.
OK, It was perfect.
I split the ComponentScan in two classes (one for the topology, one for
DAO).
My doubt now is if this "delayed" DI may result in duplicated objects
because I use this injection to initialize DAO classes in a state
(beginCommit), and for now I only have a State, but It is not the most
common situation
@Override
public void beginCommit(Long txid) {
if(cassandraOperations==null)
{ ApplicationContext context = new
AnnotationConfigApplicationContext(CassandraComponentScan.class);
DAO dao = context.getBean(DAO.class);
this.cassandraOperations = dao.getCassandraOperations();
}
}
Maybe, I should rethink how to use CassandraCqlStateFactory. For now, I
inject a State using DI, but I don't know if this is the better option.
2015-08-22 3:25 GMT+02:00 Javier Gonzalez <ja...@gmail.com>:
> We had issues with Spring and Storm. What we did is the following: don't
> do anything on the constructor. Perhaps pass a String with the location of
> the Spring configuration file. In the prepare or open method (for bolts and
> spouts respectively) initialize a context with the context file location
> and from there assign your Spring beans to the internal members of your
> spouts/bolts.
>
> Regards,
> Javier
>
> On Fri, Aug 21, 2015 at 8:29 PM, Miguel Ángel Fernández Fernández <
> miguelangelprogramacion@gmail.com> wrote:
>
>> I'm developing a project to testing the feasibility of combine Spring
>> with Trident (of course, I know
>> https://github.com/granthenke/storm-spring )
>>
>> In the beginning, I have no problem. As you can see (
>> https://github.com/miguelangelprogramacion/spring4-storm-cassandra), I
>> created a simple trident topology with an spout that emits continuous
>> values parsed from a file.
>>
>> In the next level, I wanted to insert some information in cassandra. To
>> that end, I was inspired by
>> https://github.com/hmsonline/storm-cassandra-cql, but in my code
>> I'm trying to include the Spring CassandraTemplates.
>>
>> The problem is that Spring framework objects are not serializable, and
>> once the topology is deployed, Storm seems to lost the Spring Objects (
>> java.io.NotSerializableException:
>> org.springframework.data.cassandra.core.CassandraTemplate)
>>
>> Im triying to extend some Spring classes with serializable wrappers, but
>> it not seems to be a good idea.
>>
>> Does anybody have a clue for this problem?
>>
>
>
>
> --
> Javier González Nicolini
>
Re: Spring / Trident (NotSerializableException: org.springframework)
Posted by Javier Gonzalez <ja...@gmail.com>.
We had issues with Spring and Storm. What we did is the following: don't do
anything on the constructor. Perhaps pass a String with the location of the
Spring configuration file. In the prepare or open method (for bolts and
spouts respectively) initialize a context with the context file location
and from there assign your Spring beans to the internal members of your
spouts/bolts.
Regards,
Javier
On Fri, Aug 21, 2015 at 8:29 PM, Miguel Ángel Fernández Fernández <
miguelangelprogramacion@gmail.com> wrote:
> I'm developing a project to testing the feasibility of combine Spring with
> Trident (of course, I know https://github.com/granthenke/storm-spring )
>
> In the beginning, I have no problem. As you can see (
> https://github.com/miguelangelprogramacion/spring4-storm-cassandra), I
> created a simple trident topology with an spout that emits continuous
> values parsed from a file.
>
> In the next level, I wanted to insert some information in cassandra. To
> that end, I was inspired by
> https://github.com/hmsonline/storm-cassandra-cql, but in my code
> I'm trying to include the Spring CassandraTemplates.
>
> The problem is that Spring framework objects are not serializable, and
> once the topology is deployed, Storm seems to lost the Spring Objects (
> java.io.NotSerializableException:
> org.springframework.data.cassandra.core.CassandraTemplate)
>
> Im triying to extend some Spring classes with serializable wrappers, but
> it not seems to be a good idea.
>
> Does anybody have a clue for this problem?
>
--
Javier González Nicolini