You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by "RENDLE, ANDY (Insurance Finance Transformation Portfolio)" <An...@LloydsBanking.com> on 2020/11/24 19:49:41 UTC
Failing to cluster correctly
Classification: Public
All
We have developed a Spring Ignite Kafka producer application, utilising Ignites caches and failover capabilities.
This runs perfectly in standalone mode but when configured with another host we get many serialisation errors. We have obviously made some fundamental mistake, can anyone give us a clue as to where to look?
java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64
Ignite v2.7.6 & v2.9.0
spring-boot 2.0.6.RELEASE
Most of our processes are invoked like this:
ignite.compute().withExecutor(SCANNER_POOL).callAsync(IgniteCallable)
It seems to serializing many classes that are not expected, even when both nodes have exactly the same deployment. We have many Autowired variables but all are correct and working in standalone mode. In clustered mode and we end up with a huge exceptions in the file attached:
Many thanks in advance,
Andy Rendle
Lloyds Banking Group plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC95000. Telephone: 0131 225 4555.
Lloyds Bank plc. Registered Office: 25 Gresham Street, London EC2V 7HN. Registered in England and Wales no. 2065. Telephone 0207626 1500.
Bank of Scotland plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC327000. Telephone: 03457 801 801.
Lloyds Bank Corporate Markets plc. Registered office: 25 Gresham Street, London EC2V 7HN. Registered in England and Wales no. 10399850.
Scottish Widows Schroder Personal Wealth Limited. Registered Office: 25 Gresham Street, London EC2V 7HN. Registered in England and Wales no. 11722983.
Lloyds Bank plc, Bank of Scotland plc and Lloyds Bank Corporate Markets plc are authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and Prudential Regulation Authority.
Scottish Widows Schroder Personal Wealth Limited is authorised and regulated by the Financial Conduct Authority.
Lloyds Bank Corporate Markets Wertpapierhandelsbank GmbH is a wholly-owned subsidiary of Lloyds Bank Corporate Markets plc. Lloyds Bank Corporate Markets Wertpapierhandelsbank GmbH has its registered office at Thurn-und-Taxis Platz 6, 60313 Frankfurt, Germany. The company is registered with the Amtsgericht Frankfurt am Main, HRB 111650. Lloyds Bank Corporate Markets Wertpapierhandelsbank GmbH is supervised by the Bundesanstalt für Finanzdienstleistungsaufsicht.
Halifax is a division of Bank of Scotland plc.
HBOS plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC218813.
This e-mail (including any attachments) is private and confidential and may contain privileged material. If you have received this e-mail in error, please notify the sender and delete it (including any attachments) immediately. You must not copy, distribute, disclose or use any of the information in it or any attachments. Telephone calls may be monitored or recorded.
Re: Failing to cluster correctly
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I'm not sure. If you're using Spring you can also annotate such fields with
@SpringResource.
Regards,
--
Ilya Kasnacheev
пн, 30 нояб. 2020 г. в 13:35, RENDLE, ANDY (Insurance Finance
Transformation Portfolio) <An...@lloydsbanking.com>:
> Classification: Public
>
>
>
> Thanks for the update.
>
>
>
> Is there a Spring Ignite example available to demonstrate your suggested
> techniques?
>
>
>
> Thanks
>
>
>
> *Andy Rendle*
>
> Hadoop Technical Architect
>
> *Insurance Finance Transformation Portfolio | Group Transformation*
> ------------------------------
>
> *M: *07973 878454 *| E: andy.rendle@lloydsbanking.com
> <an...@lloydsbanking.com>*
>
> *A: *Lloyds Banking Group, Harbourside, 10 Canons Way, Bristol, BS1 5LF
>
> *MM: *0203 770 3333 *CP: *79299155# *PP: *77395220#
>
>
>
> *[image: cid:image001.png@01D327DF.B322F800]*
> <http://lbg.intranet.group/lbg_transformation/default.shtm>
>
> *Absence Note* :
>
>
>
> *From:* Ilya Kasnacheev <il...@gmail.com>
> *Sent:* 27 November 2020 10:02
> *To:* user@ignite.apache.org
> *Subject:* Re: Failing to cluster correctly
>
>
>
>
> *-- This email has reached the Bank via an external source -- *
>
> Hello!
>
>
>
> It seems that you are sending compute tasks from one node to another
> with kafkaEventProcessor field set. However, you can't really send a Kafka
> instance to a different node that way. You need to remove this field / mark
> as transient, and instead inject a local Kafka on remote node before doing
> computations. Maybe remove kafkaEventProcessor with
> static kafkaEventProcessor().
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> вт, 24 нояб. 2020 г. в 22:50, RENDLE, ANDY (Insurance Finance
> Transformation Portfolio) <An...@lloydsbanking.com>:
>
> Classification: Public
>
>
>
> All
>
>
>
> We have developed a Spring Ignite Kafka producer application, utilising
> Ignites caches and failover capabilities.
>
>
>
> This runs perfectly in standalone mode but when configured with another
> host we get many serialisation errors. We have obviously made some
> fundamental mistake, can anyone give us a clue as to where to look?
>
>
>
> java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64
>
> Ignite v2.7.6 & v2.9.0
>
> spring-boot 2.0.6.RELEASE
>
>
>
> Most of our processes are invoked like this:
>
> ignite.compute().withExecutor(SCANNER_POOL).callAsync(IgniteCallable)
>
>
>
> It seems to serializing many classes that are not expected, even when both
> nodes have exactly the same deployment. We have many Autowired variables
> but all are correct and working in standalone mode. In clustered mode and
> we end up with a huge exceptions in the file attached:
>
>
>
> Many thanks in advance,
>
>
>
> *Andy Rendle*
>
>
>
> Lloyds Banking Group plc. Registered Office: The Mound, Edinburgh EH1 1YZ.
> Registered in Scotland no. SC95000. Telephone: 0131 225 4555.
>
> Lloyds Bank plc. Registered Office: 25 Gresham Street, London EC2V 7HN.
> Registered in England and Wales no. 2065. Telephone 0207626 1500.
>
> Bank of Scotland plc. Registered Office: The Mound, Edinburgh EH1 1YZ.
> Registered in Scotland no. SC327000. Telephone: 03457 801 801.
>
> Lloyds Bank Corporate Markets plc. Registered office: 25 Gresham Street,
> London EC2V 7HN. Registered in England and Wales no. 10399850.
>
> Scottish Widows Schroder Personal Wealth Limited. Registered Office: 25
> Gresham Street, London EC2V 7HN. Registered in England and Wales no.
> 11722983.
>
> Lloyds Bank plc, Bank of Scotland plc and Lloyds Bank Corporate Markets
> plc are authorised by the Prudential Regulation Authority and regulated by
> the Financial Conduct Authority and Prudential Regulation Authority.
>
> Scottish Widows Schroder Personal Wealth Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> Lloyds Bank Corporate Markets Wertpapierhandelsbank GmbH is a wholly-owned
> subsidiary of Lloyds Bank Corporate Markets plc. Lloyds Bank Corporate
> Markets Wertpapierhandelsbank GmbH has its registered office at
> Thurn-und-Taxis Platz 6, 60313 Frankfurt, Germany. The company is
> registered with the Amtsgericht Frankfurt am Main, HRB 111650. Lloyds Bank
> Corporate Markets Wertpapierhandelsbank GmbH is supervised by the
> Bundesanstalt für Finanzdienstleistungsaufsicht.
>
> Halifax is a division of Bank of Scotland plc.
>
> HBOS plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in
> Scotland no. SC218813.
>
>
>
> This e-mail (including any attachments) is private and confidential and
> may contain privileged material. If you have received this e-mail in error,
> please notify the sender and delete it (including any attachments)
> immediately. You must not copy, distribute, disclose or use any of the
> information in it or any attachments. Telephone calls may be monitored or
> recorded.
>
> Lloyds Banking Group plc. Registered Office: The Mound, Edinburgh EH1 1YZ.
> Registered in Scotland no. SC95000. Telephone: 0131 225 4555.
>
> Lloyds Bank plc. Registered Office: 25 Gresham Street, London EC2V 7HN.
> Registered in England and Wales no. 2065. Telephone 0207626 1500.
>
> Bank of Scotland plc. Registered Office: The Mound, Edinburgh EH1 1YZ.
> Registered in Scotland no. SC327000. Telephone: 03457 801 801.
>
> Lloyds Bank Corporate Markets plc. Registered office: 25 Gresham Street,
> London EC2V 7HN. Registered in England and Wales no. 10399850.
>
> Scottish Widows Schroder Personal Wealth Limited. Registered Office: 25
> Gresham Street, London EC2V 7HN. Registered in England and Wales no.
> 11722983.
>
> Lloyds Bank plc, Bank of Scotland plc and Lloyds Bank Corporate Markets
> plc are authorised by the Prudential Regulation Authority and regulated by
> the Financial Conduct Authority and Prudential Regulation Authority.
>
> Scottish Widows Schroder Personal Wealth Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> Lloyds Bank Corporate Markets Wertpapierhandelsbank GmbH is a wholly-owned
> subsidiary of Lloyds Bank Corporate Markets plc. Lloyds Bank Corporate
> Markets Wertpapierhandelsbank GmbH has its registered office at
> Thurn-und-Taxis Platz 6, 60313 Frankfurt, Germany. The company is
> registered with the Amtsgericht Frankfurt am Main, HRB 111650. Lloyds Bank
> Corporate Markets Wertpapierhandelsbank GmbH is supervised by the
> Bundesanstalt für Finanzdienstleistungsaufsicht.
>
> Halifax is a division of Bank of Scotland plc.
>
> HBOS plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in
> Scotland no. SC218813.
>
> This e-mail (including any attachments) is private and confidential and
> may contain privileged material. If you have received this e-mail in error,
> please notify the sender and delete it (including any attachments)
> immediately. You must not copy, distribute, disclose or use any of the
> information in it or any attachments. Telephone calls may be monitored or
> recorded.
>
RE: Failing to cluster correctly
Posted by "RENDLE, ANDY (Insurance Finance Transformation Portfolio)" <An...@LloydsBanking.com>.
Classification: Public
Thanks for the update.
Is there a Spring Ignite example available to demonstrate your suggested techniques?
Thanks
Andy Rendle
Hadoop Technical Architect
Insurance Finance Transformation Portfolio | Group Transformation
________________________________
M: 07973 878454 | E: andy.rendle@lloydsbanking.com<ma...@lloydsbanking.com>
A: Lloyds Banking Group, Harbourside, 10 Canons Way, Bristol, BS1 5LF
MM: 0203 770 3333 CP: 79299155# PP: 77395220#
[cid:image001.png@01D327DF.B322F800]<http://lbg.intranet.group/lbg_transformation/default.shtm>
Absence Note :
From: Ilya Kasnacheev <il...@gmail.com>
Sent: 27 November 2020 10:02
To: user@ignite.apache.org
Subject: Re: Failing to cluster correctly
-- This email has reached the Bank via an external source --
Hello!
It seems that you are sending compute tasks from one node to another with kafkaEventProcessor field set. However, you can't really send a Kafka instance to a different node that way. You need to remove this field / mark as transient, and instead inject a local Kafka on remote node before doing computations. Maybe remove kafkaEventProcessor with static kafkaEventProcessor().
Regards,
--
Ilya Kasnacheev
вт, 24 нояб. 2020 г. в 22:50, RENDLE, ANDY (Insurance Finance Transformation Portfolio) <An...@lloydsbanking.com>>:
Classification: Public
All
We have developed a Spring Ignite Kafka producer application, utilising Ignites caches and failover capabilities.
This runs perfectly in standalone mode but when configured with another host we get many serialisation errors. We have obviously made some fundamental mistake, can anyone give us a clue as to where to look?
java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64
Ignite v2.7.6 & v2.9.0
spring-boot 2.0.6.RELEASE
Most of our processes are invoked like this:
ignite.compute().withExecutor(SCANNER_POOL).callAsync(IgniteCallable)
It seems to serializing many classes that are not expected, even when both nodes have exactly the same deployment. We have many Autowired variables but all are correct and working in standalone mode. In clustered mode and we end up with a huge exceptions in the file attached:
Many thanks in advance,
Andy Rendle
Lloyds Banking Group plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC95000. Telephone: 0131 225 4555.
Lloyds Bank plc. Registered Office: 25 Gresham Street, London EC2V 7HN. Registered in England and Wales no. 2065. Telephone 0207626 1500.
Bank of Scotland plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC327000. Telephone: 03457 801 801.
Lloyds Bank Corporate Markets plc. Registered office: 25 Gresham Street, London EC2V 7HN. Registered in England and Wales no. 10399850.
Scottish Widows Schroder Personal Wealth Limited. Registered Office: 25 Gresham Street, London EC2V 7HN. Registered in England and Wales no. 11722983.
Lloyds Bank plc, Bank of Scotland plc and Lloyds Bank Corporate Markets plc are authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and Prudential Regulation Authority.
Scottish Widows Schroder Personal Wealth Limited is authorised and regulated by the Financial Conduct Authority.
Lloyds Bank Corporate Markets Wertpapierhandelsbank GmbH is a wholly-owned subsidiary of Lloyds Bank Corporate Markets plc. Lloyds Bank Corporate Markets Wertpapierhandelsbank GmbH has its registered office at Thurn-und-Taxis Platz 6, 60313 Frankfurt, Germany. The company is registered with the Amtsgericht Frankfurt am Main, HRB 111650. Lloyds Bank Corporate Markets Wertpapierhandelsbank GmbH is supervised by the Bundesanstalt für Finanzdienstleistungsaufsicht.
Halifax is a division of Bank of Scotland plc.
HBOS plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC218813.
This e-mail (including any attachments) is private and confidential and may contain privileged material. If you have received this e-mail in error, please notify the sender and delete it (including any attachments) immediately. You must not copy, distribute, disclose or use any of the information in it or any attachments. Telephone calls may be monitored or recorded.
Lloyds Banking Group plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC95000. Telephone: 0131 225 4555.
Lloyds Bank plc. Registered Office: 25 Gresham Street, London EC2V 7HN. Registered in England and Wales no. 2065. Telephone 0207626 1500.
Bank of Scotland plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC327000. Telephone: 03457 801 801.
Lloyds Bank Corporate Markets plc. Registered office: 25 Gresham Street, London EC2V 7HN. Registered in England and Wales no. 10399850.
Scottish Widows Schroder Personal Wealth Limited. Registered Office: 25 Gresham Street, London EC2V 7HN. Registered in England and Wales no. 11722983.
Lloyds Bank plc, Bank of Scotland plc and Lloyds Bank Corporate Markets plc are authorised by the Prudential Regulation Authority and regulated by the Financial Conduct Authority and Prudential Regulation Authority.
Scottish Widows Schroder Personal Wealth Limited is authorised and regulated by the Financial Conduct Authority.
Lloyds Bank Corporate Markets Wertpapierhandelsbank GmbH is a wholly-owned subsidiary of Lloyds Bank Corporate Markets plc. Lloyds Bank Corporate Markets Wertpapierhandelsbank GmbH has its registered office at Thurn-und-Taxis Platz 6, 60313 Frankfurt, Germany. The company is registered with the Amtsgericht Frankfurt am Main, HRB 111650. Lloyds Bank Corporate Markets Wertpapierhandelsbank GmbH is supervised by the Bundesanstalt für Finanzdienstleistungsaufsicht.
Halifax is a division of Bank of Scotland plc.
HBOS plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in Scotland no. SC218813.
This e-mail (including any attachments) is private and confidential and may contain privileged material. If you have received this e-mail in error, please notify the sender and delete it (including any attachments) immediately. You must not copy, distribute, disclose or use any of the information in it or any attachments. Telephone calls may be monitored or recorded.
Re: Failing to cluster correctly
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
It seems that you are sending compute tasks from one node to another
with kafkaEventProcessor field set. However, you can't really send a Kafka
instance to a different node that way. You need to remove this field / mark
as transient, and instead inject a local Kafka on remote node before doing
computations. Maybe remove kafkaEventProcessor with
static kafkaEventProcessor().
Regards,
--
Ilya Kasnacheev
вт, 24 нояб. 2020 г. в 22:50, RENDLE, ANDY (Insurance Finance
Transformation Portfolio) <An...@lloydsbanking.com>:
> Classification: Public
>
> All
>
>
>
> We have developed a Spring Ignite Kafka producer application, utilising
> Ignites caches and failover capabilities.
>
>
>
> This runs perfectly in standalone mode but when configured with another
> host we get many serialisation errors. We have obviously made some
> fundamental mistake, can anyone give us a clue as to where to look?
>
>
>
> java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64
>
> Ignite v2.7.6 & v2.9.0
>
> spring-boot 2.0.6.RELEASE
>
>
>
> Most of our processes are invoked like this:
>
> ignite.compute().withExecutor(SCANNER_POOL).callAsync(IgniteCallable)
>
>
>
> It seems to serializing many classes that are not expected, even when both
> nodes have exactly the same deployment. We have many Autowired variables
> but all are correct and working in standalone mode. In clustered mode and
> we end up with a huge exceptions in the file attached:
>
>
>
> Many thanks in advance,
>
>
>
> *Andy Rendle*
>
>
>
> Lloyds Banking Group plc. Registered Office: The Mound, Edinburgh EH1 1YZ.
> Registered in Scotland no. SC95000. Telephone: 0131 225 4555.
>
> Lloyds Bank plc. Registered Office: 25 Gresham Street, London EC2V 7HN.
> Registered in England and Wales no. 2065. Telephone 0207626 1500.
>
> Bank of Scotland plc. Registered Office: The Mound, Edinburgh EH1 1YZ.
> Registered in Scotland no. SC327000. Telephone: 03457 801 801.
>
> Lloyds Bank Corporate Markets plc. Registered office: 25 Gresham Street,
> London EC2V 7HN. Registered in England and Wales no. 10399850.
>
> Scottish Widows Schroder Personal Wealth Limited. Registered Office: 25
> Gresham Street, London EC2V 7HN. Registered in England and Wales no.
> 11722983.
>
> Lloyds Bank plc, Bank of Scotland plc and Lloyds Bank Corporate Markets
> plc are authorised by the Prudential Regulation Authority and regulated by
> the Financial Conduct Authority and Prudential Regulation Authority.
>
> Scottish Widows Schroder Personal Wealth Limited is authorised and
> regulated by the Financial Conduct Authority.
>
> Lloyds Bank Corporate Markets Wertpapierhandelsbank GmbH is a wholly-owned
> subsidiary of Lloyds Bank Corporate Markets plc. Lloyds Bank Corporate
> Markets Wertpapierhandelsbank GmbH has its registered office at
> Thurn-und-Taxis Platz 6, 60313 Frankfurt, Germany. The company is
> registered with the Amtsgericht Frankfurt am Main, HRB 111650. Lloyds Bank
> Corporate Markets Wertpapierhandelsbank GmbH is supervised by the
> Bundesanstalt für Finanzdienstleistungsaufsicht.
>
> Halifax is a division of Bank of Scotland plc.
>
> HBOS plc. Registered Office: The Mound, Edinburgh EH1 1YZ. Registered in
> Scotland no. SC218813.
>
> This e-mail (including any attachments) is private and confidential and
> may contain privileged material. If you have received this e-mail in error,
> please notify the sender and delete it (including any attachments)
> immediately. You must not copy, distribute, disclose or use any of the
> information in it or any attachments. Telephone calls may be monitored or
> recorded.
>