CyclicMetadataException

[Boyan Kolev <bk...@leanxcale.com>]

Hello,

I'm coming across this exception when Calcite prepares queries in parallel.
I see this happens if the same query is getting planned by two threads at
the same time,
but I also have the impression that this may also happen when the two
queries being planned in parallel are not exactly the same but have common
fragments of the logical plan.

Since this is hard to reproduce and debug as it happens rarely in a
multithreaded setup,
I'd like to have some workaround of the problem.

What would be a good practice?
For example, if the exception is caught and then the planning re-attempted
(a few times if needed),
would that significantly decrease the probability of running into that
exception?

Thanks in advance for any help!

--
Boyan Kolev

Re: CyclicMetadataException

[Julian Hyde <jh...@gmail.com>]

Planning is not thread-safe. For a given RelOptCluster, only one thread should be planning.

If there are multiple queries (each with their own RelOptCluster) then they can safely be planned on separate threads.

It’s possible that there are issues if queries are sharing materialized views. The RelNodes that define the materialized views are defined before the query is created, and therefore use a different RelOptCluster than the query, and potentially the same RelOptCluster as each other. If you think you are running into this, let’s dig deeper.

Julian
   

> On Feb 17, 2022, at 6:31 AM, Boyan Kolev <bk...@leanxcale.com> wrote:
> 
> Hello,
> 
> I'm coming across this exception when Calcite prepares queries in parallel.
> I see this happens if the same query is getting planned by two threads at
> the same time,
> but I also have the impression that this may also happen when the two
> queries being planned in parallel are not exactly the same but have common
> fragments of the logical plan.
> 
> Since this is hard to reproduce and debug as it happens rarely in a
> multithreaded setup,
> I'd like to have some workaround of the problem.
> 
> What would be a good practice?
> For example, if the exception is caught and then the planning re-attempted
> (a few times if needed),
> would that significantly decrease the probability of running into that
> exception?
> 
> Thanks in advance for any help!
> 
> --
> Boyan Kolev