You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by Gavin Ray <ra...@gmail.com> on 2021/12/31 22:20:23 UTC
Trying to call "PreparedStatement.run()" throws "Cannot invoke "calcite.schema.Schema.getTable(String)" because the return value of "calcite.schema.SchemaPlus.getSubSchema(String)" is null"
Unsure of where this is coming from/how to debug it.
(The code I am using is below, taken from Calcite examples)
If I print out the sub-schema names of "rootSchema", and their tables, I
get:
subSchemaNames: [hr, metadata]
"hr" subSchema table names: [depts, emps]
"metadata" subSchema table names: [COLUMNS, TABLES]
So I am not certain what value is being passed that it would fail
a "schema.getSubSchema()" call, or where to put a breakpoint to debug that.
Does anyone spot something I'm doing that's clearly wrong, or have an idea
where I could look to find the code that's causing this?
Stacktrace winds up at "Baz", so I think it's generated code
Thank you
===========================================================
fun main(args: Array<String>) {
val hr = Frameworks.createRootSchema(true).add("hr",
HrClusteredSchema())
val frameworkConfig: FrameworkConfig = Frameworks
.newConfigBuilder()
.parserConfig(SqlParser.config().withCaseSensitive(false))
.defaultSchema(hr)
.build()
Frameworks.withPlanner(::test, frameworkConfig)
}
fun test(cluster: RelOptCluster, relOptSchema: RelOptSchema, rootSchema:
Schema) {
val planner = cluster.planner
val relBuilder: RelBuilder =
RelFactories.LOGICAL_BUILDER.create(cluster, relOptSchema)
val relRoot: RelNode = // ...
val desiredTraits =
cluster.traitSet().replace(EnumerableConvention.INSTANCE)
val newRoot = cluster.planner.changeTraits(relRoot, desiredTraits)
cluster.planner.root = newRoot
val bestExp = cluster.planner.findBestExp()
val run = RelRunners.run(bestExp)
val resultSet = run.executeQuery()
}
Re: Trying to call "PreparedStatement.run()" throws "Cannot invoke "calcite.schema.Schema.getTable(String)" because the return value of "calcite.schema.SchemaPlus.getSubSchema(String)" is null"
Posted by Stamatis Zampetakis <za...@gmail.com>.
Hi Gavin,
Although the problem was not related to generated code in the end it is
useful to know that you can debug into generated classes by setting some
system properties [1].
You can also use -Dcalcite.debug property which prints generated code among
other things to stdout.
You may have found these already but I am mentioning it here since people
new to the project sometimes have a rough time debugging problems in janino
since they are not aware of these properties.
Best,
Stamatis
[1]
https://calcite.apache.org/docs/howto.html#debugging-generated-classes-in-intellij
On Sun, Jan 2, 2022 at 7:39 PM Gavin Ray <ra...@gmail.com> wrote:
> Ahh, thank you Julian, appreciate that you're always answering my
> questions.
>
> That all makes sense. I suppose the downside of using these incredibly
> handy
> test helpers/utils is that I haven't a clue what's actually going on, haha.
>
> Will try to figure out how to manually perform the process of creating a
> proper "Connection" object and adding the schemas to it.
>
> On the bright side, this was the very last blocker to having a prototype of
> my
> project working end-to-end!
>
> Took me almost a month and I had to ask for a lot of help, but I'm just
> ecstatic
> that it works =D
>
> On Sun, Jan 2, 2022 at 1:16 AM Julian Hyde <jh...@gmail.com> wrote:
>
> > Gavin,
> >
> > The reason is that you need two “ingredients” to execute a query. You
> need
> > the expression, and you need a connection. The expression is written in
> > terms of relations, which have a fully-qualified path,
> > “schema.subSchema.tableName”. But only the connection (to be precise, its
> > root schema) contains the actual table objects.
> >
> > Usually you get a RelRunner from a connection. RelRunners.run seems to
> > offer a short-cut — it creates a dummy connection using “jdbc:calcite:”.
> > But you get what you pay for. This connection is empty. Its root schema
> > contains no sub-schemas, and no tables. It’s only good for executing
> > statements that do not reference any tables.
> >
> > Sorry to hear that you had two days of frustration.
> >
> > Julian
> >
> >
> >
> > > On Jan 1, 2022, at 10:44 AM, Gavin Ray <ra...@gmail.com> wrote:
> > >
> > > After two days of banging my head against the wall, I figured it out!
> > >
> > > It only works if I run the query in the context of a
> "CalciteConnection",
> > > and I use "connection.unwrap()" to create the "RelRunner".
> > >
> > > WRONG:
> > > ==========================
> > > val run = RelRunners.run(bestExp)
> > > val resultSet = run.executeQuery()
> > >
> > > RIGHT:
> > > ==========================
> > > val runner = connection.unwrap(RelRunner::class.java)
> > > val resultSet = runner.prepareStatement(bestExp).executeQuery()
> > >
> > > Why is this out of curiosity?
> > >
> > > I only figured this out by mimicking the "FrameworksTest" case below,
> > > so I don't understand the "why".
> > >
> > >
> >
> https://github.com/apache/calcite/blob/de847c38f3544f9c7282984f32dc1093bdb2fb60/core/src/test/java/org/apache/calcite/tools/FrameworksTest.java#L342-L376
> > >
> > >
> > >
> > > On Fri, Dec 31, 2021 at 6:22 PM Gavin Ray <ra...@gmail.com>
> wrote:
> > >
> > >> It appears to be because "getSubSchema" in
> > >> "calcite.jdbc.CalciteSchema.SchemaPlusImpl" has a "subSchemaMap" that
> > loses
> > >> the "hr" schema and only contains "metadata" [0]
> > >>
> > >> But if I call the same function the map shows "hr" is present [1]
> which
> > >> doesn't make sense to me
> > >>
> > >> [0]: https://imgur.com/djlkBVf
> > >> [1]: https://imgur.com/eEOvjdS
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> On Fri, Dec 31, 2021 at 5:20 PM Gavin Ray <ra...@gmail.com>
> > wrote:
> > >>
> > >>> Unsure of where this is coming from/how to debug it.
> > >>> (The code I am using is below, taken from Calcite examples)
> > >>>
> > >>> If I print out the sub-schema names of "rootSchema", and their
> tables,
> > I
> > >>> get:
> > >>>
> > >>> subSchemaNames: [hr, metadata]
> > >>> "hr" subSchema table names: [depts, emps]
> > >>> "metadata" subSchema table names: [COLUMNS, TABLES]
> > >>>
> > >>> So I am not certain what value is being passed that it would fail
> > >>> a "schema.getSubSchema()" call, or where to put a breakpoint to debug
> > >>> that.
> > >>>
> > >>> Does anyone spot something I'm doing that's clearly wrong, or have an
> > idea
> > >>> where I could look to find the code that's causing this?
> > >>>
> > >>> Stacktrace winds up at "Baz", so I think it's generated code
> > >>> Thank you
> > >>> ===========================================================
> > >>>
> > >>> fun main(args: Array<String>) {
> > >>> val hr = Frameworks.createRootSchema(true).add("hr",
> > >>> HrClusteredSchema())
> > >>>
> > >>> val frameworkConfig: FrameworkConfig = Frameworks
> > >>> .newConfigBuilder()
> > >>> .parserConfig(SqlParser.config().withCaseSensitive(false))
> > >>> .defaultSchema(hr)
> > >>> .build()
> > >>>
> > >>> Frameworks.withPlanner(::test, frameworkConfig)
> > >>> }
> > >>>
> > >>> fun test(cluster: RelOptCluster, relOptSchema: RelOptSchema,
> > rootSchema:
> > >>> Schema) {
> > >>> val planner = cluster.planner
> > >>> val relBuilder: RelBuilder =
> > >>> RelFactories.LOGICAL_BUILDER.create(cluster, relOptSchema)
> > >>>
> > >>> val relRoot: RelNode = // ...
> > >>> val desiredTraits =
> > >>> cluster.traitSet().replace(EnumerableConvention.INSTANCE)
> > >>> val newRoot = cluster.planner.changeTraits(relRoot, desiredTraits)
> > >>>
> > >>> cluster.planner.root = newRoot
> > >>> val bestExp = cluster.planner.findBestExp()
> > >>> val run = RelRunners.run(bestExp)
> > >>> val resultSet = run.executeQuery()
> > >>> }
> > >>>
> > >>
> >
> >
>
Re: Trying to call "PreparedStatement.run()" throws "Cannot invoke "calcite.schema.Schema.getTable(String)" because the return value of "calcite.schema.SchemaPlus.getSubSchema(String)" is null"
Posted by Gavin Ray <ra...@gmail.com>.
For posterity's sake and future-searchers, I just hit this error again
under different circumstances and wanted to share why here.
The value of the second parameter ("name") to "JdbcSchema.create()" is
apparently very important.
Somehow I was passing something that was causing it to look for things that
didn't exist during query execution.
If you hit this error, check that:
1) You are getting a RelRunner from the CalciteConnection, so it can
actually execute
2) You are passing valid values to "name" in JdbcSchema.create()
I'm not sure what the extent/purpose of the "name" value is, but it's been
related to two things I've hit now, so it's got to be critical.
On Mon, Jan 3, 2022 at 3:44 PM Gavin Ray <ra...@gmail.com> wrote:
> Ahhh -- thanks for the code sample Gunnar and the clarification that you
> can do it two ways Julian.
>
> Data-source metadata updates should be infrequent
> I can try to use a mutex/arc for the global SchemaPlus
>
> Think it would be really beneficial to have a single schema tree that can
> see every data source, but
> will try both ways just to be sure
>
>
> On Mon, Jan 3, 2022 at 3:28 PM Julian Hyde <jh...@gmail.com> wrote:
>
>> Gunnar’s point is a good one. To make it a little more explicit: if the
>> value of the the “model” parameter starts with “inline:”, the model is just
>> the rest of the string.
>>
>> As to whether you use models or deal with the SchemaPlus API directly.
>> It’s your decision based on how much concurrency control you want to do. If
>> you use the SchemaPlus API, you can use the same schema tree for successive
>> requests, and even for concurrent requests. But you’d better be sure that
>> one request is not mutating the tree while another request is using it.
>> Generating a new model string (and hence a new schema tree) for each
>> request is a little less efficient but much safer.
>>
>>
>> > On Jan 3, 2022, at 12:20 PM, Gunnar Morling
>> <gu...@googlemail.com.INVALID> wrote:
>> >
>> > Am Mo., 3. Jan. 2022 um 20:44 Uhr schrieb Gavin Ray <
>> ray.gavin97@gmail.com <ma...@gmail.com>>:
>> >
>> >> @Stamatis
>> >>
>> >> Thank you, I actually did not know that! That's useful information to
>> >> have =)
>> >>
>> >> @Julian
>> >>
>> >> Is it possible to programmatically generate the JSON model? Why I ask
>> is
>> >> because part of the functionality I'm trying to mirror is the ability
>> to
>> >> add
>> >> data sources dynamically while the app server is running
>> >>
>> >> So you might send a POST /metadata/datasource saying "Here is the info
>> of a
>> >> new
>> >> data source I would like you to connect to" And then I need to figure
>> out
>> >> how to
>> >> handle creating a Calcite source from it and adding the new datasource
>> +
>> >> schemas
>> >> to the global pool of registered ones, if that makes sense
>> >>
>> >> Worst case I could write the JSON file to disk and load from the
>> tempfile I
>> >> suppose, right?
>> >>
>> >
>> > No need for writing to disk, you just can pass in a String representing
>> the
>> > JSON model. Here's an example:
>> >
>> >
>> >
>> https://github.com/moditect/jfr-analytics/blob/main/src/test/java/org/moditect/jfranalytics/JfrSchemaFactoryTest.java#L381-L388
>> <
>> https://github.com/moditect/jfr-analytics/blob/main/src/test/java/org/moditect/jfranalytics/JfrSchemaFactoryTest.java#L381-L388
>> >
>> >
>> > Hth,
>> >
>> > --Gunnar
>> >
>> >
>> >>
>> >>
>> >>
>> >> On Mon, Jan 3, 2022 at 1:30 PM Julian Hyde <jh...@gmail.com>
>> wrote:
>> >>
>> >>>> Will try to figure out how to manually perform the process of
>> creating
>> >> a
>> >>>> proper "Connection" object and adding the schemas to it.
>> >>>
>> >>>
>> >>> Generally the best way is to write a JSON model, and then create a
>> >>> connection using ‘jdbc:calcite:…model=...'. For that to work, your
>> >>> component needs to implement ‘interface SchemaFactory’, i.e. be an
>> >> adapter.
>> >>>
>> >>> Julian
>>
>>
Re: Trying to call "PreparedStatement.run()" throws "Cannot invoke "calcite.schema.Schema.getTable(String)" because the return value of "calcite.schema.SchemaPlus.getSubSchema(String)" is null"
Posted by Gavin Ray <ra...@gmail.com>.
Ahhh -- thanks for the code sample Gunnar and the clarification that you
can do it two ways Julian.
Data-source metadata updates should be infrequent
I can try to use a mutex/arc for the global SchemaPlus
Think it would be really beneficial to have a single schema tree that can
see every data source, but
will try both ways just to be sure
On Mon, Jan 3, 2022 at 3:28 PM Julian Hyde <jh...@gmail.com> wrote:
> Gunnar’s point is a good one. To make it a little more explicit: if the
> value of the the “model” parameter starts with “inline:”, the model is just
> the rest of the string.
>
> As to whether you use models or deal with the SchemaPlus API directly.
> It’s your decision based on how much concurrency control you want to do. If
> you use the SchemaPlus API, you can use the same schema tree for successive
> requests, and even for concurrent requests. But you’d better be sure that
> one request is not mutating the tree while another request is using it.
> Generating a new model string (and hence a new schema tree) for each
> request is a little less efficient but much safer.
>
>
> > On Jan 3, 2022, at 12:20 PM, Gunnar Morling
> <gu...@googlemail.com.INVALID> wrote:
> >
> > Am Mo., 3. Jan. 2022 um 20:44 Uhr schrieb Gavin Ray <
> ray.gavin97@gmail.com <ma...@gmail.com>>:
> >
> >> @Stamatis
> >>
> >> Thank you, I actually did not know that! That's useful information to
> >> have =)
> >>
> >> @Julian
> >>
> >> Is it possible to programmatically generate the JSON model? Why I ask is
> >> because part of the functionality I'm trying to mirror is the ability to
> >> add
> >> data sources dynamically while the app server is running
> >>
> >> So you might send a POST /metadata/datasource saying "Here is the info
> of a
> >> new
> >> data source I would like you to connect to" And then I need to figure
> out
> >> how to
> >> handle creating a Calcite source from it and adding the new datasource +
> >> schemas
> >> to the global pool of registered ones, if that makes sense
> >>
> >> Worst case I could write the JSON file to disk and load from the
> tempfile I
> >> suppose, right?
> >>
> >
> > No need for writing to disk, you just can pass in a String representing
> the
> > JSON model. Here's an example:
> >
> >
> >
> https://github.com/moditect/jfr-analytics/blob/main/src/test/java/org/moditect/jfranalytics/JfrSchemaFactoryTest.java#L381-L388
> <
> https://github.com/moditect/jfr-analytics/blob/main/src/test/java/org/moditect/jfranalytics/JfrSchemaFactoryTest.java#L381-L388
> >
> >
> > Hth,
> >
> > --Gunnar
> >
> >
> >>
> >>
> >>
> >> On Mon, Jan 3, 2022 at 1:30 PM Julian Hyde <jh...@gmail.com>
> wrote:
> >>
> >>>> Will try to figure out how to manually perform the process of creating
> >> a
> >>>> proper "Connection" object and adding the schemas to it.
> >>>
> >>>
> >>> Generally the best way is to write a JSON model, and then create a
> >>> connection using ‘jdbc:calcite:…model=...'. For that to work, your
> >>> component needs to implement ‘interface SchemaFactory’, i.e. be an
> >> adapter.
> >>>
> >>> Julian
>
>
Re: Trying to call "PreparedStatement.run()" throws "Cannot invoke "calcite.schema.Schema.getTable(String)" because the return value of "calcite.schema.SchemaPlus.getSubSchema(String)" is null"
Posted by Julian Hyde <jh...@gmail.com>.
Gunnar’s point is a good one. To make it a little more explicit: if the value of the the “model” parameter starts with “inline:”, the model is just the rest of the string.
As to whether you use models or deal with the SchemaPlus API directly. It’s your decision based on how much concurrency control you want to do. If you use the SchemaPlus API, you can use the same schema tree for successive requests, and even for concurrent requests. But you’d better be sure that one request is not mutating the tree while another request is using it. Generating a new model string (and hence a new schema tree) for each request is a little less efficient but much safer.
> On Jan 3, 2022, at 12:20 PM, Gunnar Morling <gu...@googlemail.com.INVALID> wrote:
>
> Am Mo., 3. Jan. 2022 um 20:44 Uhr schrieb Gavin Ray <ray.gavin97@gmail.com <ma...@gmail.com>>:
>
>> @Stamatis
>>
>> Thank you, I actually did not know that! That's useful information to
>> have =)
>>
>> @Julian
>>
>> Is it possible to programmatically generate the JSON model? Why I ask is
>> because part of the functionality I'm trying to mirror is the ability to
>> add
>> data sources dynamically while the app server is running
>>
>> So you might send a POST /metadata/datasource saying "Here is the info of a
>> new
>> data source I would like you to connect to" And then I need to figure out
>> how to
>> handle creating a Calcite source from it and adding the new datasource +
>> schemas
>> to the global pool of registered ones, if that makes sense
>>
>> Worst case I could write the JSON file to disk and load from the tempfile I
>> suppose, right?
>>
>
> No need for writing to disk, you just can pass in a String representing the
> JSON model. Here's an example:
>
>
> https://github.com/moditect/jfr-analytics/blob/main/src/test/java/org/moditect/jfranalytics/JfrSchemaFactoryTest.java#L381-L388 <https://github.com/moditect/jfr-analytics/blob/main/src/test/java/org/moditect/jfranalytics/JfrSchemaFactoryTest.java#L381-L388>
>
> Hth,
>
> --Gunnar
>
>
>>
>>
>>
>> On Mon, Jan 3, 2022 at 1:30 PM Julian Hyde <jh...@gmail.com> wrote:
>>
>>>> Will try to figure out how to manually perform the process of creating
>> a
>>>> proper "Connection" object and adding the schemas to it.
>>>
>>>
>>> Generally the best way is to write a JSON model, and then create a
>>> connection using ‘jdbc:calcite:…model=...'. For that to work, your
>>> component needs to implement ‘interface SchemaFactory’, i.e. be an
>> adapter.
>>>
>>> Julian
Re: Trying to call "PreparedStatement.run()" throws "Cannot invoke "calcite.schema.Schema.getTable(String)" because the return value of "calcite.schema.SchemaPlus.getSubSchema(String)" is null"
Posted by Gunnar Morling <gu...@googlemail.com.INVALID>.
Am Mo., 3. Jan. 2022 um 20:44 Uhr schrieb Gavin Ray <ra...@gmail.com>:
> @Stamatis
>
> Thank you, I actually did not know that! That's useful information to
> have =)
>
> @Julian
>
> Is it possible to programmatically generate the JSON model? Why I ask is
> because part of the functionality I'm trying to mirror is the ability to
> add
> data sources dynamically while the app server is running
>
> So you might send a POST /metadata/datasource saying "Here is the info of a
> new
> data source I would like you to connect to" And then I need to figure out
> how to
> handle creating a Calcite source from it and adding the new datasource +
> schemas
> to the global pool of registered ones, if that makes sense
>
> Worst case I could write the JSON file to disk and load from the tempfile I
> suppose, right?
>
No need for writing to disk, you just can pass in a String representing the
JSON model. Here's an example:
https://github.com/moditect/jfr-analytics/blob/main/src/test/java/org/moditect/jfranalytics/JfrSchemaFactoryTest.java#L381-L388
Hth,
--Gunnar
>
>
>
> On Mon, Jan 3, 2022 at 1:30 PM Julian Hyde <jh...@gmail.com> wrote:
>
> > > Will try to figure out how to manually perform the process of creating
> a
> > > proper "Connection" object and adding the schemas to it.
> >
> >
> > Generally the best way is to write a JSON model, and then create a
> > connection using ‘jdbc:calcite:…model=...'. For that to work, your
> > component needs to implement ‘interface SchemaFactory’, i.e. be an
> adapter.
> >
> > Julian
> >
> >
>
Re: Trying to call "PreparedStatement.run()" throws "Cannot invoke "calcite.schema.Schema.getTable(String)" because the return value of "calcite.schema.SchemaPlus.getSubSchema(String)" is null"
Posted by Gavin Ray <ra...@gmail.com>.
@Stamatis
Thank you, I actually did not know that! That's useful information to
have =)
@Julian
Is it possible to programmatically generate the JSON model? Why I ask is
because part of the functionality I'm trying to mirror is the ability to add
data sources dynamically while the app server is running
So you might send a POST /metadata/datasource saying "Here is the info of a
new
data source I would like you to connect to" And then I need to figure out
how to
handle creating a Calcite source from it and adding the new datasource +
schemas
to the global pool of registered ones, if that makes sense
Worst case I could write the JSON file to disk and load from the tempfile I
suppose, right?
On Mon, Jan 3, 2022 at 1:30 PM Julian Hyde <jh...@gmail.com> wrote:
> > Will try to figure out how to manually perform the process of creating a
> > proper "Connection" object and adding the schemas to it.
>
>
> Generally the best way is to write a JSON model, and then create a
> connection using ‘jdbc:calcite:…model=...'. For that to work, your
> component needs to implement ‘interface SchemaFactory’, i.e. be an adapter.
>
> Julian
>
>
Re: Trying to call "PreparedStatement.run()" throws "Cannot invoke "calcite.schema.Schema.getTable(String)" because the return value of "calcite.schema.SchemaPlus.getSubSchema(String)" is null"
Posted by Julian Hyde <jh...@gmail.com>.
> Will try to figure out how to manually perform the process of creating a
> proper "Connection" object and adding the schemas to it.
Generally the best way is to write a JSON model, and then create a connection using ‘jdbc:calcite:…model=...'. For that to work, your component needs to implement ‘interface SchemaFactory’, i.e. be an adapter.
Julian
Re: Trying to call "PreparedStatement.run()" throws "Cannot invoke "calcite.schema.Schema.getTable(String)" because the return value of "calcite.schema.SchemaPlus.getSubSchema(String)" is null"
Posted by Gavin Ray <ra...@gmail.com>.
Ahh, thank you Julian, appreciate that you're always answering my questions.
That all makes sense. I suppose the downside of using these incredibly handy
test helpers/utils is that I haven't a clue what's actually going on, haha.
Will try to figure out how to manually perform the process of creating a
proper "Connection" object and adding the schemas to it.
On the bright side, this was the very last blocker to having a prototype of
my
project working end-to-end!
Took me almost a month and I had to ask for a lot of help, but I'm just
ecstatic
that it works =D
On Sun, Jan 2, 2022 at 1:16 AM Julian Hyde <jh...@gmail.com> wrote:
> Gavin,
>
> The reason is that you need two “ingredients” to execute a query. You need
> the expression, and you need a connection. The expression is written in
> terms of relations, which have a fully-qualified path,
> “schema.subSchema.tableName”. But only the connection (to be precise, its
> root schema) contains the actual table objects.
>
> Usually you get a RelRunner from a connection. RelRunners.run seems to
> offer a short-cut — it creates a dummy connection using “jdbc:calcite:”.
> But you get what you pay for. This connection is empty. Its root schema
> contains no sub-schemas, and no tables. It’s only good for executing
> statements that do not reference any tables.
>
> Sorry to hear that you had two days of frustration.
>
> Julian
>
>
>
> > On Jan 1, 2022, at 10:44 AM, Gavin Ray <ra...@gmail.com> wrote:
> >
> > After two days of banging my head against the wall, I figured it out!
> >
> > It only works if I run the query in the context of a "CalciteConnection",
> > and I use "connection.unwrap()" to create the "RelRunner".
> >
> > WRONG:
> > ==========================
> > val run = RelRunners.run(bestExp)
> > val resultSet = run.executeQuery()
> >
> > RIGHT:
> > ==========================
> > val runner = connection.unwrap(RelRunner::class.java)
> > val resultSet = runner.prepareStatement(bestExp).executeQuery()
> >
> > Why is this out of curiosity?
> >
> > I only figured this out by mimicking the "FrameworksTest" case below,
> > so I don't understand the "why".
> >
> >
> https://github.com/apache/calcite/blob/de847c38f3544f9c7282984f32dc1093bdb2fb60/core/src/test/java/org/apache/calcite/tools/FrameworksTest.java#L342-L376
> >
> >
> >
> > On Fri, Dec 31, 2021 at 6:22 PM Gavin Ray <ra...@gmail.com> wrote:
> >
> >> It appears to be because "getSubSchema" in
> >> "calcite.jdbc.CalciteSchema.SchemaPlusImpl" has a "subSchemaMap" that
> loses
> >> the "hr" schema and only contains "metadata" [0]
> >>
> >> But if I call the same function the map shows "hr" is present [1] which
> >> doesn't make sense to me
> >>
> >> [0]: https://imgur.com/djlkBVf
> >> [1]: https://imgur.com/eEOvjdS
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Fri, Dec 31, 2021 at 5:20 PM Gavin Ray <ra...@gmail.com>
> wrote:
> >>
> >>> Unsure of where this is coming from/how to debug it.
> >>> (The code I am using is below, taken from Calcite examples)
> >>>
> >>> If I print out the sub-schema names of "rootSchema", and their tables,
> I
> >>> get:
> >>>
> >>> subSchemaNames: [hr, metadata]
> >>> "hr" subSchema table names: [depts, emps]
> >>> "metadata" subSchema table names: [COLUMNS, TABLES]
> >>>
> >>> So I am not certain what value is being passed that it would fail
> >>> a "schema.getSubSchema()" call, or where to put a breakpoint to debug
> >>> that.
> >>>
> >>> Does anyone spot something I'm doing that's clearly wrong, or have an
> idea
> >>> where I could look to find the code that's causing this?
> >>>
> >>> Stacktrace winds up at "Baz", so I think it's generated code
> >>> Thank you
> >>> ===========================================================
> >>>
> >>> fun main(args: Array<String>) {
> >>> val hr = Frameworks.createRootSchema(true).add("hr",
> >>> HrClusteredSchema())
> >>>
> >>> val frameworkConfig: FrameworkConfig = Frameworks
> >>> .newConfigBuilder()
> >>> .parserConfig(SqlParser.config().withCaseSensitive(false))
> >>> .defaultSchema(hr)
> >>> .build()
> >>>
> >>> Frameworks.withPlanner(::test, frameworkConfig)
> >>> }
> >>>
> >>> fun test(cluster: RelOptCluster, relOptSchema: RelOptSchema,
> rootSchema:
> >>> Schema) {
> >>> val planner = cluster.planner
> >>> val relBuilder: RelBuilder =
> >>> RelFactories.LOGICAL_BUILDER.create(cluster, relOptSchema)
> >>>
> >>> val relRoot: RelNode = // ...
> >>> val desiredTraits =
> >>> cluster.traitSet().replace(EnumerableConvention.INSTANCE)
> >>> val newRoot = cluster.planner.changeTraits(relRoot, desiredTraits)
> >>>
> >>> cluster.planner.root = newRoot
> >>> val bestExp = cluster.planner.findBestExp()
> >>> val run = RelRunners.run(bestExp)
> >>> val resultSet = run.executeQuery()
> >>> }
> >>>
> >>
>
>
Re: Trying to call "PreparedStatement.run()" throws "Cannot invoke "calcite.schema.Schema.getTable(String)" because the return value of "calcite.schema.SchemaPlus.getSubSchema(String)" is null"
Posted by Julian Hyde <jh...@gmail.com>.
Gavin,
The reason is that you need two “ingredients” to execute a query. You need the expression, and you need a connection. The expression is written in terms of relations, which have a fully-qualified path, “schema.subSchema.tableName”. But only the connection (to be precise, its root schema) contains the actual table objects.
Usually you get a RelRunner from a connection. RelRunners.run seems to offer a short-cut — it creates a dummy connection using “jdbc:calcite:”. But you get what you pay for. This connection is empty. Its root schema contains no sub-schemas, and no tables. It’s only good for executing statements that do not reference any tables.
Sorry to hear that you had two days of frustration.
Julian
> On Jan 1, 2022, at 10:44 AM, Gavin Ray <ra...@gmail.com> wrote:
>
> After two days of banging my head against the wall, I figured it out!
>
> It only works if I run the query in the context of a "CalciteConnection",
> and I use "connection.unwrap()" to create the "RelRunner".
>
> WRONG:
> ==========================
> val run = RelRunners.run(bestExp)
> val resultSet = run.executeQuery()
>
> RIGHT:
> ==========================
> val runner = connection.unwrap(RelRunner::class.java)
> val resultSet = runner.prepareStatement(bestExp).executeQuery()
>
> Why is this out of curiosity?
>
> I only figured this out by mimicking the "FrameworksTest" case below,
> so I don't understand the "why".
>
> https://github.com/apache/calcite/blob/de847c38f3544f9c7282984f32dc1093bdb2fb60/core/src/test/java/org/apache/calcite/tools/FrameworksTest.java#L342-L376
>
>
>
> On Fri, Dec 31, 2021 at 6:22 PM Gavin Ray <ra...@gmail.com> wrote:
>
>> It appears to be because "getSubSchema" in
>> "calcite.jdbc.CalciteSchema.SchemaPlusImpl" has a "subSchemaMap" that loses
>> the "hr" schema and only contains "metadata" [0]
>>
>> But if I call the same function the map shows "hr" is present [1] which
>> doesn't make sense to me
>>
>> [0]: https://imgur.com/djlkBVf
>> [1]: https://imgur.com/eEOvjdS
>>
>>
>>
>>
>>
>>
>>
>> On Fri, Dec 31, 2021 at 5:20 PM Gavin Ray <ra...@gmail.com> wrote:
>>
>>> Unsure of where this is coming from/how to debug it.
>>> (The code I am using is below, taken from Calcite examples)
>>>
>>> If I print out the sub-schema names of "rootSchema", and their tables, I
>>> get:
>>>
>>> subSchemaNames: [hr, metadata]
>>> "hr" subSchema table names: [depts, emps]
>>> "metadata" subSchema table names: [COLUMNS, TABLES]
>>>
>>> So I am not certain what value is being passed that it would fail
>>> a "schema.getSubSchema()" call, or where to put a breakpoint to debug
>>> that.
>>>
>>> Does anyone spot something I'm doing that's clearly wrong, or have an idea
>>> where I could look to find the code that's causing this?
>>>
>>> Stacktrace winds up at "Baz", so I think it's generated code
>>> Thank you
>>> ===========================================================
>>>
>>> fun main(args: Array<String>) {
>>> val hr = Frameworks.createRootSchema(true).add("hr",
>>> HrClusteredSchema())
>>>
>>> val frameworkConfig: FrameworkConfig = Frameworks
>>> .newConfigBuilder()
>>> .parserConfig(SqlParser.config().withCaseSensitive(false))
>>> .defaultSchema(hr)
>>> .build()
>>>
>>> Frameworks.withPlanner(::test, frameworkConfig)
>>> }
>>>
>>> fun test(cluster: RelOptCluster, relOptSchema: RelOptSchema, rootSchema:
>>> Schema) {
>>> val planner = cluster.planner
>>> val relBuilder: RelBuilder =
>>> RelFactories.LOGICAL_BUILDER.create(cluster, relOptSchema)
>>>
>>> val relRoot: RelNode = // ...
>>> val desiredTraits =
>>> cluster.traitSet().replace(EnumerableConvention.INSTANCE)
>>> val newRoot = cluster.planner.changeTraits(relRoot, desiredTraits)
>>>
>>> cluster.planner.root = newRoot
>>> val bestExp = cluster.planner.findBestExp()
>>> val run = RelRunners.run(bestExp)
>>> val resultSet = run.executeQuery()
>>> }
>>>
>>
Re: Trying to call "PreparedStatement.run()" throws "Cannot invoke "calcite.schema.Schema.getTable(String)" because the return value of "calcite.schema.SchemaPlus.getSubSchema(String)" is null"
Posted by Gavin Ray <ra...@gmail.com>.
After two days of banging my head against the wall, I figured it out!
It only works if I run the query in the context of a "CalciteConnection",
and I use "connection.unwrap()" to create the "RelRunner".
WRONG:
==========================
val run = RelRunners.run(bestExp)
val resultSet = run.executeQuery()
RIGHT:
==========================
val runner = connection.unwrap(RelRunner::class.java)
val resultSet = runner.prepareStatement(bestExp).executeQuery()
Why is this out of curiosity?
I only figured this out by mimicking the "FrameworksTest" case below,
so I don't understand the "why".
https://github.com/apache/calcite/blob/de847c38f3544f9c7282984f32dc1093bdb2fb60/core/src/test/java/org/apache/calcite/tools/FrameworksTest.java#L342-L376
On Fri, Dec 31, 2021 at 6:22 PM Gavin Ray <ra...@gmail.com> wrote:
> It appears to be because "getSubSchema" in
> "calcite.jdbc.CalciteSchema.SchemaPlusImpl" has a "subSchemaMap" that loses
> the "hr" schema and only contains "metadata" [0]
>
> But if I call the same function the map shows "hr" is present [1] which
> doesn't make sense to me
>
> [0]: https://imgur.com/djlkBVf
> [1]: https://imgur.com/eEOvjdS
>
>
>
>
>
>
>
> On Fri, Dec 31, 2021 at 5:20 PM Gavin Ray <ra...@gmail.com> wrote:
>
>> Unsure of where this is coming from/how to debug it.
>> (The code I am using is below, taken from Calcite examples)
>>
>> If I print out the sub-schema names of "rootSchema", and their tables, I
>> get:
>>
>> subSchemaNames: [hr, metadata]
>> "hr" subSchema table names: [depts, emps]
>> "metadata" subSchema table names: [COLUMNS, TABLES]
>>
>> So I am not certain what value is being passed that it would fail
>> a "schema.getSubSchema()" call, or where to put a breakpoint to debug
>> that.
>>
>> Does anyone spot something I'm doing that's clearly wrong, or have an idea
>> where I could look to find the code that's causing this?
>>
>> Stacktrace winds up at "Baz", so I think it's generated code
>> Thank you
>> ===========================================================
>>
>> fun main(args: Array<String>) {
>> val hr = Frameworks.createRootSchema(true).add("hr",
>> HrClusteredSchema())
>>
>> val frameworkConfig: FrameworkConfig = Frameworks
>> .newConfigBuilder()
>> .parserConfig(SqlParser.config().withCaseSensitive(false))
>> .defaultSchema(hr)
>> .build()
>>
>> Frameworks.withPlanner(::test, frameworkConfig)
>> }
>>
>> fun test(cluster: RelOptCluster, relOptSchema: RelOptSchema, rootSchema:
>> Schema) {
>> val planner = cluster.planner
>> val relBuilder: RelBuilder =
>> RelFactories.LOGICAL_BUILDER.create(cluster, relOptSchema)
>>
>> val relRoot: RelNode = // ...
>> val desiredTraits =
>> cluster.traitSet().replace(EnumerableConvention.INSTANCE)
>> val newRoot = cluster.planner.changeTraits(relRoot, desiredTraits)
>>
>> cluster.planner.root = newRoot
>> val bestExp = cluster.planner.findBestExp()
>> val run = RelRunners.run(bestExp)
>> val resultSet = run.executeQuery()
>> }
>>
>
Re: Trying to call "PreparedStatement.run()" throws "Cannot invoke "calcite.schema.Schema.getTable(String)" because the return value of "calcite.schema.SchemaPlus.getSubSchema(String)" is null"
Posted by Gavin Ray <ra...@gmail.com>.
It appears to be because "getSubSchema" in
"calcite.jdbc.CalciteSchema.SchemaPlusImpl" has a "subSchemaMap" that loses
the "hr" schema and only contains "metadata" [0]
But if I call the same function the map shows "hr" is present [1] which
doesn't make sense to me
[0]: https://imgur.com/djlkBVf
[1]: https://imgur.com/eEOvjdS
On Fri, Dec 31, 2021 at 5:20 PM Gavin Ray <ra...@gmail.com> wrote:
> Unsure of where this is coming from/how to debug it.
> (The code I am using is below, taken from Calcite examples)
>
> If I print out the sub-schema names of "rootSchema", and their tables, I
> get:
>
> subSchemaNames: [hr, metadata]
> "hr" subSchema table names: [depts, emps]
> "metadata" subSchema table names: [COLUMNS, TABLES]
>
> So I am not certain what value is being passed that it would fail
> a "schema.getSubSchema()" call, or where to put a breakpoint to debug that.
>
> Does anyone spot something I'm doing that's clearly wrong, or have an idea
> where I could look to find the code that's causing this?
>
> Stacktrace winds up at "Baz", so I think it's generated code
> Thank you
> ===========================================================
>
> fun main(args: Array<String>) {
> val hr = Frameworks.createRootSchema(true).add("hr",
> HrClusteredSchema())
>
> val frameworkConfig: FrameworkConfig = Frameworks
> .newConfigBuilder()
> .parserConfig(SqlParser.config().withCaseSensitive(false))
> .defaultSchema(hr)
> .build()
>
> Frameworks.withPlanner(::test, frameworkConfig)
> }
>
> fun test(cluster: RelOptCluster, relOptSchema: RelOptSchema, rootSchema:
> Schema) {
> val planner = cluster.planner
> val relBuilder: RelBuilder =
> RelFactories.LOGICAL_BUILDER.create(cluster, relOptSchema)
>
> val relRoot: RelNode = // ...
> val desiredTraits =
> cluster.traitSet().replace(EnumerableConvention.INSTANCE)
> val newRoot = cluster.planner.changeTraits(relRoot, desiredTraits)
>
> cluster.planner.root = newRoot
> val bestExp = cluster.planner.findBestExp()
> val run = RelRunners.run(bestExp)
> val resultSet = run.executeQuery()
> }
>