You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hudi.apache.org by Ambarish Giri <am...@gmail.com> on 2021/08/03 03:58:06 UTC
Reg. org.apache.hudi.exception.HoodieException: Invalid query type :read_optimized
Hi Team,
I am an Architect in a reputable Product based IT firm.
I am in the evaluation process to use Hudi to incorporate a refreshable
data lake.
I am currently running the setup in my local machine and using a spark
datasource to write and read from the Hudi temp table.
I have evaluated the Cow and MoR write mechanisms but while trying to read
the Hudi table using Read_Optimized type I am getting the below exception:
Exception in thread "main" org.apache.hudi.exception.HoodieException:
Invalid query type :read_optimized
at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:81)
at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:46)
at
org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:332)
at
org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:242)
Below is the how I am trying to read from the Hudi location:
spark.read
.format("hudi")
.option(DataSourceReadOptions.QUERY_TYPE_OPT_KEY,
DataSourceReadOptions.QUERY_TYPE_READ_OPTIMIZED_OPT_VAL)
.load(s"$basePath/$tableName")
.show(50,false)
Kindly suggest if I am doing anything wrong?
Below are the versions I am currently using :
apache hudi 0.7.0
spark 2.4.7
scala 2.12
Kindly let me know in case anymore details are required.
Regards,
Ambarish Giri
9951742695
Re: Reg. org.apache.hudi.exception.HoodieException: Invalid query
type :read_optimized
Posted by Ambarish Giri <am...@gmail.com>.
Hi Siva,
Sure will update the issue and follow there.
Just for FYI I have gone through the FAQs
<https://cwiki.apache.org/confluence/display/HUDI/FAQ> as well.
In addition to the above issue, I was also trying to run async compaction
on the MOR Hudi table but was not able to do that in the current setup.
Can you please guide me through that as well as how to
achieve scheduled compaction or explicit compaction on the Hudi table.
Regards,
Ambarish Giri
On Tue, Aug 3, 2021 at 8:40 PM Sivabalan <n....@gmail.com> wrote:
> Hey Ambarish,
> I have created a github issue.
> <https://github.com/apache/hudi/issues/3395> Let's follow up there.
>
> On Mon, Aug 2, 2021 at 11:58 PM Ambarish Giri <am...@gmail.com>
> wrote:
>
> > Hi Team,
> >
> > I am an Architect in a reputable Product based IT firm.
> > I am in the evaluation process to use Hudi to incorporate a refreshable
> > data lake.
> > I am currently running the setup in my local machine and using a spark
> > datasource to write and read from the Hudi temp table.
> > I have evaluated the Cow and MoR write mechanisms but while trying to
> read
> > the Hudi table using Read_Optimized type I am getting the below
> exception:
> >
> > Exception in thread "main" org.apache.hudi.exception.HoodieException:
> > Invalid query type :read_optimized
> > at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:81)
> > at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:46)
> > at
> >
> >
> org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:332)
> > at
> >
> >
> org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:242)
> >
> > Below is the how I am trying to read from the Hudi location:
> >
> > spark.read
> > .format("hudi")
> > .option(DataSourceReadOptions.QUERY_TYPE_OPT_KEY,
> > DataSourceReadOptions.QUERY_TYPE_READ_OPTIMIZED_OPT_VAL)
> > .load(s"$basePath/$tableName")
> > .show(50,false)
> >
> > Kindly suggest if I am doing anything wrong?
> >
> > Below are the versions I am currently using :
> > apache hudi 0.7.0
> > spark 2.4.7
> > scala 2.12
> >
> > Kindly let me know in case anymore details are required.
> >
> > Regards,
> > Ambarish Giri
> > 9951742695
> >
>
>
> --
> Regards,
> -Sivabalan
>
Re: Reg. org.apache.hudi.exception.HoodieException: Invalid query
type :read_optimized
Posted by Sivabalan <n....@gmail.com>.
Hey Ambarish,
I have created a github issue.
<https://github.com/apache/hudi/issues/3395> Let's follow up there.
On Mon, Aug 2, 2021 at 11:58 PM Ambarish Giri <am...@gmail.com>
wrote:
> Hi Team,
>
> I am an Architect in a reputable Product based IT firm.
> I am in the evaluation process to use Hudi to incorporate a refreshable
> data lake.
> I am currently running the setup in my local machine and using a spark
> datasource to write and read from the Hudi temp table.
> I have evaluated the Cow and MoR write mechanisms but while trying to read
> the Hudi table using Read_Optimized type I am getting the below exception:
>
> Exception in thread "main" org.apache.hudi.exception.HoodieException:
> Invalid query type :read_optimized
> at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:81)
> at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:46)
> at
>
> org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:332)
> at
>
> org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:242)
>
> Below is the how I am trying to read from the Hudi location:
>
> spark.read
> .format("hudi")
> .option(DataSourceReadOptions.QUERY_TYPE_OPT_KEY,
> DataSourceReadOptions.QUERY_TYPE_READ_OPTIMIZED_OPT_VAL)
> .load(s"$basePath/$tableName")
> .show(50,false)
>
> Kindly suggest if I am doing anything wrong?
>
> Below are the versions I am currently using :
> apache hudi 0.7.0
> spark 2.4.7
> scala 2.12
>
> Kindly let me know in case anymore details are required.
>
> Regards,
> Ambarish Giri
> 9951742695
>
--
Regards,
-Sivabalan