You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Chaoran Yu <yu...@gmail.com> on 2018/03/28 05:04:25 UTC
Hive broken master branch?
Hello Hive Devs,
I posted my email below in the users mailing list but realized I should
probably send it here.
I made some changes to a local Hive repo based on the latest master
branch. But somehow I got the following error when running a "CREATE TABLE"
statement in Hive shell:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Insert of object
"org.apache.hadoop.hive.metastore.model.MSerDeInfo@ace2408" using statement
"INSERT INTO SERDES
(SERDE_ID,DESCRIPTION,DESERIALIZER_CLASS,"NAME",SERDE_TYPE,SLIB,SERIALIZER_CLASS)
VALUES (?,?,?,?,?,?,?)" failed : 'DESCRIPTION' is not a column in table or
VTI 'APP.SERDES'.)
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Insert of
object "org.apache.hadoop.hive.metastore.model.MSerDeInfo@ace2408" using
statement "INSERT INTO SERDES
(SERDE_ID,DESCRIPTION,DESERIALIZER_CLASS,"NAME",SERDE_TYPE,SLIB,SERIALIZER_CLASS)
VALUES (?,?,?,?,?,?,?)" failed : 'DESCRIPTION' is not a column in table or
VTI 'APP.SERDES'.)
No matter what table name or schema I create, I got the same error. I
remember testing my changes about three weeks ago using the master branch
back then, everything was working. It only started failing today after I
merged the latest commits in master. Looking at the Travis CI build
history, the build for master started failing since about 21 days ago.
Is anyone aware of recent commits that might break this write code path?
And do we know when the next release of Hive is coming out?
Thank you,
Chaoran Yu
Re: Hive broken master branch?
Posted by Chaoran Yu <yu...@gmail.com>.
Ok I found out why. My branch doesn’t really have the latest updates from the master. Mine was only up to date until two weeks ago.
Looks like this is the commit that fixed the bug: https://github.com/apache/hive/commit/e6daf43f0c41ff91014ae9769ba8ad8cab4e5920 <https://github.com/apache/hive/commit/e6daf43f0c41ff91014ae9769ba8ad8cab4e5920>
Merging the latest commits and rebuilding Hive fixed the problem for me.
> On Mar 28, 2018, at 12:39, Chaoran Yu <yu...@gmail.com> wrote:
>
> Yes I’m using Derby as the metastore.
>
> I didn’t have the settings for the two parameters you mentioned. I just tried setting them in hive-site.xml:
>
> <property>
> <name>datanucleus.autoCreateTables</name>
> <value>True</value>
> </property>
> <property>
> <name>datanucleus.autoCreateSchema</name>
> <value>True</value>
> </property>
>
> Then I re-initialized derby by running: "./bin/schematool -dbType derby -initSchema”, but still got the same error when I run a “CREATE TABLE” statement.
>
>
>> On Mar 28, 2018, at 12:27, Vihang Karajgaonkar <vihang@cloudera.com <ma...@cloudera.com>> wrote:
>>
>> Are you using derby for the backing DB for HMS? If yes, what is the value
>> of datanucleus.autoCreateTables and datanucleus.autoCreateSchema? This
>> looks like the schema is not updated for some reason. Can you confirm if
>> you have the latest schema (or try re-initializing the schema)?
>>
>> On Tue, Mar 27, 2018 at 10:04 PM, Chaoran Yu <yuchaoran2011@gmail.com <ma...@gmail.com>>
>> wrote:
>>
>>> Hello Hive Devs,
>>>
>>> I posted my email below in the users mailing list but realized I should
>>> probably send it here.
>>>
>>> I made some changes to a local Hive repo based on the latest master
>>> branch. But somehow I got the following error when running a "CREATE TABLE"
>>> statement in Hive shell:
>>>
>>> org.apache.hadoop.hive.ql.metadata.HiveException:
>>> MetaException(message:Insert of object
>>> "org.apache.hadoop.hive.metastore.model.MSerDeInfo@ace2408" using
>>> statement
>>> "INSERT INTO SERDES
>>> (SERDE_ID,DESCRIPTION,DESERIALIZER_CLASS,"NAME",
>>> SERDE_TYPE,SLIB,SERIALIZER_CLASS)
>>> VALUES (?,?,?,?,?,?,?)" failed : 'DESCRIPTION' is not a column in table or
>>> VTI 'APP.SERDES'.)
>>> FAILED: Execution Error, return code 1 from
>>> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Insert of
>>> object "org.apache.hadoop.hive.metastore.model.MSerDeInfo@ace2408" using
>>> statement "INSERT INTO SERDES
>>> (SERDE_ID,DESCRIPTION,DESERIALIZER_CLASS,"NAME",
>>> SERDE_TYPE,SLIB,SERIALIZER_CLASS)
>>> VALUES (?,?,?,?,?,?,?)" failed : 'DESCRIPTION' is not a column in table or
>>> VTI 'APP.SERDES'.)
>>>
>>> No matter what table name or schema I create, I got the same error. I
>>> remember testing my changes about three weeks ago using the master branch
>>> back then, everything was working. It only started failing today after I
>>> merged the latest commits in master. Looking at the Travis CI build
>>> history, the build for master started failing since about 21 days ago.
>>>
>>> Is anyone aware of recent commits that might break this write code path?
>>> And do we know when the next release of Hive is coming out?
>>>
>>> Thank you,
>>> Chaoran Yu
>>>
>
Re: Hive broken master branch?
Posted by Chaoran Yu <yu...@gmail.com>.
Yes I’m using Derby as the metastore.
I didn’t have the settings for the two parameters you mentioned. I just tried setting them in hive-site.xml:
<property>
<name>datanucleus.autoCreateTables</name>
<value>True</value>
</property>
<property>
<name>datanucleus.autoCreateSchema</name>
<value>True</value>
</property>
Then I re-initialized derby by running: "./bin/schematool -dbType derby -initSchema”, but still got the same error when I run a “CREATE TABLE” statement.
> On Mar 28, 2018, at 12:27, Vihang Karajgaonkar <vi...@cloudera.com> wrote:
>
> Are you using derby for the backing DB for HMS? If yes, what is the value
> of datanucleus.autoCreateTables and datanucleus.autoCreateSchema? This
> looks like the schema is not updated for some reason. Can you confirm if
> you have the latest schema (or try re-initializing the schema)?
>
> On Tue, Mar 27, 2018 at 10:04 PM, Chaoran Yu <yu...@gmail.com>
> wrote:
>
>> Hello Hive Devs,
>>
>> I posted my email below in the users mailing list but realized I should
>> probably send it here.
>>
>> I made some changes to a local Hive repo based on the latest master
>> branch. But somehow I got the following error when running a "CREATE TABLE"
>> statement in Hive shell:
>>
>> org.apache.hadoop.hive.ql.metadata.HiveException:
>> MetaException(message:Insert of object
>> "org.apache.hadoop.hive.metastore.model.MSerDeInfo@ace2408" using
>> statement
>> "INSERT INTO SERDES
>> (SERDE_ID,DESCRIPTION,DESERIALIZER_CLASS,"NAME",
>> SERDE_TYPE,SLIB,SERIALIZER_CLASS)
>> VALUES (?,?,?,?,?,?,?)" failed : 'DESCRIPTION' is not a column in table or
>> VTI 'APP.SERDES'.)
>> FAILED: Execution Error, return code 1 from
>> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Insert of
>> object "org.apache.hadoop.hive.metastore.model.MSerDeInfo@ace2408" using
>> statement "INSERT INTO SERDES
>> (SERDE_ID,DESCRIPTION,DESERIALIZER_CLASS,"NAME",
>> SERDE_TYPE,SLIB,SERIALIZER_CLASS)
>> VALUES (?,?,?,?,?,?,?)" failed : 'DESCRIPTION' is not a column in table or
>> VTI 'APP.SERDES'.)
>>
>> No matter what table name or schema I create, I got the same error. I
>> remember testing my changes about three weeks ago using the master branch
>> back then, everything was working. It only started failing today after I
>> merged the latest commits in master. Looking at the Travis CI build
>> history, the build for master started failing since about 21 days ago.
>>
>> Is anyone aware of recent commits that might break this write code path?
>> And do we know when the next release of Hive is coming out?
>>
>> Thank you,
>> Chaoran Yu
>>
Re: Hive broken master branch?
Posted by Vihang Karajgaonkar <vi...@cloudera.com>.
Are you using derby for the backing DB for HMS? If yes, what is the value
of datanucleus.autoCreateTables and datanucleus.autoCreateSchema? This
looks like the schema is not updated for some reason. Can you confirm if
you have the latest schema (or try re-initializing the schema)?
On Tue, Mar 27, 2018 at 10:04 PM, Chaoran Yu <yu...@gmail.com>
wrote:
> Hello Hive Devs,
>
> I posted my email below in the users mailing list but realized I should
> probably send it here.
>
> I made some changes to a local Hive repo based on the latest master
> branch. But somehow I got the following error when running a "CREATE TABLE"
> statement in Hive shell:
>
> org.apache.hadoop.hive.ql.metadata.HiveException:
> MetaException(message:Insert of object
> "org.apache.hadoop.hive.metastore.model.MSerDeInfo@ace2408" using
> statement
> "INSERT INTO SERDES
> (SERDE_ID,DESCRIPTION,DESERIALIZER_CLASS,"NAME",
> SERDE_TYPE,SLIB,SERIALIZER_CLASS)
> VALUES (?,?,?,?,?,?,?)" failed : 'DESCRIPTION' is not a column in table or
> VTI 'APP.SERDES'.)
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Insert of
> object "org.apache.hadoop.hive.metastore.model.MSerDeInfo@ace2408" using
> statement "INSERT INTO SERDES
> (SERDE_ID,DESCRIPTION,DESERIALIZER_CLASS,"NAME",
> SERDE_TYPE,SLIB,SERIALIZER_CLASS)
> VALUES (?,?,?,?,?,?,?)" failed : 'DESCRIPTION' is not a column in table or
> VTI 'APP.SERDES'.)
>
> No matter what table name or schema I create, I got the same error. I
> remember testing my changes about three weeks ago using the master branch
> back then, everything was working. It only started failing today after I
> merged the latest commits in master. Looking at the Travis CI build
> history, the build for master started failing since about 21 days ago.
>
> Is anyone aware of recent commits that might break this write code path?
> And do we know when the next release of Hive is coming out?
>
> Thank you,
> Chaoran Yu
>