You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Q Long (JIRA)" <ji...@apache.org> on 2011/03/31 20:06:05 UTC
[jira] [Created] (HIVE-2086) Data loss with external table
Data loss with external table
-----------------------------
Key: HIVE-2086
URL: https://issues.apache.org/jira/browse/HIVE-2086
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.7.0
Environment: Amazon elastics mapreduce cluster
Reporter: Q Long
Data loss when using "create external table like" statement.
1) Set up an external table S, point to location L. Populate data in S.
2) Create another external table T, using statement like this:
create external table T like S location L
Make sure table T point to the same location as the original table S.
3) Query table T, see the same set of data in S.
4) drop table T.
5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Data loss with external table
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13020158#comment-13020158 ]
jiraposter@reviews.apache.org commented on HIVE-2086:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/604/
-----------------------------------------------------------
Review request for hive.
Summary
-------
Review request for HIVE-2086. The external parameters were being set, but then being zeroed out by a call to params.clear(), resulting in an external table that wasn't marked as external.
This addresses bug HIVE-2086.
https://issues.apache.org/jira/browse/HIVE-2086
Diffs
-----
build-common.xml 9f21a69
data/files/ext_test/test.dat PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 9d8919c
ql/src/test/queries/clientpositive/create_like.q 2edde83
ql/src/test/results/clientpositive/create_like.q.out 63a8939
Diff: https://reviews.apache.org/r/604/diff
Testing
-------
Automated tests added
Thanks,
Jonathan
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Data loss with external table
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13022615#comment-13022615 ]
Ning Zhang commented on HIVE-2086:
----------------------------------
Jonathan, there are a few diffs in the unit tests (1 in TestHBaseCliDriver and 6 in TestCliDriver). Can you take a look?
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2086) Data loss with external table
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang updated HIVE-2086:
-----------------------------
Status: Open (was: Patch Available)
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Data loss with external table
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068007#comment-13068007 ]
Carl Steinbach commented on HIVE-2086:
--------------------------------------
+1. Will commit if tests pass.
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch, HIVE-2086.3.patch, create_like.q.out
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2086) Data loss with external table
Posted by "Jonathan Natkins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Natkins updated HIVE-2086:
-----------------------------------
Attachment: HIVE-2086.2.patch
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Data loss with external table
Posted by "Jonathan Natkins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025862#comment-13025862 ]
Jonathan Natkins commented on HIVE-2086:
----------------------------------------
Sorry about that, I've updated the diffs. This time, there shouldn't be any failures
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Data loss with external table
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025863#comment-13025863 ]
jiraposter@reviews.apache.org commented on HIVE-2086:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/604/
-----------------------------------------------------------
(Updated 2011-04-27 16:00:50.150750)
Review request for hive.
Changes
-------
Fixing test diffs
Summary
-------
Review request for HIVE-2086. The external parameters were being set, but then being zeroed out by a call to params.clear(), resulting in an external table that wasn't marked as external.
This addresses bug HIVE-2086.
https://issues.apache.org/jira/browse/HIVE-2086
Diffs (updated)
-----
build-common.xml 00c3680
ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 22fde53
ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java 06a0447
ql/src/test/queries/clientpositive/create_like.q 2edde83
ql/src/test/results/clientpositive/create_like.q.out 63a8939
Diff: https://reviews.apache.org/r/604/diff
Testing
-------
Automated tests added
Thanks,
Jonathan
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2086) Add test coverage for external table
data loss issue
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2086:
---------------------------------
Resolution: Fixed
Fix Version/s: 0.8.0
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks Natty!
> Add test coverage for external table data loss issue
> ----------------------------------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Fix For: 0.8.0
>
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch, HIVE-2086.3.patch, create_like.q.out
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Data loss with external table
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13022269#comment-13022269 ]
jiraposter@reviews.apache.org commented on HIVE-2086:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/604/#review508
-----------------------------------------------------------
Ship it!
- Ning
On 2011-04-15 01:34:56, Jonathan Natkins wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/604/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-04-15 01:34:56)
bq.
bq.
bq. Review request for hive.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Review request for HIVE-2086. The external parameters were being set, but then being zeroed out by a call to params.clear(), resulting in an external table that wasn't marked as external.
bq.
bq.
bq. This addresses bug HIVE-2086.
bq. https://issues.apache.org/jira/browse/HIVE-2086
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. build-common.xml 9f21a69
bq. data/files/ext_test/test.dat PRE-CREATION
bq. ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 9d8919c
bq. ql/src/test/queries/clientpositive/create_like.q 2edde83
bq. ql/src/test/results/clientpositive/create_like.q.out 63a8939
bq.
bq. Diff: https://reviews.apache.org/r/604/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Automated tests added
bq.
bq.
bq. Thanks,
bq.
bq. Jonathan
bq.
bq.
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2086) Data loss with external table
Posted by "Jonathan Natkins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Natkins updated HIVE-2086:
-----------------------------------
Status: Patch Available (was: Open)
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Data loss with external table
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13022270#comment-13022270 ]
Ning Zhang commented on HIVE-2086:
----------------------------------
+1. Will commit if tests pass.
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2086) Data loss with external table
Posted by "Jonathan Natkins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Natkins updated HIVE-2086:
-----------------------------------
Status: Patch Available (was: Open)
Just regression tests
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch, HIVE-2086.3.patch, create_like.q.out
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2086) Data loss with external table
Posted by "Jonathan Natkins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Natkins updated HIVE-2086:
-----------------------------------
Attachment: HIVE-2086.1.patch
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2086) Data loss with external table
Posted by "Jonathan Natkins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Natkins updated HIVE-2086:
-----------------------------------
Attachment: HIVE-2086.3.patch
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch, HIVE-2086.3.patch, create_like.q.out
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Data loss with external table
Posted by "Jonathan Natkins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067522#comment-13067522 ]
Jonathan Natkins commented on HIVE-2086:
----------------------------------------
It looks like this was actually just fixed today, as a part of HIVE-1218. However, that fix didn't add any regression tests for this bug, so I'm going to prepare an updated patch once I finish running the test suite.
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch, create_like.q.out
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Data loss with external table
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13067833#comment-13067833 ]
jiraposter@reviews.apache.org commented on HIVE-2086:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/604/
-----------------------------------------------------------
(Updated 2011-07-19 16:50:07.742470)
Review request for hive.
Changes
-------
The bug was fixed as a part of HIVE-1218, and this diff adds some regression tests for this particular issue.
Summary
-------
Review request for HIVE-2086. The external parameters were being set, but then being zeroed out by a call to params.clear(), resulting in an external table that wasn't marked as external.
This addresses bug HIVE-2086.
https://issues.apache.org/jira/browse/HIVE-2086
Diffs (updated)
-----
build-common.xml b6807fa
data/files/ext_test/test.dat PRE-CREATION
ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java 6d742e5
ql/src/test/queries/clientpositive/create_like.q 3f8e58d
ql/src/test/results/clientpositive/create_like.q.out 118235e
Diff: https://reviews.apache.org/r/604/diff
Testing
-------
Automated tests added
Thanks,
Jonathan
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch, create_like.q.out
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2086) Data loss with external table
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach reassigned HIVE-2086:
------------------------------------
Assignee: Jonathan Natkins
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2086) Add test coverage for external table
data loss issue
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-2086:
---------------------------------
Summary: Add test coverage for external table data loss issue (was: Data loss with external table)
> Add test coverage for external table data loss issue
> ----------------------------------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch, HIVE-2086.3.patch, create_like.q.out
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Data loss with external table
Posted by "Edward Capriolo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13014109#comment-13014109 ]
Edward Capriolo commented on HIVE-2086:
---------------------------------------
Dropping an external table should not delete data. Are you saying that 'create table like' does not preserver the external property?
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2086) Data loss with external table
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang updated HIVE-2086:
-----------------------------
Status: Open (was: Patch Available)
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch, create_like.q.out
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2086) Data loss with external table
Posted by "Jonathan Natkins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Natkins updated HIVE-2086:
-----------------------------------
Attachment: HIVE-2086.3.patch
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch, HIVE-2086.3.patch, create_like.q.out
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2086) Data loss with external table
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ning Zhang updated HIVE-2086:
-----------------------------
Attachment: create_like.q.out
@Johnathan, there are some tests failing. In particular the newly added test also failed (the output attached). Can you take a look and see if it is caused by newly committed patches (I've tested on the lasted trunk).
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch, create_like.q.out
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Add test coverage for external table
data loss issue
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13069164#comment-13069164 ]
Hudson commented on HIVE-2086:
------------------------------
Integrated in Hive-trunk-h0.21 #841 (See [https://builds.apache.org/job/Hive-trunk-h0.21/841/])
HIVE-2086. Add test coverage for external table data loss issue (Jonathan Natkins via cws)
cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1149331
Files :
* /hive/trunk/data/files/ext_test
* /hive/trunk/ql/src/test/queries/clientpositive/create_like.q
* /hive/trunk/ql/src/test/results/clientpositive/create_like.q.out
* /hive/trunk/data/files/ext_test/test.dat
* /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java
* /hive/trunk/build-common.xml
> Add test coverage for external table data loss issue
> ----------------------------------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Fix For: 0.8.0
>
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch, HIVE-2086.3.patch, create_like.q.out
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Data loss with external table
Posted by "Vaibhav Aggarwal (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054550#comment-13054550 ]
Vaibhav Aggarwal commented on HIVE-2086:
----------------------------------------
Has this patch been committed or is anyone still working on this particular patch?
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch, create_like.q.out
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2086) Data loss with external table
Posted by "Jonathan Natkins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Natkins updated HIVE-2086:
-----------------------------------
Attachment: (was: HIVE-2086.3.patch)
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch, HIVE-2086.3.patch, create_like.q.out
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Data loss with external table
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13022268#comment-13022268 ]
jiraposter@reviews.apache.org commented on HIVE-2086:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/604/#review507
-----------------------------------------------------------
ql/src/test/queries/clientpositive/create_like.q
<https://reviews.apache.org/r/604/#comment1031>
the $var is cool!
- Ning
On 2011-04-15 01:34:56, Jonathan Natkins wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/604/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-04-15 01:34:56)
bq.
bq.
bq. Review request for hive.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Review request for HIVE-2086. The external parameters were being set, but then being zeroed out by a call to params.clear(), resulting in an external table that wasn't marked as external.
bq.
bq.
bq. This addresses bug HIVE-2086.
bq. https://issues.apache.org/jira/browse/HIVE-2086
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. build-common.xml 9f21a69
bq. data/files/ext_test/test.dat PRE-CREATION
bq. ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 9d8919c
bq. ql/src/test/queries/clientpositive/create_like.q 2edde83
bq. ql/src/test/results/clientpositive/create_like.q.out 63a8939
bq.
bq. Diff: https://reviews.apache.org/r/604/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Automated tests added
bq.
bq.
bq. Thanks,
bq.
bq. Jonathan
bq.
bq.
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Data loss with external table
Posted by "Ning Zhang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027424#comment-13027424 ]
Ning Zhang commented on HIVE-2086:
----------------------------------
+1
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
> Attachments: HIVE-2086.1.patch, HIVE-2086.2.patch
>
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2086) Data loss with external table
Posted by "Jonathan Natkins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Natkins updated HIVE-2086:
-----------------------------------
Status: Patch Available (was: Open)
https://reviews.apache.org/r/604/
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
> Assignee: Jonathan Natkins
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2086) Data loss with external table
Posted by "Q Long (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-2086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13014126#comment-13014126 ]
Q Long commented on HIVE-2086:
------------------------------
It seems that "create external table like" does not preserve the external property.
Note that both the original table S and the new table T are external, and data loss will only occur when creating T using statement
"create external table T like S location L". No data loss if T with full table definitions (i.e, does not use like statement)
> Data loss with external table
> -----------------------------
>
> Key: HIVE-2086
> URL: https://issues.apache.org/jira/browse/HIVE-2086
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 0.7.0
> Environment: Amazon elastics mapreduce cluster
> Reporter: Q Long
>
> Data loss when using "create external table like" statement.
> 1) Set up an external table S, point to location L. Populate data in S.
> 2) Create another external table T, using statement like this:
> create external table T like S location L
> Make sure table T point to the same location as the original table S.
> 3) Query table T, see the same set of data in S.
> 4) drop table T.
> 5) Query table S will return nothing, and location L is deleted.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira