You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Johan Oskarsson (JIRA)" <ji...@apache.org> on 2008/12/09 16:06:44 UTC
[jira] Created: (HIVE-144) Ql java source copied to build/gen-java
Ql java source copied to build/gen-java
---------------------------------------
Key: HIVE-144
URL: https://issues.apache.org/jira/browse/HIVE-144
Project: Hadoop Hive
Issue Type: Bug
Components: Build Infrastructure
Affects Versions: 0.19.0
Reporter: Johan Oskarsson
Priority: Trivial
Fix For: 0.19.0
Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-144) Ql java source copied to
build/gen-java
Posted by "Joydeep Sen Sarma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654983#action_12654983 ]
Joydeep Sen Sarma commented on HIVE-144:
----------------------------------------
I would really like to see Hive be available for multiple versions of Hadoop. As a company we are using Hadoop-17 - and i know of many other companies that don't keep up with latest releases of Hadoop (typically - once they have things working well on a particular version of hadoop - they tend to be conservative with hadoop upgrades).
the difference is that Hive is relatively immature and fast moving software right now where people are willing/wanting to try out the latest version (even while they continue using older version of Hadoop). So having latest features from Hive be available on older releases will really help in adoption.
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-144) Ql java source copied to build/gen-java
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zheng Shao updated HIVE-144:
----------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
Status: Resolved (was: Patch Available)
HIVE-144. Hive/ql java source copied to build/ql/java to make it work
with eclipse. (Johan Oskarsson via zshao)
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Assignee: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
> Attachments: HIVE-144.patch, HIVE-144.patch
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-144) Ql java source copied to
build/gen-java
Posted by "Johan Oskarsson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655180#action_12655180 ]
Johan Oskarsson commented on HIVE-144:
--------------------------------------
I wasn't sure if you wanted to avoid including the generated code from Hive.g in the repository. It will of course mean patches that change the query language will have more changes in them but on the other hand it's more consistent with including the generated Thrift code as is already done.
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Assignee: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
> Attachments: HIVE-144.patch
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-144) Ql java source copied to
build/gen-java
Posted by "Johan Oskarsson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654869#action_12654869 ]
Johan Oskarsson commented on HIVE-144:
--------------------------------------
Personally I would avoid preprocessing altogether, it sounds like a messy path to go down. For example unit testing becomes a pain and new code might have to be developed in two versions. And for example the unit testing template code is impossible to edit with an IDE.
It seems the reason behind using the preprocessor is to support multiple Hadoop versions? Would it not be simpler and easier for both developers and the end user if for example Hive 0.19 only worked with Hadoop 0.19? That way there wouldn't be a need for preprocessed code. This is how HBase does it.
Are there any major drawbacks of that solution?
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-144) Ql java source copied to
build/gen-java
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655559#action_12655559 ]
Zheng Shao commented on HIVE-144:
---------------------------------
Johan, just found my idea is just your option 2.
I'd vote for option 2.
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Assignee: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
> Attachments: HIVE-144.patch
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Assigned: (HIVE-144) Ql java source copied to build/gen-java
Posted by "Johan Oskarsson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Johan Oskarsson reassigned HIVE-144:
------------------------------------
Assignee: Johan Oskarsson
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Assignee: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
> Attachments: HIVE-144.patch
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-144) Ql java source copied to
build/gen-java
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655741#action_12655741 ]
Ashish Thusoo commented on HIVE-144:
------------------------------------
+1
looks good to me.
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Assignee: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
> Attachments: HIVE-144.patch, HIVE-144.patch
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-144) Ql java source copied to
build/gen-java
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655534#action_12655534 ]
Zheng Shao commented on HIVE-144:
---------------------------------
Earlier before Hive is open-sourced we actually do check in HiveParse.java and HiveLexer.java, however they change greatly whenever Hive.g changes so the commits become very noisy.
@Johan Can you take a look at the idea on https://issues.apache.org/jira/browse/HIVE-159 ? Let me know if that makes sense to you, and then we can merge the 2 issues.
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Assignee: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
> Attachments: HIVE-144.patch
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-144) Ql java source copied to
build/gen-java
Posted by "Johan Oskarsson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654891#action_12654891 ]
Johan Oskarsson commented on HIVE-144:
--------------------------------------
Perhaps I'm misunderstanding, but what I meant was a release schedule similar to what HBase uses. Meaning that once a Hadoop version is out they try to have a new HBase version that runs on it within a month, they don't make sure it works with any previous version of Hadoop.
Once the branch of a new version has been created no new features goes in there, just serious bugfixes. This is also how Hadoop does it. So there would be very few patches of bugfixes to backport to a previous version of Hive. HBase also only supports the last two versions with new bugfixes, limiting how far back to port fixes.
This naturally forces people to upgrade to a recent version of Hadoop to get the latest Hive version but I don't think that's much of a drawback.
Perhaps doing it differently in the early days of the project is ok, but it also sets a precedent that people are likely to follow further down the line.
>From the HBase wiki on the version numbering: "Going forward, new releases of HBase will have the same version number as the Hadoop release so there will be less confusion over which release of HBase works with which version of Hadoop."
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-144) Ql java source copied to build/gen-java
Posted by "Johan Oskarsson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Johan Oskarsson updated HIVE-144:
---------------------------------
Attachment: HIVE-144.patch
Updated patch for option 2. It just copies the preprocessed data into a folder that is different from the one containing the Hive.g generated code.
With the arguments above taken into consideration I agree that it's the best option.
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Assignee: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
> Attachments: HIVE-144.patch, HIVE-144.patch
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-144) Ql java source copied to build/gen-java
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-144:
--------------------------------
Fix Version/s: 0.3.0
(was: 0.6.0)
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Reporter: Johan Oskarsson
> Assignee: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.3.0
>
> Attachments: HIVE-144.patch, HIVE-144.patch
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-144) Ql java source copied to
build/gen-java
Posted by "Zheng Shao (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656234#action_12656234 ]
Zheng Shao commented on HIVE-144:
---------------------------------
Committed revision 726160.
Thanks Johan!
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Assignee: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
> Attachments: HIVE-144.patch, HIVE-144.patch
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-144) Ql java source copied to build/gen-java
Posted by "Johan Oskarsson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Johan Oskarsson updated HIVE-144:
---------------------------------
Status: Patch Available (was: Open)
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Assignee: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
> Attachments: HIVE-144.patch
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-144) Ql java source copied to
build/gen-java
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654986#action_12654986 ]
Ashish Thusoo commented on HIVE-144:
------------------------------------
As I mentioned, we had followed the same exact approach that Hbase before switching to the current one and considering that there are a lot of fixes that we are making and we want to ensure that those fixes work with all the 3 versions of hadoop, having 3 branches to check those fixes into is very painful. The reality right now is that there are a lot of bug fixes and fixing them in all the 3 versions at the same time is too much of an overhead. Once we start cutting release branches, we would also follow the same methodology (which is quite standard) in that only blocker bugs get checked into the old branches, but right now, we have a number of those bugs which we want to get in before we embark on creating the first release. We do need an exercise to categorize those bugs - which I think we should do this week.
One thing we can do to alleviate some of the problems is to restrict the preprocessing to a specific list of files (right now there is only 1 file that needs this). That would alleviate some of the problems with eclipse till we come up with a better solution for eclipse.
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-144) Ql java source copied to
build/gen-java
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654821#action_12654821 ]
Ashish Thusoo commented on HIVE-144:
------------------------------------
hmm...
the reason that these files are copied there is to support pre processing as that is needed due to differences in the hadoop APIs. Consequently, the files in ql/src/java are preprocessed and then copied to build/gen-java where everything is built.
Agreed though, that if this breaks eclipse then we need to figure out a way to make pre-processing work with eclipse. What are the standard practices to deal with pre processing in eclipse?
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-144) Ql java source copied to
build/gen-java
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655515#action_12655515 ]
Ashish Thusoo commented on HIVE-144:
------------------------------------
Also wanted to add that the reason we could not get the thrift generated files out from svn was that the thrift compiler is C++ code and so we were not sure how portable it would be on different *nix platforms.
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Assignee: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
> Attachments: HIVE-144.patch
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-144) Ql java source copied to
build/gen-java
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12655481#action_12655481 ]
Ashish Thusoo commented on HIVE-144:
------------------------------------
We actually went through a lot of discussion and then decided not to include the generated sources into the svn tree and we actually wanted to do the same for the thrift generated stuff in the long run.
Can the antlr stuff be moved to a different directory within build and we can achieve both the benefits there (no generated code and also easier for eclipse users). Something on the lines as zheng suggested in
https://issues.apache.org/jira/browse/HIVE-159
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Assignee: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
> Attachments: HIVE-144.patch
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HIVE-144) Ql java source copied to build/gen-java
Posted by "Johan Oskarsson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Johan Oskarsson updated HIVE-144:
---------------------------------
Attachment: HIVE-144.patch
I can see you've made your minds up, fair enough. Going back to the original issue, I've got a patch that should help some with the Eclipse issue.
It simply generates the Hive.g Java code into a different directory, ql/src/gen-java so that Eclipse users can include that separately from the preprocessed source code.
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
> Attachments: HIVE-144.patch
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-144) Ql java source copied to
build/gen-java
Posted by "Ashish Thusoo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654878#action_12654878 ]
Ashish Thusoo commented on HIVE-144:
------------------------------------
The overhead for that approach is too high. We had gone that route originally and were maintaining three different code drops for 0.17 (which we use in production at facebook), 0.18 which was a port requested by some external folks and 0.19 which was the hadoop version at the time hive was open sourced, and making these bug fixes and checking it into 3 different places was 3x the amount of work for the developers. Add to that, these branches would invariably diverge in the short term because the sequence of the checkins would not be the same, it became a big pain to port ones changes from one branch to the other because of the numerous merge conflict. So the solution for us, specially during this active phase of development was to have a single branch and compile it and run it with the different versions of hadoop.
For Hbase this probably works because they are perhaps a tad more mature at this point or at least have figured out the dependencies with hadoop to a large extent. It seems that they were also having a separate versioning scheme (different from hadoop) intially - I can hazard a guess that perhaps it was for the same reasons.
Anyway, given the minor tweaks that were needed in the code to port it to 0.18 and 0.19, we saw this as the better of two evils.
Regarding the unit testing template code - we have really taken the pains to ensure that the actual run time code is in QTestUtil.java and the velocity templates are just used to generate the front end junit test cases for different queries.
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HIVE-144) Ql java source copied to
build/gen-java
Posted by "Johan Oskarsson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654818#action_12654818 ]
Johan Oskarsson commented on HIVE-144:
--------------------------------------
A few alternatives:
# If we'd want to make the handling of generated code a bit more consistent we could put the code from Hive.g into svn under ql/src/gen-java much like the thrift generated code in other modules.
# Copy ql/src/java into build/ql/java or a similar directory that is not build/gen-java
Personally I'd vote for the first option.
> Ql java source copied to build/gen-java
> ---------------------------------------
>
> Key: HIVE-144
> URL: https://issues.apache.org/jira/browse/HIVE-144
> Project: Hadoop Hive
> Issue Type: Bug
> Components: Build Infrastructure
> Affects Versions: 0.19.0
> Reporter: Johan Oskarsson
> Priority: Trivial
> Fix For: 0.19.0
>
>
> Since HIVE-98, the build process copies the contents of the ql/src/java directory to build/gen-java where the code generated from Hive.g is located. This makes it a bit more difficult to work on the project in Eclipse, since one has to add the build/gen-java folder to the project and not ql/src/java. This means we'll edit the ql java files in the wrong location. Adding both of these directories will of course result in duplicate entries.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.