You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@phoenix.apache.org by Istvan Toth <st...@cloudera.com> on 2021/10/06 08:07:10 UTC
Re: Experience building Hbase and Phoenix for the first time
This topic is fine for the user list, others may have similar problems,
hence copying there.
I don't think we should link to the archives.
If you do a fresh install, you should use the latest HBase patch releases.
If you have an existing cluster, you should generally still build with that
latest patch
release for the corresponding minor release, unless there is as
incompatibility (like with Hbase 2.4.0)
If you have an existing older cluster (Hbase 2.1, or 2.2) then we kind of
expect that you are the cluster administrator, and know
where the sources are.
The test failures are because you need to rebuild and reinstall HBase
locally with Hadoop 3 before building Phoenix.
This is detailed in BUILDING.md under "HBase 2 and Hadoop 3"
phoenix_sandbox should work with Python 3.
We've fixed the startup scripts, but this seems to have been missed.
Could you open a JIRA for that please ?
The same goes for the sqlline issue, sqlline should work in the build
directory after "mvn package"
Could you open a JIRA for that as well please ?
Istvan
On Wed, Oct 6, 2021 at 5:51 AM Simon Mottram <Si...@cucumber.co.nz>
wrote:
> Hi
>
> I'm sending this direct as I'm not sure it's relevant to the user
> channel. I did get it all running, but it wasn't plain sailing. Not
> surprising given the size of the project.
>
> It can be really useful to get the journey of someone starting from
> scratch so I thought you might find this interesting. I have put a few
> 'Suggestion' headings in there, things I would like to have known in advance
>
> Hi
>
> Took a long time to get to a working SqlLine prompt, here's my journey! I
> think it might be useful to update the BUILDING.md description for people
> like me who are new to the tech and such large complex builds
>
> OS Version
> =========
> Distributor ID: Ubuntu
> Description: Ubuntu 20.04.3 LTS
> Release: 20.04
> Codename: focal
>
> Java version
> ===============
> openjdk 11.0.11 2021-04-20
> OpenJDK Runtime Environment (build 11.0.11+9-Ubuntu-0ubuntu2.20.04)
> OpenJDK 64-Bit Server VM (build 11.0.11+9-Ubuntu-0ubuntu2.20.04, mixed
> mode, sharing)
>
> Compiling and installing Hbase
> ==============================
> The only versions of hbase on the url in the BUILDING.md are 2.3.6 and
> 2.4.6 so I went with those.
>
> None of the 2.4 profile hbase versions that get built are available at the
> url in the BUILDING.md file. I assume this doesn't really matter?
> [INFO] Phoenix Hbase 2.4.1 compatibility .................. SUCCESS [
> 9.229 s]
> [INFO] Phoenix Hbase 2.4.0 compatibility .................. SUCCESS [
> 7.604 s]
> [INFO] Phoenix Hbase 2.3.0 compatibility .................. SUCCESS [
> 8.074 s]
> [INFO] Phoenix Hbase 2.2.5 compatibility .................. SUCCESS [
> 7.233 s]
> [INFO] Phoenix Hbase 2.1.6 compatibility .................. SUCCESS [
> 7.002 s]
>
> I compiled and installed both versions without problem
> wget https://downloads.apache.org/hbase/2.4.6/hbase-2.4.6-src.tar.gz
> mvn install -Dhadoop.profile=3.0 -DskipTests
> mvn clean install -Dhbase.profile=2.4
>
> Repeating the above with 2.3.6. All good!
>
> Suggestion: A more useful URL for wget would be the archive URL:
> https://archive.apache.org/dist/hbase/
>
> Phoenix
> =======
>
> Summary: can build, but can't get tests to work. JDK?
>
> With both versions i tried just specifying profiles 2.3 and 2.4
> I then tried specifying the exact versions
>
> mvn clean install -Dhbase.profile=2.4 -Dhbase.version=2.4.6
> mvn clean install -Dhbase.profile=2.3 -Dhbase.version=2.3.6
>
> I get the same error regardless during tests
>
> [INFO] Running org.apache.phoenix.end2end.ExtendedQueryExecIT
> [ERROR] Tests run: 61, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 300.264 s <<< FAILURE! - in org.apache.phoenix.end2end.DateTimeIT
> [ERROR] org.apache.phoenix.end2end.DateTimeIT.testAncientDatesWithJavaTime
> Time elapsed: 4.537 s <<< FAILURE!
> java.lang.AssertionError: expected:<10> but was:<9>
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.failNotEquals(Assert.java:835)
> at org.junit.Assert.assertEquals(Assert.java:647)
> at org.junit.Assert.assertEquals(Assert.java:633)
> at
> org.apache.phoenix.end2end.DateTimeIT.testAncientDatesWithJavaTime(DateTimeIT.java:2147)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at
> org.apache.phoenix.SystemExitRule$1.evaluate(SystemExitRule.java:40)
> at
> org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at org.junit.runners.Suite.runChild(Suite.java:128)
> at org.junit.runners.Suite.runChild(Suite.java:27)
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> at
> org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
> at
> org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeLazy(JUnitCoreWrapper.java:119)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:87)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:157)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
> at
> org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548)
>
>
> Phoenix with no Tests
> =====================
>
> If I skipTests it does build
>
> mvn clean install -Dhbase.profile=2.4 -Dhbase.version=2.4.6 -DskipTests
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Detecting the operating system and CPU architecture
> [INFO]
> ------------------------------------------------------------------------
> [INFO] os.detected.name: linux
> [INFO] os.detected.arch: x86_64
> [INFO] os.detected.version: 4.19
> [INFO] os.detected.version.major: 4
> [INFO] os.detected.version.minor: 19
> [INFO] os.detected.release: ubuntu
> [INFO] os.detected.release.version: 20.04
> [INFO] os.detected.release.like.ubuntu: true
> [INFO] os.detected.release.like.debian: true
> [INFO] os.detected.classifier: linux-x86_64
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Build Order:
> [INFO]
> [INFO] Apache Phoenix
> [pom]
> [INFO] Phoenix Hbase 2.4.1 compatibility
> [jar]
> [INFO] Phoenix Hbase 2.4.0 compatibility
> [jar]
> [INFO] Phoenix Hbase 2.3.0 compatibility
> [jar]
> [INFO] Phoenix Hbase 2.2.5 compatibility
> [jar]
> [INFO] Phoenix Hbase 2.1.6 compatibility
> [jar]
> [INFO] Phoenix Core
> [jar]
> [INFO] Phoenix - Pherf
> [jar]
> [INFO] Phoenix - Tracing Web Application
> [jar]
> [INFO] Phoenix Client Parent
> [pom]
> [INFO] Phoenix Client
> [jar]
> [INFO] Phoenix Client Embedded
> [jar]
> [INFO] Phoenix Server JAR
> [jar]
> [INFO] Phoenix Assembly
> [pom]
> [INFO]
> [INFO] ---------------------< org.apache.phoenix:phoenix
> >---------------------
> [INFO] Building Apache Phoenix 5.2.0-SNAPSHOT
> [1/14]
> [INFO] --------------------------------[ pom
> ]---------------------------------
>
>
> Running Sandbox
> =================
>
> Python
> -----------
> That just gets me to the python script which gave the following error:
>
> Traceback (most recent call last):
> File "bin/phoenix_sandbox.py", line 49, in <module>
> logging_config, ":".join(cp_components))
> TypeError: sequence item 1: expected str instance, bytes found
>
> I found that I needed to switch to python2 which I did
> ----------------------------------------------------
>
> sudo update-alternatives --config python
> There are 2 choices for the alternative python (providing /usr/bin/python).
>
> Selection Path Priority Status
> ------------------------------------------------------------
> 0 /usr/bin/python3 2 auto mode
> * 1 /usr/bin/python2 1 manual mode
> 2 /usr/bin/python3 2 manual mode
>
> /work/phoenix/phoenix$ python --version
> Python 2.7.18
>
> Still didn't work, but I tried running python directly, so 'python
> bin/phoenix_sandbox.py and it starts.
>
> I presume this is down to specifying which python handles python bang? I
> think people using python3 wouldn't be happy changing a system level
> setting,
>
> Suggestion: for people who have only python3 or no python, a link and some
> instructions, also recommend using python command instead of the python
> bang.
>
> SqlLine
> --------
> I tried sqlline but get the error:
> Error: Could not find or load main class sqlline.SqlLine
> Caused by: java.lang.ClassNotFoundException: sqlline.SqlLine
>
> After much to-ing and fro-ing I discovered
> https://search.maven.org/search?q=g:sqlline and you have to download the
> sqlline*jar-with-dependences.jar for phoenix_utils.py to find it.
>
> Then I found you need to create a lib folder at the same level as bin for
> it to be found by
>
> Suggestion: Add URL to download the 'with-dependencies' jar and
> instruction to add to lib folder for phoenix (or on path)
>
--
*István Tóth* | Staff Software Engineer
stoty@cloudera.com <https://www.cloudera.com>
[image: Cloudera] <https://www.cloudera.com/>
[image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
on LinkedIn] <https://www.linkedin.com/company/cloudera>
<https://www.cloudera.com/>
------------------------------