You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pegasus.apache.org by Lan Liang <li...@163.com> on 2022/08/24 02:14:52 UTC
Re: How can i run pegasus from container image
Just fill the link for [0].
[0] https://pegasus.apache.org/docs/build/compile-by-docker/
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<li...@163.com> |
| Date | 8/23/2022 23:43 |
| To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
| Subject | How can i run pegasus from container image |
Hi pegasus community:
I want to run pegasus from docker and i follow the doc of compile-by-docker [0] to try it. And the container image apachepegasus/build-env:2.1.0-centos7 is not exist.
So i found the image of apache/pegasus:build-env-centos7 and run command of *** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build -c" *** to compile on the pegasus source, and just got the error of * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or directory *.
And it would be great if i can run the pegasus from some docker image.
Remind me if i missed something,Thanks.
Best Regards,
Lan Liang
Re: How can i run pegasus from container image
Posted by Lan Liang <li...@163.com>.
Hi Yingchun,
In the 2nd step, i'm really use command of *** docker run -v
$PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh start_onebox" *** to start onebox, and just ZK is working.
I say ***docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "DSN_ROOT/bin/pegasus_server/pegasus_server "*** and this is a misstake.
Thanks for your help, use command of ***docker/pegasus-docker-compose/start_onebox.sh*** run success pegasus from myself pegasus docker image.
I have request meta server and got the response :
***
[root@chaos incubator-pegasus]# curl 192.168.56.11:34601
{
/: "Lists all supported calls",
/config: "get the details of a specified config",
/configs: "list all configs",
/meta/app: "ip:port/meta/app?app_name=temp",
/meta/app/duplication: "ip:port/meta/app/duplication?name=<app_name>",
/meta/app/query_bulk_load: "ip:port/meta/query_bulk_load?name=temp",
/meta/app/start_bulk_load: "ip:port/meta/start_bulk_load",
/meta/app/start_compaction: "ip:port/meta/start_compaction",
/meta/app/usage_scenario: "ip:port/meta/app/usage_scenario",
/meta/app_envs: "ip:port/meta/app_envs?name=temp",
/meta/apps: "ip:port/meta/apps",
/meta/backup_policy: "ip:port/meta/backup_policy",
/meta/cluster: "ip:port/meta/cluster",
/meta/nodes: "ip:port/meta/nodes",
/perfCounter: "Gets the value of a perf counter",
/pprof/cmdline: "ip:port/pprof/cmdline",
/pprof/growth: "ip:port/pprof/growth",
/pprof/heap: "ip:port/pprof/heap",
/pprof/profile: "ip:port/pprof/profile",
/pprof/symbol: "ip:port/pprof/symbol",
/recentStartTime: "Gets the server start time.",
/updateConfig: "Updates the value of a config",
/version: "Gets the server version."
}
***
Naturally, I begin use client to run example, i use go-client for it, and got the error *** panic: pegasus table configuration query failed: failed to connect table(temp): ERR_OBJECT_NOT_FOUND ***, it happen on [0].
I have not idea for resolve it, looking forward to your reply, Thanks.
[0] https://github.com/apache/incubator-pegasus/blob/master/go-client/example/main.go#L55
Best Regards,
Lan Lian
---- Replied Message ----
| From | Yingchun Lai<ac...@gmail.com> |
| Date | 8/27/2022 22:27 |
| To | <de...@pegasus.apache.org> |
| Subject | Re: How can i run pegasus from container image |
Hi Lan Liang,
It's true that build Pegasus will consume some memory, you can reduce
the memory consumed with lower parallel by using:
$ docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh build -c -j 1"
In the 2nd step, you can start the cluster by using *** docker run -v
$PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh start_onebox" ***
And, have you tried this way [1] to start a Pegasus cluster?
Your report is very valuable for us, we will improve the docs and make
the demo more easier to use, thanks!
1. https://github.com/apache/incubator-pegasus/tree/master/docker/pegasus-docker-compose
Best regards,
Yingchun Lai
On Sat, Aug 27, 2022 at 7:43 PM Lan Liang <li...@163.com> wrote:
Hi Yingchun,
Thanks for your confirmed.
I have run it success on the github action [0] and recompile pegasus and got the different error.
Run command of ** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build -c" **,
And Got error:
The first time,
***
...
[ 76%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service_app.cpp.o
[ 77%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_split_service.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica.cpp.o] Error 4
make[2]: *** Waiting for unfinished jobs....
[ 77%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_simple.cpp.o
[ 77%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_utils.cpp.o
[ 78%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_zookeeper.cpp.o
make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 78%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/partition_guardian.cpp.o
[ 78%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_load_balancer.cpp.o
[ 79%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state.cpp.o
[ 79%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state_restore.cpp.o
[ 79%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/duplication_info.cpp.o
[ 80%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/meta_duplication_service.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service.cpp.o] Error 4
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
make: *** [all] Error 2
***
The second time,
***
[ 76%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
[ 76%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_stub.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/cluster_balance_policy.cpp.o] Error 4
make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 76%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_test_utils.cpp.o
[ 77%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server
***
Third time
***
[ 78%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_config.cpp.o] Error 4
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all] Error 2
make: *** [all] Error 2
***
It's working on Centos7, virtual machine of 6G4U.
Then i upgrade the machine to 10G6U and compile successed !,
Finally, ready to run pegasus, and Got nothing...
1. pack server
docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh pack_server"
2. start service for pegasus
docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "DSN_ROOT/bin/pegasus_server/pegasus_server "
And return log say:
***
...
12000K .......... .......... .......... .......... .......... 99% 6.59M 0s
12050K .......... .......... .......... .......... ....... 100% 7.11M=2.9s
2022-08-27 11:25:31 (4.14 MB/s) - 'apache-zookeeper-3.7.0-bin.tar.gz' saved [12387614/12387614]
Decompressing zookeeper...
ZooKeeper JMX enabled by default
Using config: /root/apache/.zk_install/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
Zookeeper started at port 22181
starting server
cd /root/apache/onebox/meta1 && /root/apache/onebox/meta1/pegasus_server config.ini -app_list meta &>result &
***
It working for zookeeper and have no any process about pegasus, IMO, It's too hard to starte pegasus, this is so frustrating for me :(
[0] https://github.com/liangyuanpeng/pegasus/actions/runs/2920594283
Best Regards,
Lan Liang
---- Replied Message ----
| From | Yingchun Lai<ac...@gmail.com> |
| Date | 8/25/2022 23:52 |
| To | <de...@pegasus.apache.org> |
| Subject | Re: How can i run pegasus from container image |
Hi Lan Liang,
Thanks to your report! The doc is out of date, we will update it later.
What is the full command when you got the last error you mentioned
above. I have tried but not reproduced it by commands:
***
$ git clone git@github.com:apache/incubator-pegasus.git pegasus_test
$ cd pegasus_test
$ docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh build -c"
Build start time: Thu Aug 25 15:36:59 UTC 2022
Start building third-parties...
~/apache/thirdparty/build ~/apache/thirdparty
-- The C compiler identification is GNU 7.3.1
-- The CXX compiler identification is GNU 7.3.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/rh/devtoolset-7/root/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/rh/devtoolset-7/root/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Setting up third-parties...
-- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW
-- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW - Success
-- USE_JEMALLOC = OFF
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
BOOST_ROOT
Boost_NO_BOOST_CMAKE
Boost_NO_SYSTEM_PATHS
ENABLE_GCOV
ENABLE_GPERF
-- Build files have been written to: /root/apache/thirdparty/build
[ 2%] Creating directories for 'googletest'
...
***
And is it the full output, it seems the build stage of thirdparty has
been skipped?
compiling thrift file dsn.layer2.thrift ...
mkdir /root/apache/idl/output
/root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out output dsn.layer2.thrift
...
Best regards,
Yingchun Lai
On Wed, Aug 24, 2022 at 11:53 PM Lan Liang <li...@163.com> wrote:
* /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or directory *.
The reason is run the source code packaged by windwos in linux, which contains the symbol of the windwos file.
Download source again and compile it on the Centos7, also failed. From image of apache/pegasus:build-env-centos7.
***
compiling thrift file dsn.layer2.thrift ...
mkdir /root/apache/idl/output
/root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out output dsn.layer2.thrift
mv output/dsn.layer2_types.h /root/apache/src/rdsn/include/dsn/cpp/serialization_helper
mv output/dsn.layer2_types.cpp /root/apache/src/rdsn/src/runtime
rm -rf /root/apache/idl/output
done
Gen git_commit.h ...
~/apache/src ~/apache ~/apache
fatal: Not a git repository (or any parent up to mount point /root/apache)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
PEGASUS_GIT_COMMIT=non-git-repo
Generating /root/apache/src/include/pegasus/git_commit.h...
[Wed Aug 24 15:47:02 UTC 2022] Building Pegasus ...
~/apache/src/release__builder ~/apache/src ~/apache ~/apache
make: /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake: Command not found
make: *** [cmake_check_build_system] Error 127
***
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<li...@163.com> |
| Date | 8/24/2022 10:15 |
| To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
| Subject | Re: How can i run pegasus from container image |
Just fill the link for [0].
[0] https://pegasus.apache.org/docs/build/compile-by-docker/
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<li...@163.com> |
| Date | 8/23/2022 23:43 |
| To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
| Subject | How can i run pegasus from container image |
Hi pegasus community:
I want to run pegasus from docker and i follow the doc of compile-by-docker [0] to try it. And the container image apachepegasus/build-env:2.1.0-centos7 is not exist.
So i found the image of apache/pegasus:build-env-centos7 and run command of *** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build -c" *** to compile on the pegasus source, and just got the error of * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or directory *.
And it would be great if i can run the pegasus from some docker image.
Remind me if i missed something,Thanks.
Best Regards,
Lan Liang
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
Re: How can i run pegasus from container image
Posted by Yingchun Lai <ac...@gmail.com>.
Hi Lan Liang,
It's true that build Pegasus will consume some memory, you can reduce
the memory consumed with lower parallel by using:
$ docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh build -c -j 1"
In the 2nd step, you can start the cluster by using *** docker run -v
$PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh start_onebox" ***
And, have you tried this way [1] to start a Pegasus cluster?
Your report is very valuable for us, we will improve the docs and make
the demo more easier to use, thanks!
1. https://github.com/apache/incubator-pegasus/tree/master/docker/pegasus-docker-compose
Best regards,
Yingchun Lai
On Sat, Aug 27, 2022 at 7:43 PM Lan Liang <li...@163.com> wrote:
>
> Hi Yingchun,
>
> Thanks for your confirmed.
>
> I have run it success on the github action [0] and recompile pegasus and got the different error.
>
> Run command of ** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build -c" **,
>
> And Got error:
>
> The first time,
> ***
> ...
> [ 76%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service_app.cpp.o
> [ 77%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_split_service.cpp.o
>
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica.cpp.o] Error 4
> make[2]: *** Waiting for unfinished jobs....
> [ 77%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_simple.cpp.o
> [ 77%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_utils.cpp.o
> [ 78%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_zookeeper.cpp.o
> make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all] Error 2
> make[1]: *** Waiting for unfinished jobs....
> [ 78%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/partition_guardian.cpp.o
> [ 78%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_load_balancer.cpp.o
> [ 79%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state.cpp.o
> [ 79%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state_restore.cpp.o
> [ 79%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/duplication_info.cpp.o
> [ 80%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/meta_duplication_service.cpp.o
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service.cpp.o] Error 4
> make[2]: *** Waiting for unfinished jobs....
> make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
> make: *** [all] Error 2
> ***
>
>
> The second time,
> ***
> [ 76%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
> [ 76%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_stub.cpp.o
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/cluster_balance_policy.cpp.o] Error 4
> make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
> make[1]: *** Waiting for unfinished jobs....
> [ 76%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_test_utils.cpp.o
> [ 77%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server
> ***
>
> Third time
> ***
> [ 78%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_config.cpp.o] Error 4
> make[2]: *** Waiting for unfinished jobs....
> make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all] Error 2
> make: *** [all] Error 2
> ***
>
>
> It's working on Centos7, virtual machine of 6G4U.
>
> Then i upgrade the machine to 10G6U and compile successed !,
>
>
> Finally, ready to run pegasus, and Got nothing...
>
> 1. pack server
>
> docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh pack_server"
>
> 2. start service for pegasus
>
>
> docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "DSN_ROOT/bin/pegasus_server/pegasus_server "
>
> And return log say:
>
> ***
> ...
> 12000K .......... .......... .......... .......... .......... 99% 6.59M 0s
> 12050K .......... .......... .......... .......... ....... 100% 7.11M=2.9s
>
> 2022-08-27 11:25:31 (4.14 MB/s) - 'apache-zookeeper-3.7.0-bin.tar.gz' saved [12387614/12387614]
>
> Decompressing zookeeper...
> ZooKeeper JMX enabled by default
> Using config: /root/apache/.zk_install/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
> Starting zookeeper ... STARTED
> Zookeeper started at port 22181
> starting server
> cd /root/apache/onebox/meta1 && /root/apache/onebox/meta1/pegasus_server config.ini -app_list meta &>result &
> ***
>
> It working for zookeeper and have no any process about pegasus, IMO, It's too hard to starte pegasus, this is so frustrating for me :(
>
>
>
>
> [0] https://github.com/liangyuanpeng/pegasus/actions/runs/2920594283
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Yingchun Lai<ac...@gmail.com> |
> | Date | 8/25/2022 23:52 |
> | To | <de...@pegasus.apache.org> |
> | Subject | Re: How can i run pegasus from container image |
> Hi Lan Liang,
>
> Thanks to your report! The doc is out of date, we will update it later.
> What is the full command when you got the last error you mentioned
> above. I have tried but not reproduced it by commands:
> ***
> $ git clone git@github.com:apache/incubator-pegasus.git pegasus_test
> $ cd pegasus_test
> $ docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
> /bin/bash -c "./run.sh build -c"
> Build start time: Thu Aug 25 15:36:59 UTC 2022
> Start building third-parties...
> ~/apache/thirdparty/build ~/apache/thirdparty
> -- The C compiler identification is GNU 7.3.1
> -- The CXX compiler identification is GNU 7.3.1
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Check for working C compiler: /opt/rh/devtoolset-7/root/bin/gcc - skipped
> -- Detecting C compile features
> -- Detecting C compile features - done
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Check for working CXX compiler: /opt/rh/devtoolset-7/root/bin/g++ - skipped
> -- Detecting CXX compile features
> -- Detecting CXX compile features - done
> -- Setting up third-parties...
> -- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW
> -- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW - Success
> -- USE_JEMALLOC = OFF
> -- Configuring done
> -- Generating done
> CMake Warning:
> Manually-specified variables were not used by the project:
>
> BOOST_ROOT
> Boost_NO_BOOST_CMAKE
> Boost_NO_SYSTEM_PATHS
> ENABLE_GCOV
> ENABLE_GPERF
>
>
> -- Build files have been written to: /root/apache/thirdparty/build
> [ 2%] Creating directories for 'googletest'
> ...
> ***
>
> And is it the full output, it seems the build stage of thirdparty has
> been skipped?
> compiling thrift file dsn.layer2.thrift ...
> mkdir /root/apache/idl/output
> /root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out output dsn.layer2.thrift
> ...
>
> Best regards,
> Yingchun Lai
>
> On Wed, Aug 24, 2022 at 11:53 PM Lan Liang <li...@163.com> wrote:
>
>
>
> * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or directory *.
> The reason is run the source code packaged by windwos in linux, which contains the symbol of the windwos file.
>
> Download source again and compile it on the Centos7, also failed. From image of apache/pegasus:build-env-centos7.
>
> ***
> compiling thrift file dsn.layer2.thrift ...
> mkdir /root/apache/idl/output
> /root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out output dsn.layer2.thrift
> mv output/dsn.layer2_types.h /root/apache/src/rdsn/include/dsn/cpp/serialization_helper
> mv output/dsn.layer2_types.cpp /root/apache/src/rdsn/src/runtime
> rm -rf /root/apache/idl/output
>
> done
> Gen git_commit.h ...
> ~/apache/src ~/apache ~/apache
> fatal: Not a git repository (or any parent up to mount point /root/apache)
> Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
> PEGASUS_GIT_COMMIT=non-git-repo
> Generating /root/apache/src/include/pegasus/git_commit.h...
> [Wed Aug 24 15:47:02 UTC 2022] Building Pegasus ...
> ~/apache/src/release__builder ~/apache/src ~/apache ~/apache
> make: /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake: Command not found
> make: *** [cmake_check_build_system] Error 127
> ***
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Lan Liang<li...@163.com> |
> | Date | 8/24/2022 10:15 |
> | To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
> | Subject | Re: How can i run pegasus from container image |
> Just fill the link for [0].
>
> [0] https://pegasus.apache.org/docs/build/compile-by-docker/
>
>
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Lan Liang<li...@163.com> |
> | Date | 8/23/2022 23:43 |
> | To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
> | Subject | How can i run pegasus from container image |
> Hi pegasus community:
>
> I want to run pegasus from docker and i follow the doc of compile-by-docker [0] to try it. And the container image apachepegasus/build-env:2.1.0-centos7 is not exist.
>
> So i found the image of apache/pegasus:build-env-centos7 and run command of *** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build -c" *** to compile on the pegasus source, and just got the error of * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or directory *.
>
> And it would be great if i can run the pegasus from some docker image.
>
> Remind me if i missed something,Thanks.
>
>
>
>
> Best Regards,
> Lan Liang
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
> For additional commands, e-mail: dev-help@pegasus.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
Re: How can i run pegasus from container image
Posted by Yingchun Lai <ac...@gmail.com>.
Hi Lan Liang,
Good news!
./pegasus_shell: error while loading shared libraries: libjvm.so: cannot
> open shared object file: No such file or directory
libjvm.so should be added to $LD_LIBRARY_PATH, the GitHub workflow yaml [1]
shows the full steps to run the tools/test. Of course, I think the C++
shell tool is not necessary to load libjvm.so if I don't miss anything, we
can check and improve it later.
It always say *CreateApp failed: ErrorCode({Errno:ERR_INVALID_PARAMETERS})*
> , it doesn't any help for me, it would be great if the command can tell me
> what should i do for it.
Indeed, it can return the detailed error messages for that. In fact the
detailed error message is that the table named 'temp' exists in the cluster
by default as you can see above. Since the 'temp' table has 4
partitions with 3 replica factor, when you attempted to create it again
with different parameters, it will return error like
'ERR_INVALID_PARAMETERS'.
You are welcome! Thanks to your report in the mail list, we will try to
make Pegasus easier to run later, as what you mentioned above.
And we are willing to see if you can create issues and submit patches for
that.
1.
https://github.com/apache/incubator-pegasus/blob/master/.github/workflows/lint_and_test_cpp.yaml#L167
Best regards,
Yingchun Lai
On Mon, Aug 29, 2022 at 10:27 PM Lan Liang <li...@163.com> wrote:
> Hi Yingchun,
>
> Thanks for your reply.
>
>
> I have try shell cli and just got the error:
>
> ***
> [root@chaos incubator-pegasus]# ./run.sh shell
> ./pegasus_shell: error while loading shared libraries: libjvm.so: cannot
> open shared object file: No such file or directory
> ***
>
>
> I see that have a admin-cli from golang implement, So i decided to use
> admin-cli instead of shell and ignore this error.
>
> In link [0] I can not find doc on how i can start running an example, so i
> just want to create a table on admin-cli first.
>
> ***
> Pegasus-AdminCli-1.2.0 » create temp
> error: CreateApp failed: ErrorCode({Errno:ERR_INVALID_PARAMETERS})
> Pegasus-AdminCli-1.2.0 » create -p 2 -r 1 temp
> error: CreateApp failed: ErrorCode({Errno:ERR_INVALID_PARAMETERS})
> Pegasus-AdminCli-1.2.0 » create -p 1 -r 2 temp
> error: partitions number must be a multiply of 2
> ***
>
> It always say *CreateApp failed:
> ErrorCode({Errno:ERR_INVALID_PARAMETERS})* , it doesn't any help for me,
> it would be great if the command can tell me what should i do for it.
>
> Luckily, have two table is exist on server.
>
> ***
> Pegasus-AdminCli-1.2.0 » ls
>
> +----+------+------------+-----------+----------------+---------------+------------+---------------+-----------------+
> | ID | Name | Partitions | Unhealthy | WriteUnhealthy | ReadUnhealthy |
> CreateTime | WReqRateLimit | WBytesRateLimit |
>
> +----+------+------------+-----------+----------------+---------------+------------+---------------+-----------------+
> | 1 | stat | 4 | 4 | 4 | 0 |
> 2022-08-29 | | |
> | 2 | test | 4 | 4 | 4 | 0 |
> 2022-08-29 | | |
>
> +----+------+------------+-----------+----------------+---------------+------------+---------------+-----------------+
> ***
>
> And use command of pegic run example for get set successed !
>
> ***
> pegic » use test
> ok
> pegic » set h s world
>
> ok
>
> Encoding:
> - HashKey: UTF8
> - SortKey: UTF8
> - Value: UTF8
>
> pegic » get h s world
> error: invalid usage of command 'get' (unconsumed input 'world'), try
> 'help'
> pegic » get h s
> h : s : world
>
> TTL(Time-To-Live) is not set
>
> Encoding:
> - HashKey: UTF8
> - SortKey: UTF8
> - Value: UTF8
> ***
>
> Thanks for work for golang command, this helped me a lot.
>
>
>
>
> [0]. https://pegasus.apache.org/docs/tools/shell/
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Yingchun Lai<ac...@gmail.com> |
> | Date | 8/29/2022 00:25 |
> | To | <de...@pegasus.apache.org> |
> | Subject | Re: How can i run pegasus from container image |
> Hi Lan Liang,
>
> 1.
>
> I have request meta server and got the response :
> ***
> [root@chaos incubator-pegasus]# curl 192.168.56.11:34601
> {
> /: "Lists all supported calls",
> ...
> /version: "Gets the server version."
> }
> ***
>
> Good, that is to say the cluster is running! You requested the HTTP
> service, the server responded with the full requests it supports.
>
> 2,
>
> Naturally, I begin use client to run example, i use go-client for it, and
> got the error *** panic: pegasus table configuration query failed: failed
> to connect table(temp): ERR_OBJECT_NOT_FOUND ***, it happen on [0].
>
> I'm not sure if the table named 'temp' has been created or not when you use
> the Go client to access it. You can use the command "ls -d" in shell tools
> [1] to check it.
>
> 3.
>
> container always download file for hadoop on the start,it's so weird for
> me.
>
> Yes, it is. The feature "Backup & Restore Pegasus data on HDFS" uses
> libhdfs to operate on HDFS, in fact, we download it only for tests, we
> can remove it later.
>
> Thanks for your report!
>
>
> 1. https://pegasus.apache.org/docs/tools/shell/
>
> Best regards,
> Yingchun Lai
>
> Best regards,
> Yingchun Lai
>
>
> On Sun, Aug 28, 2022 at 10:17 PM Lan Liang <li...@163.com>
> wrote:
>
> Have three container is running,But replica server and meta server have
> not any log for request.
>
>
> The replica container log,
> ******
> [root@chaos incubator-pegasus]# docker ps|grep onebox
> 2562f51268c2 0246c7eb5547
> "/entrypoint.sh repl…" 4 minutes ago Up 4 minutes 0.0.0.0:49159
> ->34801/tcp,
> :::49159->34801/tcp onebox-docker_replica1_1
> 21a506db99f4 zookeeper:3.4
> "/docker-entrypoint.…" 4 minutes ago Up 4 minutes 2888/tcp,
> 3888/tcp, 0.0.0.0:49160->2181/tcp, :::49160->2181/tcp
> onebox-docker_zookeeper_1
> 1b769b90e65f 0246c7eb5547
> "/entrypoint.sh meta" 4 minutes ago Up 4 minutes 0.0.0.0:34601
> ->34601/tcp,
> :::34601->34601/tcp onebox-docker_meta1_1
>
> [root@chaos incubator-pegasus]# docker logs -f --tail=20 256
> 239600K .......... .......... .......... .......... .......... 99% 588K 0s
> 239650K .......... .......... .......... .......... .......... 99% 243K 0s
> 239700K .......... .......... .......... .......... .......... 99% 9.90M 0s
> 239750K .......... .......... .......... .......... .......... 99% 8.90M 0s
> 239800K .......... .......... .......... .......... .......... 99% 15.8M 0s
> 239850K .......... .......... .......... .......... .......... 99% 7.62M 0s
> 239900K .......... .......... .......... .......... .......... 99% 19.2M 0s
> 239950K .......... .......... .......... .......... .......... 99% 1.86M 0s
> 240000K .......... .......... .......... .......... .......... 99% 1.68M 0s
> 240050K .......... .......... .......... .......... .......... 99% 11.0M 0s
> 240100K .......... .......... .......... .......... .......... 99% 434K 0s
> 240150K .......... .......... .......... .......... .......... 99% 10.8M 0s
> 240200K .......... .......... .......... .......... .......... 99% 1.89M 0s
> 240250K .......... .......... .......... .......... .......... 99% 627K 0s
> 240300K .......... .......... .......... .......... .......... 99% 2.38M 0s
> 240350K ..... 100%
> 105M=1m53s
>
> 2022-08-28 12:32:54 (2.09 MB/s) - 'hadoop-2.8.4.tar.gz' saved
> [246123562/246123562]
>
> Decompressing HDFS...
>
> ******
>
>
> BTW, container always download file for hadoop on the start,it's so weird
> for me.
>
>
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Lan Liang<li...@163.com> |
> | Date | 8/27/2022 19:42 |
> | To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
> | Subject | Re: How can i run pegasus from container image |
> Hi Yingchun,
>
> Thanks for your confirmed.
>
> I have run it success on the github action [0] and recompile pegasus and
> got the different error.
>
> Run command of ** docker run -v $PWD:/root/apache
> apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build
> -c" **,
>
> And Got error:
>
> The first time,
> ***
> ...
> [ 76%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service_app.cpp.o
> [ 77%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_split_service.cpp.o
>
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: ***
> [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica.cpp.o]
> Error 4
> make[2]: *** Waiting for unfinished jobs....
> [ 77%] Building CXX object
>
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_simple.cpp.o
> [ 77%] Building CXX object
>
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_utils.cpp.o
> [ 78%] Building CXX object
>
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_zookeeper.cpp.o
> make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all]
> Error 2
> make[1]: *** Waiting for unfinished jobs....
> [ 78%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/partition_guardian.cpp.o
> [ 78%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_load_balancer.cpp.o
> [ 79%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state.cpp.o
> [ 79%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state_restore.cpp.o
> [ 79%] Building CXX object
>
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/duplication_info.cpp.o
> [ 80%] Building CXX object
>
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/meta_duplication_service.cpp.o
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: ***
> [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service.cpp.o] Error
> 4
> make[2]: *** Waiting for unfinished jobs....
> make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
> make: *** [all] Error 2
> ***
>
>
> The second time,
> ***
> [ 76%] Building CXX object
>
> src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
> [ 76%] Building CXX object
> src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_stub.cpp.o
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: ***
>
> [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/cluster_balance_policy.cpp.o]
> Error 4
> make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
> make[1]: *** Waiting for unfinished jobs....
> [ 76%] Building CXX object
>
> src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_test_utils.cpp.o
> [ 77%] Building CXX object
> src/rdsn/src/replica/CMakeFiles/dsn_replica_server
> ***
>
> Third time
> ***
> [ 78%] Building CXX object
>
> src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: ***
>
> [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_config.cpp.o]
> Error 4
> make[2]: *** Waiting for unfinished jobs....
> make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all]
> Error 2
> make: *** [all] Error 2
> ***
>
>
> It's working on Centos7, virtual machine of 6G4U.
>
> Then i upgrade the machine to 10G6U and compile successed !,
>
>
> Finally, ready to run pegasus, and Got nothing...
>
> 1. pack server
>
> docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
> /bin/bash -c "./run.sh pack_server"
>
> 2. start service for pegasus
>
>
> docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
> /bin/bash -c "DSN_ROOT/bin/pegasus_server/pegasus_server "
>
> And return log say:
>
> ***
> ...
> 12000K .......... .......... .......... .......... .......... 99% 6.59M 0s
> 12050K .......... .......... .......... .......... ....... 100%
> 7.11M=2.9s
>
> 2022-08-27 11:25:31 (4.14 MB/s) - 'apache-zookeeper-3.7.0-bin.tar.gz'
> saved [12387614/12387614]
>
> Decompressing zookeeper...
> ZooKeeper JMX enabled by default
> Using config:
> /root/apache/.zk_install/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
> Starting zookeeper ... STARTED
> Zookeeper started at port 22181
> starting server
> cd /root/apache/onebox/meta1 && /root/apache/onebox/meta1/pegasus_server
> config.ini -app_list meta &>result &
> ***
>
> It working for zookeeper and have no any process about pegasus, IMO, It's
> too hard to starte pegasus, this is so frustrating for me :(
>
>
>
>
> [0] https://github.com/liangyuanpeng/pegasus/actions/runs/2920594283
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Yingchun Lai<ac...@gmail.com> |
> | Date | 8/25/2022 23:52 |
> | To | <de...@pegasus.apache.org> |
> | Subject | Re: How can i run pegasus from container image |
> Hi Lan Liang,
>
> Thanks to your report! The doc is out of date, we will update it later.
> What is the full command when you got the last error you mentioned
> above. I have tried but not reproduced it by commands:
> ***
> $ git clone git@github.com:apache/incubator-pegasus.git pegasus_test
> $ cd pegasus_test
> $ docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
> /bin/bash -c "./run.sh build -c"
> Build start time: Thu Aug 25 15:36:59 UTC 2022
> Start building third-parties...
> ~/apache/thirdparty/build ~/apache/thirdparty
> -- The C compiler identification is GNU 7.3.1
> -- The CXX compiler identification is GNU 7.3.1
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Check for working C compiler: /opt/rh/devtoolset-7/root/bin/gcc -
> skipped
> -- Detecting C compile features
> -- Detecting C compile features - done
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Check for working CXX compiler: /opt/rh/devtoolset-7/root/bin/g++ -
> skipped
> -- Detecting CXX compile features
> -- Detecting CXX compile features - done
> -- Setting up third-parties...
> -- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW
> -- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW - Success
> -- USE_JEMALLOC = OFF
> -- Configuring done
> -- Generating done
> CMake Warning:
> Manually-specified variables were not used by the project:
>
> BOOST_ROOT
> Boost_NO_BOOST_CMAKE
> Boost_NO_SYSTEM_PATHS
> ENABLE_GCOV
> ENABLE_GPERF
>
>
> -- Build files have been written to: /root/apache/thirdparty/build
> [ 2%] Creating directories for 'googletest'
> ...
> ***
>
> And is it the full output, it seems the build stage of thirdparty has
> been skipped?
> compiling thrift file dsn.layer2.thrift ...
> mkdir /root/apache/idl/output
> /root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out
> output dsn.layer2.thrift
> ...
>
> Best regards,
> Yingchun Lai
>
> On Wed, Aug 24, 2022 at 11:53 PM Lan Liang <li...@163.com>
> wrote:
>
>
>
> * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or
> directory *.
> The reason is run the source code packaged by windwos in linux, which
> contains the symbol of the windwos file.
>
> Download source again and compile it on the Centos7, also failed. From
> image of apache/pegasus:build-env-centos7.
>
> ***
> compiling thrift file dsn.layer2.thrift ...
> mkdir /root/apache/idl/output
> /root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out
> output dsn.layer2.thrift
> mv output/dsn.layer2_types.h
> /root/apache/src/rdsn/include/dsn/cpp/serialization_helper
> mv output/dsn.layer2_types.cpp /root/apache/src/rdsn/src/runtime
> rm -rf /root/apache/idl/output
>
> done
> Gen git_commit.h ...
> ~/apache/src ~/apache ~/apache
> fatal: Not a git repository (or any parent up to mount point /root/apache)
> Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
> PEGASUS_GIT_COMMIT=non-git-repo
> Generating /root/apache/src/include/pegasus/git_commit.h...
> [Wed Aug 24 15:47:02 UTC 2022] Building Pegasus ...
> ~/apache/src/release__builder ~/apache/src ~/apache ~/apache
> make: /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake: Command
> not found
> make: *** [cmake_check_build_system] Error 127
> ***
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Lan Liang<li...@163.com> |
> | Date | 8/24/2022 10:15 |
> | To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
> | Subject | Re: How can i run pegasus from container image |
> Just fill the link for [0].
>
> [0] https://pegasus.apache.org/docs/build/compile-by-docker/
>
>
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Lan Liang<li...@163.com> |
> | Date | 8/23/2022 23:43 |
> | To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
> | Subject | How can i run pegasus from container image |
> Hi pegasus community:
>
> I want to run pegasus from docker and i follow the doc of
> compile-by-docker [0] to try it. And the container image
> apachepegasus/build-env:2.1.0-centos7 is not exist.
>
> So i found the image of apache/pegasus:build-env-centos7 and run command
> of *** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
> /bin/bash -c "./run.sh build -c" *** to compile on the pegasus source, and
> just got the error of * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter:
> No such file or directory *.
>
> And it would be great if i can run the pegasus from some docker image.
>
> Remind me if i missed something,Thanks.
>
>
>
>
> Best Regards,
> Lan Liang
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
> For additional commands, e-mail: dev-help@pegasus.apache.org
>
>
Re: How can i run pegasus from container image
Posted by Lan Liang <li...@163.com>.
Hi Yingchun,
Thanks for your reply.
I have try shell cli and just got the error:
***
[root@chaos incubator-pegasus]# ./run.sh shell
./pegasus_shell: error while loading shared libraries: libjvm.so: cannot open shared object file: No such file or directory
***
I see that have a admin-cli from golang implement, So i decided to use admin-cli instead of shell and ignore this error.
In link [0] I can not find doc on how i can start running an example, so i just want to create a table on admin-cli first.
***
Pegasus-AdminCli-1.2.0 » create temp
error: CreateApp failed: ErrorCode({Errno:ERR_INVALID_PARAMETERS})
Pegasus-AdminCli-1.2.0 » create -p 2 -r 1 temp
error: CreateApp failed: ErrorCode({Errno:ERR_INVALID_PARAMETERS})
Pegasus-AdminCli-1.2.0 » create -p 1 -r 2 temp
error: partitions number must be a multiply of 2
***
It always say *CreateApp failed: ErrorCode({Errno:ERR_INVALID_PARAMETERS})* , it doesn't any help for me, it would be great if the command can tell me what should i do for it.
Luckily, have two table is exist on server.
***
Pegasus-AdminCli-1.2.0 » ls
+----+------+------------+-----------+----------------+---------------+------------+---------------+-----------------+
| ID | Name | Partitions | Unhealthy | WriteUnhealthy | ReadUnhealthy | CreateTime | WReqRateLimit | WBytesRateLimit |
+----+------+------------+-----------+----------------+---------------+------------+---------------+-----------------+
| 1 | stat | 4 | 4 | 4 | 0 | 2022-08-29 | | |
| 2 | test | 4 | 4 | 4 | 0 | 2022-08-29 | | |
+----+------+------------+-----------+----------------+---------------+------------+---------------+-----------------+
***
And use command of pegic run example for get set successed !
***
pegic » use test
ok
pegic » set h s world
ok
Encoding:
- HashKey: UTF8
- SortKey: UTF8
- Value: UTF8
pegic » get h s world
error: invalid usage of command 'get' (unconsumed input 'world'), try 'help'
pegic » get h s
h : s : world
TTL(Time-To-Live) is not set
Encoding:
- HashKey: UTF8
- SortKey: UTF8
- Value: UTF8
***
Thanks for work for golang command, this helped me a lot.
[0]. https://pegasus.apache.org/docs/tools/shell/
Best Regards,
Lan Liang
---- Replied Message ----
| From | Yingchun Lai<ac...@gmail.com> |
| Date | 8/29/2022 00:25 |
| To | <de...@pegasus.apache.org> |
| Subject | Re: How can i run pegasus from container image |
Hi Lan Liang,
1.
I have request meta server and got the response :
***
[root@chaos incubator-pegasus]# curl 192.168.56.11:34601
{
/: "Lists all supported calls",
...
/version: "Gets the server version."
}
***
Good, that is to say the cluster is running! You requested the HTTP
service, the server responded with the full requests it supports.
2,
Naturally, I begin use client to run example, i use go-client for it, and
got the error *** panic: pegasus table configuration query failed: failed
to connect table(temp): ERR_OBJECT_NOT_FOUND ***, it happen on [0].
I'm not sure if the table named 'temp' has been created or not when you use
the Go client to access it. You can use the command "ls -d" in shell tools
[1] to check it.
3.
container always download file for hadoop on the start,it's so weird for
me.
Yes, it is. The feature "Backup & Restore Pegasus data on HDFS" uses
libhdfs to operate on HDFS, in fact, we download it only for tests, we
can remove it later.
Thanks for your report!
1. https://pegasus.apache.org/docs/tools/shell/
Best regards,
Yingchun Lai
Best regards,
Yingchun Lai
On Sun, Aug 28, 2022 at 10:17 PM Lan Liang <li...@163.com> wrote:
Have three container is running,But replica server and meta server have
not any log for request.
The replica container log,
******
[root@chaos incubator-pegasus]# docker ps|grep onebox
2562f51268c2 0246c7eb5547
"/entrypoint.sh repl…" 4 minutes ago Up 4 minutes 0.0.0.0:49159->34801/tcp,
:::49159->34801/tcp onebox-docker_replica1_1
21a506db99f4 zookeeper:3.4
"/docker-entrypoint.…" 4 minutes ago Up 4 minutes 2888/tcp,
3888/tcp, 0.0.0.0:49160->2181/tcp, :::49160->2181/tcp
onebox-docker_zookeeper_1
1b769b90e65f 0246c7eb5547
"/entrypoint.sh meta" 4 minutes ago Up 4 minutes 0.0.0.0:34601->34601/tcp,
:::34601->34601/tcp onebox-docker_meta1_1
[root@chaos incubator-pegasus]# docker logs -f --tail=20 256
239600K .......... .......... .......... .......... .......... 99% 588K 0s
239650K .......... .......... .......... .......... .......... 99% 243K 0s
239700K .......... .......... .......... .......... .......... 99% 9.90M 0s
239750K .......... .......... .......... .......... .......... 99% 8.90M 0s
239800K .......... .......... .......... .......... .......... 99% 15.8M 0s
239850K .......... .......... .......... .......... .......... 99% 7.62M 0s
239900K .......... .......... .......... .......... .......... 99% 19.2M 0s
239950K .......... .......... .......... .......... .......... 99% 1.86M 0s
240000K .......... .......... .......... .......... .......... 99% 1.68M 0s
240050K .......... .......... .......... .......... .......... 99% 11.0M 0s
240100K .......... .......... .......... .......... .......... 99% 434K 0s
240150K .......... .......... .......... .......... .......... 99% 10.8M 0s
240200K .......... .......... .......... .......... .......... 99% 1.89M 0s
240250K .......... .......... .......... .......... .......... 99% 627K 0s
240300K .......... .......... .......... .......... .......... 99% 2.38M 0s
240350K ..... 100%
105M=1m53s
2022-08-28 12:32:54 (2.09 MB/s) - 'hadoop-2.8.4.tar.gz' saved
[246123562/246123562]
Decompressing HDFS...
******
BTW, container always download file for hadoop on the start,it's so weird
for me.
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<li...@163.com> |
| Date | 8/27/2022 19:42 |
| To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
| Subject | Re: How can i run pegasus from container image |
Hi Yingchun,
Thanks for your confirmed.
I have run it success on the github action [0] and recompile pegasus and
got the different error.
Run command of ** docker run -v $PWD:/root/apache
apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build
-c" **,
And Got error:
The first time,
***
...
[ 76%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service_app.cpp.o
[ 77%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_split_service.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: ***
[src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica.cpp.o]
Error 4
make[2]: *** Waiting for unfinished jobs....
[ 77%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_simple.cpp.o
[ 77%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_utils.cpp.o
[ 78%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_zookeeper.cpp.o
make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all]
Error 2
make[1]: *** Waiting for unfinished jobs....
[ 78%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/partition_guardian.cpp.o
[ 78%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_load_balancer.cpp.o
[ 79%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state.cpp.o
[ 79%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state_restore.cpp.o
[ 79%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/duplication_info.cpp.o
[ 80%] Building CXX object
src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/meta_duplication_service.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: ***
[src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service.cpp.o] Error
4
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
make: *** [all] Error 2
***
The second time,
***
[ 76%] Building CXX object
src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
[ 76%] Building CXX object
src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_stub.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: ***
[src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/cluster_balance_policy.cpp.o]
Error 4
make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 76%] Building CXX object
src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_test_utils.cpp.o
[ 77%] Building CXX object
src/rdsn/src/replica/CMakeFiles/dsn_replica_server
***
Third time
***
[ 78%] Building CXX object
src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: ***
[src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_config.cpp.o]
Error 4
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all]
Error 2
make: *** [all] Error 2
***
It's working on Centos7, virtual machine of 6G4U.
Then i upgrade the machine to 10G6U and compile successed !,
Finally, ready to run pegasus, and Got nothing...
1. pack server
docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh pack_server"
2. start service for pegasus
docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "DSN_ROOT/bin/pegasus_server/pegasus_server "
And return log say:
***
...
12000K .......... .......... .......... .......... .......... 99% 6.59M 0s
12050K .......... .......... .......... .......... ....... 100%
7.11M=2.9s
2022-08-27 11:25:31 (4.14 MB/s) - 'apache-zookeeper-3.7.0-bin.tar.gz'
saved [12387614/12387614]
Decompressing zookeeper...
ZooKeeper JMX enabled by default
Using config:
/root/apache/.zk_install/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
Zookeeper started at port 22181
starting server
cd /root/apache/onebox/meta1 && /root/apache/onebox/meta1/pegasus_server
config.ini -app_list meta &>result &
***
It working for zookeeper and have no any process about pegasus, IMO, It's
too hard to starte pegasus, this is so frustrating for me :(
[0] https://github.com/liangyuanpeng/pegasus/actions/runs/2920594283
Best Regards,
Lan Liang
---- Replied Message ----
| From | Yingchun Lai<ac...@gmail.com> |
| Date | 8/25/2022 23:52 |
| To | <de...@pegasus.apache.org> |
| Subject | Re: How can i run pegasus from container image |
Hi Lan Liang,
Thanks to your report! The doc is out of date, we will update it later.
What is the full command when you got the last error you mentioned
above. I have tried but not reproduced it by commands:
***
$ git clone git@github.com:apache/incubator-pegasus.git pegasus_test
$ cd pegasus_test
$ docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh build -c"
Build start time: Thu Aug 25 15:36:59 UTC 2022
Start building third-parties...
~/apache/thirdparty/build ~/apache/thirdparty
-- The C compiler identification is GNU 7.3.1
-- The CXX compiler identification is GNU 7.3.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/rh/devtoolset-7/root/bin/gcc -
skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/rh/devtoolset-7/root/bin/g++ -
skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Setting up third-parties...
-- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW
-- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW - Success
-- USE_JEMALLOC = OFF
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
BOOST_ROOT
Boost_NO_BOOST_CMAKE
Boost_NO_SYSTEM_PATHS
ENABLE_GCOV
ENABLE_GPERF
-- Build files have been written to: /root/apache/thirdparty/build
[ 2%] Creating directories for 'googletest'
...
***
And is it the full output, it seems the build stage of thirdparty has
been skipped?
compiling thrift file dsn.layer2.thrift ...
mkdir /root/apache/idl/output
/root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out
output dsn.layer2.thrift
...
Best regards,
Yingchun Lai
On Wed, Aug 24, 2022 at 11:53 PM Lan Liang <li...@163.com>
wrote:
* /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or
directory *.
The reason is run the source code packaged by windwos in linux, which
contains the symbol of the windwos file.
Download source again and compile it on the Centos7, also failed. From
image of apache/pegasus:build-env-centos7.
***
compiling thrift file dsn.layer2.thrift ...
mkdir /root/apache/idl/output
/root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out
output dsn.layer2.thrift
mv output/dsn.layer2_types.h
/root/apache/src/rdsn/include/dsn/cpp/serialization_helper
mv output/dsn.layer2_types.cpp /root/apache/src/rdsn/src/runtime
rm -rf /root/apache/idl/output
done
Gen git_commit.h ...
~/apache/src ~/apache ~/apache
fatal: Not a git repository (or any parent up to mount point /root/apache)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
PEGASUS_GIT_COMMIT=non-git-repo
Generating /root/apache/src/include/pegasus/git_commit.h...
[Wed Aug 24 15:47:02 UTC 2022] Building Pegasus ...
~/apache/src/release__builder ~/apache/src ~/apache ~/apache
make: /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake: Command
not found
make: *** [cmake_check_build_system] Error 127
***
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<li...@163.com> |
| Date | 8/24/2022 10:15 |
| To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
| Subject | Re: How can i run pegasus from container image |
Just fill the link for [0].
[0] https://pegasus.apache.org/docs/build/compile-by-docker/
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<li...@163.com> |
| Date | 8/23/2022 23:43 |
| To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
| Subject | How can i run pegasus from container image |
Hi pegasus community:
I want to run pegasus from docker and i follow the doc of
compile-by-docker [0] to try it. And the container image
apachepegasus/build-env:2.1.0-centos7 is not exist.
So i found the image of apache/pegasus:build-env-centos7 and run command
of *** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh build -c" *** to compile on the pegasus source, and
just got the error of * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter:
No such file or directory *.
And it would be great if i can run the pegasus from some docker image.
Remind me if i missed something,Thanks.
Best Regards,
Lan Liang
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
Re: How can i run pegasus from container image
Posted by Yingchun Lai <ac...@gmail.com>.
Hi Lan Liang,
1.
> I have request meta server and got the response :
> ***
> [root@chaos incubator-pegasus]# curl 192.168.56.11:34601
> {
> /: "Lists all supported calls",
> ...
> /version: "Gets the server version."
> }
> ***
Good, that is to say the cluster is running! You requested the HTTP
service, the server responded with the full requests it supports.
2,
> Naturally, I begin use client to run example, i use go-client for it, and
> got the error *** panic: pegasus table configuration query failed: failed
> to connect table(temp): ERR_OBJECT_NOT_FOUND ***, it happen on [0].
I'm not sure if the table named 'temp' has been created or not when you use
the Go client to access it. You can use the command "ls -d" in shell tools
[1] to check it.
3.
> container always download file for hadoop on the start,it's so weird for
> me.
Yes, it is. The feature "Backup & Restore Pegasus data on HDFS" uses
libhdfs to operate on HDFS, in fact, we download it only for tests, we
can remove it later.
Thanks for your report!
1. https://pegasus.apache.org/docs/tools/shell/
Best regards,
Yingchun Lai
Best regards,
Yingchun Lai
On Sun, Aug 28, 2022 at 10:17 PM Lan Liang <li...@163.com> wrote:
> Have three container is running,But replica server and meta server have
> not any log for request.
>
>
> The replica container log,
> ******
> [root@chaos incubator-pegasus]# docker ps|grep onebox
> 2562f51268c2 0246c7eb5547
> "/entrypoint.sh repl…" 4 minutes ago Up 4 minutes 0.0.0.0:49159->34801/tcp,
> :::49159->34801/tcp onebox-docker_replica1_1
> 21a506db99f4 zookeeper:3.4
> "/docker-entrypoint.…" 4 minutes ago Up 4 minutes 2888/tcp,
> 3888/tcp, 0.0.0.0:49160->2181/tcp, :::49160->2181/tcp
> onebox-docker_zookeeper_1
> 1b769b90e65f 0246c7eb5547
> "/entrypoint.sh meta" 4 minutes ago Up 4 minutes 0.0.0.0:34601->34601/tcp,
> :::34601->34601/tcp onebox-docker_meta1_1
>
> [root@chaos incubator-pegasus]# docker logs -f --tail=20 256
> 239600K .......... .......... .......... .......... .......... 99% 588K 0s
> 239650K .......... .......... .......... .......... .......... 99% 243K 0s
> 239700K .......... .......... .......... .......... .......... 99% 9.90M 0s
> 239750K .......... .......... .......... .......... .......... 99% 8.90M 0s
> 239800K .......... .......... .......... .......... .......... 99% 15.8M 0s
> 239850K .......... .......... .......... .......... .......... 99% 7.62M 0s
> 239900K .......... .......... .......... .......... .......... 99% 19.2M 0s
> 239950K .......... .......... .......... .......... .......... 99% 1.86M 0s
> 240000K .......... .......... .......... .......... .......... 99% 1.68M 0s
> 240050K .......... .......... .......... .......... .......... 99% 11.0M 0s
> 240100K .......... .......... .......... .......... .......... 99% 434K 0s
> 240150K .......... .......... .......... .......... .......... 99% 10.8M 0s
> 240200K .......... .......... .......... .......... .......... 99% 1.89M 0s
> 240250K .......... .......... .......... .......... .......... 99% 627K 0s
> 240300K .......... .......... .......... .......... .......... 99% 2.38M 0s
> 240350K ..... 100%
> 105M=1m53s
>
> 2022-08-28 12:32:54 (2.09 MB/s) - 'hadoop-2.8.4.tar.gz' saved
> [246123562/246123562]
>
> Decompressing HDFS...
>
> ******
>
>
> BTW, container always download file for hadoop on the start,it's so weird
> for me.
>
>
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Lan Liang<li...@163.com> |
> | Date | 8/27/2022 19:42 |
> | To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
> | Subject | Re: How can i run pegasus from container image |
> Hi Yingchun,
>
> Thanks for your confirmed.
>
> I have run it success on the github action [0] and recompile pegasus and
> got the different error.
>
> Run command of ** docker run -v $PWD:/root/apache
> apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build
> -c" **,
>
> And Got error:
>
> The first time,
> ***
> ...
> [ 76%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service_app.cpp.o
> [ 77%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_split_service.cpp.o
>
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: ***
> [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica.cpp.o]
> Error 4
> make[2]: *** Waiting for unfinished jobs....
> [ 77%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_simple.cpp.o
> [ 77%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_utils.cpp.o
> [ 78%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_zookeeper.cpp.o
> make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all]
> Error 2
> make[1]: *** Waiting for unfinished jobs....
> [ 78%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/partition_guardian.cpp.o
> [ 78%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_load_balancer.cpp.o
> [ 79%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state.cpp.o
> [ 79%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state_restore.cpp.o
> [ 79%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/duplication_info.cpp.o
> [ 80%] Building CXX object
> src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/meta_duplication_service.cpp.o
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: ***
> [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service.cpp.o] Error
> 4
> make[2]: *** Waiting for unfinished jobs....
> make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
> make: *** [all] Error 2
> ***
>
>
> The second time,
> ***
> [ 76%] Building CXX object
> src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
> [ 76%] Building CXX object
> src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_stub.cpp.o
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: ***
> [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/cluster_balance_policy.cpp.o]
> Error 4
> make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
> make[1]: *** Waiting for unfinished jobs....
> [ 76%] Building CXX object
> src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_test_utils.cpp.o
> [ 77%] Building CXX object
> src/rdsn/src/replica/CMakeFiles/dsn_replica_server
> ***
>
> Third time
> ***
> [ 78%] Building CXX object
> src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
> g++: internal compiler error: Killed (program cc1plus)
> Please submit a full bug report,
> with preprocessed source if appropriate.
> See for instructions.
> make[2]: ***
> [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_config.cpp.o]
> Error 4
> make[2]: *** Waiting for unfinished jobs....
> make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all]
> Error 2
> make: *** [all] Error 2
> ***
>
>
> It's working on Centos7, virtual machine of 6G4U.
>
> Then i upgrade the machine to 10G6U and compile successed !,
>
>
> Finally, ready to run pegasus, and Got nothing...
>
> 1. pack server
>
> docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
> /bin/bash -c "./run.sh pack_server"
>
> 2. start service for pegasus
>
>
> docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
> /bin/bash -c "DSN_ROOT/bin/pegasus_server/pegasus_server "
>
> And return log say:
>
> ***
> ...
> 12000K .......... .......... .......... .......... .......... 99% 6.59M 0s
> 12050K .......... .......... .......... .......... ....... 100%
> 7.11M=2.9s
>
> 2022-08-27 11:25:31 (4.14 MB/s) - 'apache-zookeeper-3.7.0-bin.tar.gz'
> saved [12387614/12387614]
>
> Decompressing zookeeper...
> ZooKeeper JMX enabled by default
> Using config:
> /root/apache/.zk_install/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
> Starting zookeeper ... STARTED
> Zookeeper started at port 22181
> starting server
> cd /root/apache/onebox/meta1 && /root/apache/onebox/meta1/pegasus_server
> config.ini -app_list meta &>result &
> ***
>
> It working for zookeeper and have no any process about pegasus, IMO, It's
> too hard to starte pegasus, this is so frustrating for me :(
>
>
>
>
> [0] https://github.com/liangyuanpeng/pegasus/actions/runs/2920594283
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Yingchun Lai<ac...@gmail.com> |
> | Date | 8/25/2022 23:52 |
> | To | <de...@pegasus.apache.org> |
> | Subject | Re: How can i run pegasus from container image |
> Hi Lan Liang,
>
> Thanks to your report! The doc is out of date, we will update it later.
> What is the full command when you got the last error you mentioned
> above. I have tried but not reproduced it by commands:
> ***
> $ git clone git@github.com:apache/incubator-pegasus.git pegasus_test
> $ cd pegasus_test
> $ docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
> /bin/bash -c "./run.sh build -c"
> Build start time: Thu Aug 25 15:36:59 UTC 2022
> Start building third-parties...
> ~/apache/thirdparty/build ~/apache/thirdparty
> -- The C compiler identification is GNU 7.3.1
> -- The CXX compiler identification is GNU 7.3.1
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Check for working C compiler: /opt/rh/devtoolset-7/root/bin/gcc -
> skipped
> -- Detecting C compile features
> -- Detecting C compile features - done
> -- Detecting CXX compiler ABI info
> -- Detecting CXX compiler ABI info - done
> -- Check for working CXX compiler: /opt/rh/devtoolset-7/root/bin/g++ -
> skipped
> -- Detecting CXX compile features
> -- Detecting CXX compile features - done
> -- Setting up third-parties...
> -- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW
> -- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW - Success
> -- USE_JEMALLOC = OFF
> -- Configuring done
> -- Generating done
> CMake Warning:
> Manually-specified variables were not used by the project:
>
> BOOST_ROOT
> Boost_NO_BOOST_CMAKE
> Boost_NO_SYSTEM_PATHS
> ENABLE_GCOV
> ENABLE_GPERF
>
>
> -- Build files have been written to: /root/apache/thirdparty/build
> [ 2%] Creating directories for 'googletest'
> ...
> ***
>
> And is it the full output, it seems the build stage of thirdparty has
> been skipped?
> compiling thrift file dsn.layer2.thrift ...
> mkdir /root/apache/idl/output
> /root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out
> output dsn.layer2.thrift
> ...
>
> Best regards,
> Yingchun Lai
>
> On Wed, Aug 24, 2022 at 11:53 PM Lan Liang <li...@163.com>
> wrote:
>
>
>
> * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or
> directory *.
> The reason is run the source code packaged by windwos in linux, which
> contains the symbol of the windwos file.
>
> Download source again and compile it on the Centos7, also failed. From
> image of apache/pegasus:build-env-centos7.
>
> ***
> compiling thrift file dsn.layer2.thrift ...
> mkdir /root/apache/idl/output
> /root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out
> output dsn.layer2.thrift
> mv output/dsn.layer2_types.h
> /root/apache/src/rdsn/include/dsn/cpp/serialization_helper
> mv output/dsn.layer2_types.cpp /root/apache/src/rdsn/src/runtime
> rm -rf /root/apache/idl/output
>
> done
> Gen git_commit.h ...
> ~/apache/src ~/apache ~/apache
> fatal: Not a git repository (or any parent up to mount point /root/apache)
> Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
> PEGASUS_GIT_COMMIT=non-git-repo
> Generating /root/apache/src/include/pegasus/git_commit.h...
> [Wed Aug 24 15:47:02 UTC 2022] Building Pegasus ...
> ~/apache/src/release__builder ~/apache/src ~/apache ~/apache
> make: /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake: Command
> not found
> make: *** [cmake_check_build_system] Error 127
> ***
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Lan Liang<li...@163.com> |
> | Date | 8/24/2022 10:15 |
> | To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
> | Subject | Re: How can i run pegasus from container image |
> Just fill the link for [0].
>
> [0] https://pegasus.apache.org/docs/build/compile-by-docker/
>
>
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Lan Liang<li...@163.com> |
> | Date | 8/23/2022 23:43 |
> | To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
> | Subject | How can i run pegasus from container image |
> Hi pegasus community:
>
> I want to run pegasus from docker and i follow the doc of
> compile-by-docker [0] to try it. And the container image
> apachepegasus/build-env:2.1.0-centos7 is not exist.
>
> So i found the image of apache/pegasus:build-env-centos7 and run command
> of *** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
> /bin/bash -c "./run.sh build -c" *** to compile on the pegasus source, and
> just got the error of * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter:
> No such file or directory *.
>
> And it would be great if i can run the pegasus from some docker image.
>
> Remind me if i missed something,Thanks.
>
>
>
>
> Best Regards,
> Lan Liang
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
> For additional commands, e-mail: dev-help@pegasus.apache.org
>
Re: How can i run pegasus from container image
Posted by Lan Liang <li...@163.com>.
Have three container is running,But replica server and meta server have not any log for request.
The replica container log,
******
[root@chaos incubator-pegasus]# docker ps|grep onebox
2562f51268c2 0246c7eb5547 "/entrypoint.sh repl…" 4 minutes ago Up 4 minutes 0.0.0.0:49159->34801/tcp, :::49159->34801/tcp onebox-docker_replica1_1
21a506db99f4 zookeeper:3.4 "/docker-entrypoint.…" 4 minutes ago Up 4 minutes 2888/tcp, 3888/tcp, 0.0.0.0:49160->2181/tcp, :::49160->2181/tcp onebox-docker_zookeeper_1
1b769b90e65f 0246c7eb5547 "/entrypoint.sh meta" 4 minutes ago Up 4 minutes 0.0.0.0:34601->34601/tcp, :::34601->34601/tcp onebox-docker_meta1_1
[root@chaos incubator-pegasus]# docker logs -f --tail=20 256
239600K .......... .......... .......... .......... .......... 99% 588K 0s
239650K .......... .......... .......... .......... .......... 99% 243K 0s
239700K .......... .......... .......... .......... .......... 99% 9.90M 0s
239750K .......... .......... .......... .......... .......... 99% 8.90M 0s
239800K .......... .......... .......... .......... .......... 99% 15.8M 0s
239850K .......... .......... .......... .......... .......... 99% 7.62M 0s
239900K .......... .......... .......... .......... .......... 99% 19.2M 0s
239950K .......... .......... .......... .......... .......... 99% 1.86M 0s
240000K .......... .......... .......... .......... .......... 99% 1.68M 0s
240050K .......... .......... .......... .......... .......... 99% 11.0M 0s
240100K .......... .......... .......... .......... .......... 99% 434K 0s
240150K .......... .......... .......... .......... .......... 99% 10.8M 0s
240200K .......... .......... .......... .......... .......... 99% 1.89M 0s
240250K .......... .......... .......... .......... .......... 99% 627K 0s
240300K .......... .......... .......... .......... .......... 99% 2.38M 0s
240350K ..... 100% 105M=1m53s
2022-08-28 12:32:54 (2.09 MB/s) - 'hadoop-2.8.4.tar.gz' saved [246123562/246123562]
Decompressing HDFS...
******
BTW, container always download file for hadoop on the start,it's so weird for me.
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<li...@163.com> |
| Date | 8/27/2022 19:42 |
| To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
| Subject | Re: How can i run pegasus from container image |
Hi Yingchun,
Thanks for your confirmed.
I have run it success on the github action [0] and recompile pegasus and got the different error.
Run command of ** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build -c" **,
And Got error:
The first time,
***
...
[ 76%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service_app.cpp.o
[ 77%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_split_service.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica.cpp.o] Error 4
make[2]: *** Waiting for unfinished jobs....
[ 77%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_simple.cpp.o
[ 77%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_utils.cpp.o
[ 78%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_zookeeper.cpp.o
make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 78%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/partition_guardian.cpp.o
[ 78%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_load_balancer.cpp.o
[ 79%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state.cpp.o
[ 79%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state_restore.cpp.o
[ 79%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/duplication_info.cpp.o
[ 80%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/meta_duplication_service.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service.cpp.o] Error 4
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
make: *** [all] Error 2
***
The second time,
***
[ 76%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
[ 76%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_stub.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/cluster_balance_policy.cpp.o] Error 4
make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 76%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_test_utils.cpp.o
[ 77%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server
***
Third time
***
[ 78%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_config.cpp.o] Error 4
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all] Error 2
make: *** [all] Error 2
***
It's working on Centos7, virtual machine of 6G4U.
Then i upgrade the machine to 10G6U and compile successed !,
Finally, ready to run pegasus, and Got nothing...
1. pack server
docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh pack_server"
2. start service for pegasus
docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "DSN_ROOT/bin/pegasus_server/pegasus_server "
And return log say:
***
...
12000K .......... .......... .......... .......... .......... 99% 6.59M 0s
12050K .......... .......... .......... .......... ....... 100% 7.11M=2.9s
2022-08-27 11:25:31 (4.14 MB/s) - 'apache-zookeeper-3.7.0-bin.tar.gz' saved [12387614/12387614]
Decompressing zookeeper...
ZooKeeper JMX enabled by default
Using config: /root/apache/.zk_install/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
Zookeeper started at port 22181
starting server
cd /root/apache/onebox/meta1 && /root/apache/onebox/meta1/pegasus_server config.ini -app_list meta &>result &
***
It working for zookeeper and have no any process about pegasus, IMO, It's too hard to starte pegasus, this is so frustrating for me :(
[0] https://github.com/liangyuanpeng/pegasus/actions/runs/2920594283
Best Regards,
Lan Liang
---- Replied Message ----
| From | Yingchun Lai<ac...@gmail.com> |
| Date | 8/25/2022 23:52 |
| To | <de...@pegasus.apache.org> |
| Subject | Re: How can i run pegasus from container image |
Hi Lan Liang,
Thanks to your report! The doc is out of date, we will update it later.
What is the full command when you got the last error you mentioned
above. I have tried but not reproduced it by commands:
***
$ git clone git@github.com:apache/incubator-pegasus.git pegasus_test
$ cd pegasus_test
$ docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh build -c"
Build start time: Thu Aug 25 15:36:59 UTC 2022
Start building third-parties...
~/apache/thirdparty/build ~/apache/thirdparty
-- The C compiler identification is GNU 7.3.1
-- The CXX compiler identification is GNU 7.3.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/rh/devtoolset-7/root/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/rh/devtoolset-7/root/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Setting up third-parties...
-- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW
-- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW - Success
-- USE_JEMALLOC = OFF
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
BOOST_ROOT
Boost_NO_BOOST_CMAKE
Boost_NO_SYSTEM_PATHS
ENABLE_GCOV
ENABLE_GPERF
-- Build files have been written to: /root/apache/thirdparty/build
[ 2%] Creating directories for 'googletest'
...
***
And is it the full output, it seems the build stage of thirdparty has
been skipped?
compiling thrift file dsn.layer2.thrift ...
mkdir /root/apache/idl/output
/root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out output dsn.layer2.thrift
...
Best regards,
Yingchun Lai
On Wed, Aug 24, 2022 at 11:53 PM Lan Liang <li...@163.com> wrote:
* /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or directory *.
The reason is run the source code packaged by windwos in linux, which contains the symbol of the windwos file.
Download source again and compile it on the Centos7, also failed. From image of apache/pegasus:build-env-centos7.
***
compiling thrift file dsn.layer2.thrift ...
mkdir /root/apache/idl/output
/root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out output dsn.layer2.thrift
mv output/dsn.layer2_types.h /root/apache/src/rdsn/include/dsn/cpp/serialization_helper
mv output/dsn.layer2_types.cpp /root/apache/src/rdsn/src/runtime
rm -rf /root/apache/idl/output
done
Gen git_commit.h ...
~/apache/src ~/apache ~/apache
fatal: Not a git repository (or any parent up to mount point /root/apache)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
PEGASUS_GIT_COMMIT=non-git-repo
Generating /root/apache/src/include/pegasus/git_commit.h...
[Wed Aug 24 15:47:02 UTC 2022] Building Pegasus ...
~/apache/src/release__builder ~/apache/src ~/apache ~/apache
make: /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake: Command not found
make: *** [cmake_check_build_system] Error 127
***
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<li...@163.com> |
| Date | 8/24/2022 10:15 |
| To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
| Subject | Re: How can i run pegasus from container image |
Just fill the link for [0].
[0] https://pegasus.apache.org/docs/build/compile-by-docker/
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<li...@163.com> |
| Date | 8/23/2022 23:43 |
| To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
| Subject | How can i run pegasus from container image |
Hi pegasus community:
I want to run pegasus from docker and i follow the doc of compile-by-docker [0] to try it. And the container image apachepegasus/build-env:2.1.0-centos7 is not exist.
So i found the image of apache/pegasus:build-env-centos7 and run command of *** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build -c" *** to compile on the pegasus source, and just got the error of * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or directory *.
And it would be great if i can run the pegasus from some docker image.
Remind me if i missed something,Thanks.
Best Regards,
Lan Liang
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
Re: How can i run pegasus from container image
Posted by Lan Liang <li...@163.com>.
Hi Yingchun,
Thanks for your confirmed.
I have run it success on the github action [0] and recompile pegasus and got the different error.
Run command of ** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build -c" **,
And Got error:
The first time,
***
...
[ 76%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service_app.cpp.o
[ 77%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_split_service.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica.cpp.o] Error 4
make[2]: *** Waiting for unfinished jobs....
[ 77%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_simple.cpp.o
[ 77%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_utils.cpp.o
[ 78%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_state_service_zookeeper.cpp.o
make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 78%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/partition_guardian.cpp.o
[ 78%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_load_balancer.cpp.o
[ 79%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state.cpp.o
[ 79%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/server_state_restore.cpp.o
[ 79%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/duplication_info.cpp.o
[ 80%] Building CXX object src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/duplication/meta_duplication_service.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/meta_service.cpp.o] Error 4
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
make: *** [all] Error 2
***
The second time,
***
[ 76%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
[ 76%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_stub.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/cluster_balance_policy.cpp.o] Error 4
make[1]: *** [src/rdsn/src/meta/CMakeFiles/dsn_meta_server.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 76%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_test_utils.cpp.o
[ 77%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server
***
Third time
***
[ 78%] Building CXX object src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_restore.cpp.o
g++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See for instructions.
make[2]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/replica_config.cpp.o] Error 4
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [src/rdsn/src/replica/CMakeFiles/dsn_replica_server.dir/all] Error 2
make: *** [all] Error 2
***
It's working on Centos7, virtual machine of 6G4U.
Then i upgrade the machine to 10G6U and compile successed !,
Finally, ready to run pegasus, and Got nothing...
1. pack server
docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh pack_server"
2. start service for pegasus
docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "DSN_ROOT/bin/pegasus_server/pegasus_server "
And return log say:
***
...
12000K .......... .......... .......... .......... .......... 99% 6.59M 0s
12050K .......... .......... .......... .......... ....... 100% 7.11M=2.9s
2022-08-27 11:25:31 (4.14 MB/s) - 'apache-zookeeper-3.7.0-bin.tar.gz' saved [12387614/12387614]
Decompressing zookeeper...
ZooKeeper JMX enabled by default
Using config: /root/apache/.zk_install/apache-zookeeper-3.7.0-bin/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
Zookeeper started at port 22181
starting server
cd /root/apache/onebox/meta1 && /root/apache/onebox/meta1/pegasus_server config.ini -app_list meta &>result &
***
It working for zookeeper and have no any process about pegasus, IMO, It's too hard to starte pegasus, this is so frustrating for me :(
[0] https://github.com/liangyuanpeng/pegasus/actions/runs/2920594283
Best Regards,
Lan Liang
---- Replied Message ----
| From | Yingchun Lai<ac...@gmail.com> |
| Date | 8/25/2022 23:52 |
| To | <de...@pegasus.apache.org> |
| Subject | Re: How can i run pegasus from container image |
Hi Lan Liang,
Thanks to your report! The doc is out of date, we will update it later.
What is the full command when you got the last error you mentioned
above. I have tried but not reproduced it by commands:
***
$ git clone git@github.com:apache/incubator-pegasus.git pegasus_test
$ cd pegasus_test
$ docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh build -c"
Build start time: Thu Aug 25 15:36:59 UTC 2022
Start building third-parties...
~/apache/thirdparty/build ~/apache/thirdparty
-- The C compiler identification is GNU 7.3.1
-- The CXX compiler identification is GNU 7.3.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/rh/devtoolset-7/root/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/rh/devtoolset-7/root/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Setting up third-parties...
-- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW
-- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW - Success
-- USE_JEMALLOC = OFF
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
BOOST_ROOT
Boost_NO_BOOST_CMAKE
Boost_NO_SYSTEM_PATHS
ENABLE_GCOV
ENABLE_GPERF
-- Build files have been written to: /root/apache/thirdparty/build
[ 2%] Creating directories for 'googletest'
...
***
And is it the full output, it seems the build stage of thirdparty has
been skipped?
compiling thrift file dsn.layer2.thrift ...
mkdir /root/apache/idl/output
/root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out output dsn.layer2.thrift
...
Best regards,
Yingchun Lai
On Wed, Aug 24, 2022 at 11:53 PM Lan Liang <li...@163.com> wrote:
* /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or directory *.
The reason is run the source code packaged by windwos in linux, which contains the symbol of the windwos file.
Download source again and compile it on the Centos7, also failed. From image of apache/pegasus:build-env-centos7.
***
compiling thrift file dsn.layer2.thrift ...
mkdir /root/apache/idl/output
/root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out output dsn.layer2.thrift
mv output/dsn.layer2_types.h /root/apache/src/rdsn/include/dsn/cpp/serialization_helper
mv output/dsn.layer2_types.cpp /root/apache/src/rdsn/src/runtime
rm -rf /root/apache/idl/output
done
Gen git_commit.h ...
~/apache/src ~/apache ~/apache
fatal: Not a git repository (or any parent up to mount point /root/apache)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
PEGASUS_GIT_COMMIT=non-git-repo
Generating /root/apache/src/include/pegasus/git_commit.h...
[Wed Aug 24 15:47:02 UTC 2022] Building Pegasus ...
~/apache/src/release__builder ~/apache/src ~/apache ~/apache
make: /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake: Command not found
make: *** [cmake_check_build_system] Error 127
***
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<li...@163.com> |
| Date | 8/24/2022 10:15 |
| To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
| Subject | Re: How can i run pegasus from container image |
Just fill the link for [0].
[0] https://pegasus.apache.org/docs/build/compile-by-docker/
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<li...@163.com> |
| Date | 8/23/2022 23:43 |
| To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
| Subject | How can i run pegasus from container image |
Hi pegasus community:
I want to run pegasus from docker and i follow the doc of compile-by-docker [0] to try it. And the container image apachepegasus/build-env:2.1.0-centos7 is not exist.
So i found the image of apache/pegasus:build-env-centos7 and run command of *** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build -c" *** to compile on the pegasus source, and just got the error of * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or directory *.
And it would be great if i can run the pegasus from some docker image.
Remind me if i missed something,Thanks.
Best Regards,
Lan Liang
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
Re: How can i run pegasus from container image
Posted by Yingchun Lai <ac...@gmail.com>.
Hi Lan Liang,
Thanks to your report! The doc is out of date, we will update it later.
What is the full command when you got the last error you mentioned
above. I have tried but not reproduced it by commands:
***
$ git clone git@github.com:apache/incubator-pegasus.git pegasus_test
$ cd pegasus_test
$ docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7
/bin/bash -c "./run.sh build -c"
Build start time: Thu Aug 25 15:36:59 UTC 2022
Start building third-parties...
~/apache/thirdparty/build ~/apache/thirdparty
-- The C compiler identification is GNU 7.3.1
-- The CXX compiler identification is GNU 7.3.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/rh/devtoolset-7/root/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/rh/devtoolset-7/root/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Setting up third-parties...
-- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW
-- Performing Test COMPILER_SUPPORTS_FORMAT_OVERFLOW - Success
-- USE_JEMALLOC = OFF
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
BOOST_ROOT
Boost_NO_BOOST_CMAKE
Boost_NO_SYSTEM_PATHS
ENABLE_GCOV
ENABLE_GPERF
-- Build files have been written to: /root/apache/thirdparty/build
[ 2%] Creating directories for 'googletest'
...
***
And is it the full output, it seems the build stage of thirdparty has
been skipped?
> >>> compiling thrift file dsn.layer2.thrift ...
> mkdir /root/apache/idl/output
> /root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out output dsn.layer2.thrift
> ...
Best regards,
Yingchun Lai
On Wed, Aug 24, 2022 at 11:53 PM Lan Liang <li...@163.com> wrote:
>
>
>
> > * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or directory *.
> The reason is run the source code packaged by windwos in linux, which contains the symbol of the windwos file.
>
> Download source again and compile it on the Centos7, also failed. From image of apache/pegasus:build-env-centos7.
>
> ***
> >>> compiling thrift file dsn.layer2.thrift ...
> mkdir /root/apache/idl/output
> /root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out output dsn.layer2.thrift
> mv output/dsn.layer2_types.h /root/apache/src/rdsn/include/dsn/cpp/serialization_helper
> mv output/dsn.layer2_types.cpp /root/apache/src/rdsn/src/runtime
> rm -rf /root/apache/idl/output
>
> done
> Gen git_commit.h ...
> ~/apache/src ~/apache ~/apache
> fatal: Not a git repository (or any parent up to mount point /root/apache)
> Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
> PEGASUS_GIT_COMMIT=non-git-repo
> Generating /root/apache/src/include/pegasus/git_commit.h...
> [Wed Aug 24 15:47:02 UTC 2022] Building Pegasus ...
> ~/apache/src/release__builder ~/apache/src ~/apache ~/apache
> make: /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake: Command not found
> make: *** [cmake_check_build_system] Error 127
> ***
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Lan Liang<li...@163.com> |
> | Date | 8/24/2022 10:15 |
> | To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
> | Subject | Re: How can i run pegasus from container image |
> Just fill the link for [0].
>
> [0] https://pegasus.apache.org/docs/build/compile-by-docker/
>
>
>
>
>
>
> Best Regards,
> Lan Liang
> ---- Replied Message ----
> | From | Lan Liang<li...@163.com> |
> | Date | 8/23/2022 23:43 |
> | To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
> | Subject | How can i run pegasus from container image |
> Hi pegasus community:
>
> I want to run pegasus from docker and i follow the doc of compile-by-docker [0] to try it. And the container image apachepegasus/build-env:2.1.0-centos7 is not exist.
>
> So i found the image of apache/pegasus:build-env-centos7 and run command of *** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build -c" *** to compile on the pegasus source, and just got the error of * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or directory *.
>
> And it would be great if i can run the pegasus from some docker image.
>
> Remind me if i missed something,Thanks.
>
>
>
>
> Best Regards,
> Lan Liang
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@pegasus.apache.org
For additional commands, e-mail: dev-help@pegasus.apache.org
Re: How can i run pegasus from container image
Posted by Lan Liang <li...@163.com>.
> * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or directory *.
The reason is run the source code packaged by windwos in linux, which contains the symbol of the windwos file.
Download source again and compile it on the Centos7, also failed. From image of apache/pegasus:build-env-centos7.
***
>>> compiling thrift file dsn.layer2.thrift ...
mkdir /root/apache/idl/output
/root/apache/thirdparty/output/bin/thrift -gen cpp:moveable_types -out output dsn.layer2.thrift
mv output/dsn.layer2_types.h /root/apache/src/rdsn/include/dsn/cpp/serialization_helper
mv output/dsn.layer2_types.cpp /root/apache/src/rdsn/src/runtime
rm -rf /root/apache/idl/output
done
Gen git_commit.h ...
~/apache/src ~/apache ~/apache
fatal: Not a git repository (or any parent up to mount point /root/apache)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
PEGASUS_GIT_COMMIT=non-git-repo
Generating /root/apache/src/include/pegasus/git_commit.h...
[Wed Aug 24 15:47:02 UTC 2022] Building Pegasus ...
~/apache/src/release__builder ~/apache/src ~/apache ~/apache
make: /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake: Command not found
make: *** [cmake_check_build_system] Error 127
***
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<li...@163.com> |
| Date | 8/24/2022 10:15 |
| To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
| Subject | Re: How can i run pegasus from container image |
Just fill the link for [0].
[0] https://pegasus.apache.org/docs/build/compile-by-docker/
Best Regards,
Lan Liang
---- Replied Message ----
| From | Lan Liang<li...@163.com> |
| Date | 8/23/2022 23:43 |
| To | dev@pegasus.apache.org<de...@pegasus.apache.org> |
| Subject | How can i run pegasus from container image |
Hi pegasus community:
I want to run pegasus from docker and i follow the doc of compile-by-docker [0] to try it. And the container image apachepegasus/build-env:2.1.0-centos7 is not exist.
So i found the image of apache/pegasus:build-env-centos7 and run command of *** docker run -v $PWD:/root/apache apache/pegasus:build-env-centos7 /bin/bash -c "./run.sh build -c" *** to compile on the pegasus source, and just got the error of * /bin/bash: ./run.sh: /bin/bash^M: bad interpreter: No such file or directory *.
And it would be great if i can run the pegasus from some docker image.
Remind me if i missed something,Thanks.
Best Regards,
Lan Liang