You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pagespeed.apache.org by GitBox <gi...@apache.org> on 2021/06/08 07:02:50 UTC
[GitHub] [incubator-pagespeed-mod] acachy opened a new issue #2069: PSOL with bazel
acachy opened a new issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069
Trying to build nginx-module with current master for very long time...
Since no manuals or PSOL-related internal scripts know about bazel:
- I've built everything except envoy
- Manually fixed include errors
- Changed a bit bazel_merge.sh to skip envoy and test libs
Finally I get pagespeed_automatic.a with:
```
/pagespeed/opt/http/libproperty_cache_proto.a
/pagespeed/opt/http/libhttp.a
/pagespeed/opt/ads/libads.a
/pagespeed/opt/logging/liblogging_proto.a
/pagespeed/opt/logging/liblogging.a
/pagespeed/apache/libapache.a
/pagespeed/automatic/libstatic_rewriter_lib.a
/pagespeed/automatic/libautomatic.a
/pagespeed/kernel/js/libjs.a
/pagespeed/kernel/image/libimage.a
/pagespeed/kernel/image/libimage_optimizer_proto.a
/pagespeed/kernel/util/libutil.a
/pagespeed/kernel/html/libhtml.a
/pagespeed/kernel/sharedmem/libshared_mem_cache_shapshot_proto.a
/pagespeed/kernel/sharedmem/libsharedmem.a
/pagespeed/kernel/http/libhttp_proto.a
/pagespeed/kernel/http/libhttp.a
/pagespeed/kernel/cache/libcache.a
/pagespeed/kernel/thread/libthread.a
/pagespeed/kernel/base/libpagespeed_base.a
/pagespeed/kernel/base/libpagespeed_base_core.a
/pagespeed/system/libsystem.a
/pagespeed/controller/libcontroller_proto.a
/pagespeed/controller/libcontroller_proto_grpc.a
/pagespeed/controller/libcontroller.a
/net/instaweb/js/libdata2c_lib.a
/net/instaweb/spriter/libspriter.a
/net/instaweb/spriter/libspriter_protos.a
/net/instaweb/http/libhttp.a
/net/instaweb/htmlparse/libhtmlparse.a
/net/instaweb/rewriter/librewriter_protos.a
/net/instaweb/rewriter/librewriter.a
/third_party/css_parser/libcss_parser.a
/third_party/css_parser/libutf.a
/base/liblog_shim.a
```
Module build fails with errors about google::LogMessage like
```
/usr/bin/ld: /incubator-pagespeed-mod/bazel-incubator-pagespeed-mod/pagespeed/automatic/pagespeed_automatic.a(13.html_parse.pic.o.o): in function `net_instaweb::HtmlParse::CheckParentFromAddEvent(net_instaweb::HtmlEvent*)':
html_parse.cc:(.text+0x61a): undefined reference to `google::LogMessageFatal::LogMessageFatal(char const*, int, google::CheckOpString const&)'
```
Any suggestions? Probably it was completely wrong way to try
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] eilandert commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
eilandert commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-971650205
@Lofesa I think we need to include the logging.h from the glog package.
@acachy:
My approach is not to include /root/.bazel/* but to create a psol.tar.gz package with the libautomatic.a and the needed headers so people can untar it in nginx-1.21.4/modules/ngx_pagespeed/psol without the need to build it with bazel. See the rsync commands in https://github.com/eilandert/psol-bazel/blob/master/docker/build.sh
My approach is:
1) create a functional psol (done, I suppose)
2) create a psol.tar.gz with lib+headers (done, I suppose)
3) clone the ngx pagespeed module and make some changes.
When 3 works we have a proof of concept. Then we continue to:
4) adjust master/pagespeed/automatic/BUILD and stuff so everything will be build by "bazel build -c fastbuild //pagespeed/automatic:automatic" or something.
5) find out the bazel way to build libicu. When I build ICU with bazel, the build fails because gURL wants to build a target that doesn;t exist.
6) optimize the build. not everything has to be in the lib I think. the lib is around 111MB now, while in the old way it was only half of that.
7) do a pull request
I think it is not that difficult, but since I am no dev I have to find everything out for myself with common sense, and currently I haven't got the time. I'm not home for a couple of weeks, i've been sent to a hospital far away due to covid to help out (i am actually an healthcare provider)
So, it will take time ;-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] eilandert commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
eilandert commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-966219159
> > Or we need a @glog//:glog dependency
> > bazel build -c fastbuild @glog//:glog seems to do something ;-)
>
> Looks like your way is the good one
>
> I've build a bunch of extra .a-libs with
>
> ```
> bazel build -c fastbuild @//...
> ```
>
> Now I have different error (but that build was not completed)
This is what I do now, I have some troubles with Google URL and ABSL. Can't seem to find the correct dependencies for now, but it's a step further ;-)
```
bazel build -c fastbuild \
@glog//:glog \
@com_google_absl//absl/algorithm:algorithm @com_google_absl//absl/base:raw_logging_internal \
@com_google_absl//absl/base @com_google_absl//absl/strings @com_google_absl//absl/hash
@com_google_absl//absl/memory @com_google_absl//absl/status @com_google_absl//absl/synchronization \
@com_google_absl//absl/utility \
@com_github_gflags_gflags//:gflags @com_googlesource_googleurl//base @com_googlesource_googleurl//url \
//pagespeed/kernel/... //pagespeed/automatic/... //pagespeed/system/... //pagespeed/controller/... \
//pagespeed/opt/... //base/... //net/instaweb/... //third_party/... \
mod_pagespeed
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] eilandert commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
eilandert commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-962703264
Or we need a @glog//:glog dependency
bazel build -c fastbuild @glog//:glog seems to do something ;-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] eilandert commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
eilandert commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-920379346
@acachy how did you manage to build?
I get a lot of errors like this in net_instaweb
pagespeed/kernel/base/string_util.cc: In function 'GoogleString net_instaweb::internal::StrCatNineOrMore(const StringPiece*, ...)':
pagespeed/kernel/base/string_util.cc:110:3: error: 'va_start' was not declared in this scope
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] eilandert commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
eilandert commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-967521102
ok, I do this: https://github.com/eilandert/psol-bazel/blob/master/docker/build.sh
```
bazel build -c fastbuild \
@glog//:glog \
@com_googlesource_googleurl//base/third_party/icu \
@com_google_absl//... \
@com_github_gflags_gflags//... \
@com_googlesource_googleurl//... \
@google_sparsehash//... \
//pagespeed/kernel/... //pagespeed/automatic/... //pagespeed/system/... //pagespeed/controller/... \
//pagespeed/opt/... //base/... //net/instaweb/... //third_party/... \
mod_pagespeed
```
But whatever I do, I get the following error:
```
----------------------------------------
checking for psol
/usr/bin/ld: src/incubator-pagespeed-ngx//psol/lib/Release/linux/x64/pagespeed_automatic.a(94.url_idna_icu.pic.o.o): in function `url::(anonymous
namespace)::UIDNAWrapper::UIDNAWrapper()':
(.text+0x41): undefined reference to `uidna_openUTS46_66'
/usr/bin/ld: (.text+0x8c): undefined reference to `u_errorName_66'
/usr/bin/ld: src/incubator-pagespeed-ngx//psol/lib/Release/linux/x64/pagespeed_automatic.a(94.url_idna_icu.pic.o.o): in function `url::IDNToASCII(
unsigned short const*, int, url::CanonOutputT<unsigned short>*)':
(.text+0x232): undefined reference to `uidna_nameToASCII_66'
collect2: error: ld returned 1 exit status
----------
```
Getting close though, I suppose this is the last fence to jump.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] eilandert commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
eilandert commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-968358107
Having the pagespeed_automatic.a is not enough, we need the headerfiles to be copied, otherwise the NGINX module is not aware of the functions and how to use them. I think I took care of that in https://github.com/eilandert/psol-bazel/blob/master/docker/build.sh (see the rsync section)
Now we have to mess around in the nginx module itself.
For example:
log_message_handler.cc:31:#include "base/debug/debugger.h"
That is a hard include to (I believe) chromium. So I am not sure to include chromium or to delete or change that line.
And this is only the first item I come across, I think there are a lot more.
I am not a dev, but I am willing to take a dive into it, but I am limited in time at the moment. So please bear with me ;-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] oschaaf commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
oschaaf commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-972764335
Hey @eilandert & @acachy & @Lofesa awesome job; I think that with some persistence this will be resolved, as it should be viable. Earlier I had hoped that there would be a variant of nginx out there which would support the bazel build system.
That would have made life much easier; but alas there are no (up-to-date) efforts in that direction that I know of.
It looks like supporting a "bazelified" build of nginx ourselves would require an ongoing and significant effort; so this project will very probably be better off with what is being attempted here: package psol like we used to do earlier.
I'm hoping I can find some time around the end of this year to contribute to this effort.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] eilandert edited a comment on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
eilandert edited a comment on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-966219159
> > Or we need a @glog//:glog dependency
> > bazel build -c fastbuild @glog//:glog seems to do something ;-)
>
> Looks like your way is the good one
>
> I've build a bunch of extra .a-libs with
>
> ```
> bazel build -c fastbuild @//...
> ```
>
> Now I have different error (but that build was not completed)
This is what I do now, I have some troubles with Google URL and ABSL. Can't seem to find the correct dependencies for now, but it's a step further ;-)
```
bazel build -c fastbuild \
@glog//:glog \
@com_google_absl//absl/algorithm:algorithm @com_google_absl//absl/base:raw_logging_internal \
@com_google_absl//absl/base @com_google_absl//absl/strings @com_google_absl//absl/hash \
@com_google_absl//absl/memory @com_google_absl//absl/status @com_google_absl//absl/synchronization \
@com_google_absl//absl/utility \
@com_github_gflags_gflags//:gflags @com_googlesource_googleurl//base @com_googlesource_googleurl//url \
//pagespeed/kernel/... //pagespeed/automatic/... //pagespeed/system/... //pagespeed/controller/... \
//pagespeed/opt/... //base/... //net/instaweb/... //third_party/... \
mod_pagespeed
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] eilandert commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
eilandert commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-968079554
Hi @Lofesa
I am not a dev either ;-) But I like to puzzle.
I have it working now with latest master, well at least it passes ./configure from nginx. So PSOL itself should be good to go.
I need headers from chromium (I think) to compile the ngx_pagespeed module, but there is no chromium anymore in /third_party. It will take some time but we'll get there sooner or later when I find the time ;-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] acachy edited a comment on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
acachy edited a comment on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-971253138
@eilandert
0. I use your bazel-build command, but extra includes in pagespeed-ngx/config looks like still necessary
1. Bazel-build with "-c opt" creates more libraries
I didn't go deep. Got error on symbols renaming about duplicates. But probably it might be because of next point.
2. My bazel-merge.sh had to be fixed against eating pagespeed_automatic.a or it became constantly growing with each build by including itself
3. I had your error about _uidna_openUTS46_66_ but other suspicions ate time
"find+grep" shows that it's related to libicu-lib
There should be bazel-way to include... And maybe this alternative way would bring problems.
But don't care at the moment
Just add system's package libs to merge_libraries.sh call in bazel-merge.sh
/usr/lib/x86_64-linux-gnu/libicuuc.a
/usr/lib/x86_64-linux-gnu/libicudata.a
```
#!/bin/bash
ADIR=$(bazel info bazel-bin)
ALIST=$(find $ADIR | grep \\.a$ | grep -v main | grep -v copy | grep -v envoy | grep -v /pagespeed_automatic\\.a | xargs echo)
for item in ${ALIST[*]}
do
echo $item
done
#find $ADIR | grep \\.a$ | grep -v main | grep -v copy
set -e
echo "merging libs"
bash ./merge_libraries.sh ~/pagespeed_automatic.a.dirty $ALIST /usr/lib/x86_64-linux-gnu/libicuuc.a /usr/lib/x86_64-linux-gnu/libicudata.a > merge.log
echo "renaming symbols"
# XXX(oschaaf): objcopy 2.31 doesn't like clang-7's output unless we pass in -fno-addrsig
# https://github.com/travitch/whole-program-llvm/issues/75
# not passing this in will make the scripts that rename symbols fail
./rename_c_symbols.sh ~/pagespeed_automatic.a.dirty ~/pagespeed_automatic.a > symbol-rename.log
rm ~/pagespeed_automatic.a.dirty
cp ~/pagespeed_automatic.a ./../../bazel-out/k8-fastbuild/bin/pagespeed/automatic/
mv ~/pagespeed_automatic.a ./
echo "done"
```
4. Then we get "opendl" error - it is fixed in "pagespeed-ngx/config"
```
pagespeed_libs="-lstdc++"
->
pagespeed_libs="-lstdc++ -ldl"
```
5. Now configure is successful
6. Haha. Make-stage shows a lot of lost files
They may be fixed with more extra includes (maybe good way is in some other place for includes)
```
pagespeed_include="\
$mod_pagespeed_dir \
$mod_pagespeed_dir/pagespeed/kernel \
$mod_pagespeed_dir/bazel-out/../../../external/com_google_absl \
$ngx_addon_dir/testing-dependencies/mod_pagespeed/third_party/chromium/src \
$mod_pagespeed_dir/third_party/chromium/src \
$mod_pagespeed_dir/third_party/google-sparsehash/src/src \
$mod_pagespeed_dir/third_party/google-sparsehash/gen/arch/$os_name/$arch_name/include \
$mod_pagespeed_dir/third_party/grpc/src/include \
$mod_pagespeed_dir/third_party/protobuf/src/src \
$mod_pagespeed_dir/third_party/re2/src \
$mod_pagespeed_dir/out/$buildtype/obj/gen \
$mod_pagespeed_dir/out/$buildtype/obj/gen/protoc_out/instaweb \
$mod_pagespeed_dir/third_party/apr/src/include \
$mod_pagespeed_dir/third_party/aprutil/src/include \
$mod_pagespeed_dir/third_party/apr/gen/arch/$os_name/$arch_name/include \
$mod_pagespeed_dir/third_party/aprutil/gen/arch/$os_name/$arch_name/include \
$mod_pagespeed_dir/url \
$mod_pagespeed_dir/bazel-bin/external/glog/_virtual_includes/default_glog_headers \
$mod_pagespeed_dir/bazel-out/k8-fastbuild/bin/external/com_github_gflags_gflags/_virtual_includes/gflags \
$mod_pagespeed_dir/bazel-out/k8-fastbuild/bin/external/google_sparsehash/_virtual_includes/google_sparsehash \
$mod_pagespeed_dir/bazel-incubator-pagespeed-mod \
$mod_pagespeed_dir/bazel-incubator-pagespeed-mod/external/com_google_protobuf/src \
$mod_pagespeed_dir/bazel-incubator-pagespeed-mod/external/re2 \
$mod_pagespeed_dir/bazel-bin \
$mod_pagespeed_dir/bazel-incubator-pagespeed-mod/external/com_github_grpc_grpc/include \
$mod_pagespeed_dir/bazel-incubator-pagespeed-mod/external/apr/include \
$mod_pagespeed_dir/bazel-incubator-pagespeed-mod/external/aprutil/include"
```
7. Then new funny things appear like
```
/ngx/incubator-pagespeed-ngx/src/ngx_pagespeed.cc: In function ‘char* net_instaweb::{anonymous}::ps_init_dir(const StringPiece&, const StringPiece&, ngx_conf_t*)’:
/ngx/incubator-pagespeed-ngx/src/ngx_pagespeed.cc:780:33: error: ‘StrCat’ is not a member of ‘net_instaweb’
780 | cf->pool, net_instaweb::StrCat(directive, " ", path,
```
```
/ngx/incubator-pagespeed-ngx/src/log_message_handler.cc:54:19: error: ‘LOG_ERROR_REPORT’ is not a member of ‘logging’
54 | case logging::LOG_ERROR_REPORT:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] Lofesa commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
Lofesa commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-971452547
But now there is base/loggin.h
I found SetLogMessageHandler in pagespeed/envoy/log_message_handler.cc, but in a comment in an epmty function
https://github.com/apache/incubator-pagespeed-mod/blob/d8070cf1cd0dbfb70a05e28b739707c66b1cf635/pagespeed/envoy/log_message_handler.cc#L66
I think envoy is the sustitute for chromium and must be included
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] acachy commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
acachy commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-971253138
@eilandert
0. I use your bazel-build command, but extra includes in pagespeed-ngx/config looks like still necessary
1. Bazel-build with "-c opt" creates more libraries
I didn't go deep. Got error on symbols renaming about duplicates. But probably it might be because of next point.
2. My bazel-merge.sh had to be fixed against eating pagespeed_automatic.a or it became constantly growing with each build by including itself
3. I had your error about _uidna_openUTS46_66_ but other suspicions ate time
"find+grep" shows that it's related to libicu-lib
There should be bazel-way to include... And maybe this alternative way would bring problems.
But don't care at the moment
Just add system's package libs to merge_libraries.sh call in bazel-merge.sh
/usr/lib/x86_64-linux-gnu/libicuuc.a
/usr/lib/x86_64-linux-gnu/libicudata.a
```
#!/bin/bash
ADIR=$(bazel info bazel-bin)
ALIST=$(find $ADIR | grep \\.a$ | grep -v main | grep -v copy | grep -v envoy | grep -v /pagespeed_automatic\\.a | xargs echo)
for item in ${ALIST[*]}
do
echo $item
done
#find $ADIR | grep \\.a$ | grep -v main | grep -v copy
set -e
echo "merging libs"
bash ./merge_libraries.sh ~/pagespeed_automatic.a.dirty $ALIST /usr/lib/x86_64-linux-gnu/libicuuc.a /usr/lib/x86_64-linux-gnu/libicudata.a > merge.log
echo "renaming symbols"
# XXX(oschaaf): objcopy 2.31 doesn't like clang-7's output unless we pass in -fno-addrsig
# https://github.com/travitch/whole-program-llvm/issues/75
# not passing this in will make the scripts that rename symbols fail
./rename_c_symbols.sh ~/pagespeed_automatic.a.dirty ~/pagespeed_automatic.a > symbol-rename.log
rm ~/pagespeed_automatic.a.dirty
cp ~/pagespeed_automatic.a ./../../bazel-out/k8-fastbuild/bin/pagespeed/automatic/
mv ~/pagespeed_automatic.a ./
echo "done"
```
4. Then we get "opendl" error - it is fixed in "pagespeed-ngx/config"
```
pagespeed_libs="-lstdc++"
->
pagespeed_libs="-lstdc++ -ldl"
```
5. Now configure is succussful
6. Haha. Make-stage shows a lot of lost files
They may be fixed with more extra includes (maybe good way is in some other place for includes)
```
pagespeed_include="\
$mod_pagespeed_dir \
$mod_pagespeed_dir/pagespeed/kernel \
$mod_pagespeed_dir/bazel-out/../../../external/com_google_absl \
$ngx_addon_dir/testing-dependencies/mod_pagespeed/third_party/chromium/src \
$mod_pagespeed_dir/third_party/chromium/src \
$mod_pagespeed_dir/third_party/google-sparsehash/src/src \
$mod_pagespeed_dir/third_party/google-sparsehash/gen/arch/$os_name/$arch_name/include \
$mod_pagespeed_dir/third_party/grpc/src/include \
$mod_pagespeed_dir/third_party/protobuf/src/src \
$mod_pagespeed_dir/third_party/re2/src \
$mod_pagespeed_dir/out/$buildtype/obj/gen \
$mod_pagespeed_dir/out/$buildtype/obj/gen/protoc_out/instaweb \
$mod_pagespeed_dir/third_party/apr/src/include \
$mod_pagespeed_dir/third_party/aprutil/src/include \
$mod_pagespeed_dir/third_party/apr/gen/arch/$os_name/$arch_name/include \
$mod_pagespeed_dir/third_party/aprutil/gen/arch/$os_name/$arch_name/include \
$mod_pagespeed_dir/url \
$mod_pagespeed_dir/bazel-bin/external/glog/_virtual_includes/default_glog_headers \
$mod_pagespeed_dir/bazel-out/k8-fastbuild/bin/external/com_github_gflags_gflags/_virtual_includes/gflags \
$mod_pagespeed_dir/bazel-out/k8-fastbuild/bin/external/google_sparsehash/_virtual_includes/google_sparsehash \
$mod_pagespeed_dir/bazel-incubator-pagespeed-mod \
$mod_pagespeed_dir/bazel-incubator-pagespeed-mod/external/com_google_protobuf/src \
$mod_pagespeed_dir/bazel-incubator-pagespeed-mod/external/re2 \
$mod_pagespeed_dir/bazel-bin \
$mod_pagespeed_dir/bazel-incubator-pagespeed-mod/external/com_github_grpc_grpc/include \
$mod_pagespeed_dir/bazel-incubator-pagespeed-mod/external/apr/include \
$mod_pagespeed_dir/bazel-incubator-pagespeed-mod/external/aprutil/include"
```
7. Then new funny things appear like
```
/ngx/incubator-pagespeed-ngx/src/ngx_pagespeed.cc: In function ‘char* net_instaweb::{anonymous}::ps_init_dir(const StringPiece&, const StringPiece&, ngx_conf_t*)’:
/ngx/incubator-pagespeed-ngx/src/ngx_pagespeed.cc:780:33: error: ‘StrCat’ is not a member of ‘net_instaweb’
780 | cf->pool, net_instaweb::StrCat(directive, " ", path,
```
```
/ngx/incubator-pagespeed-ngx/src/log_message_handler.cc:54:19: error: ‘LOG_ERROR_REPORT’ is not a member of ‘logging’
54 | case logging::LOG_ERROR_REPORT:
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] Lofesa commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
Lofesa commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-968075205
Hi @eilandert
I´m not a developer, so can´t help here.
But I have a clue.... Greping in and old copy of the repo I have found that icu_whatever_things are in /third_party/icu.
This is not in the master, but you can find this folder in version 36 the last before bazel.
the folder is a pointer to https://github.com/apache/incubator-pagespeed-icu. Maybe this must be included.....
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] eilandert edited a comment on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
eilandert edited a comment on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-920379346
@acachy how did you manage to build?
I get a lot of errors like this:
pagespeed/kernel/base/string_util.cc: In function 'GoogleString net_instaweb::internal::StrCatNineOrMore(const StringPiece*, ...)':
pagespeed/kernel/base/string_util.cc:110:3: error: 'va_start' was not declared in this scope
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] eilandert commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
eilandert commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-962562306
Thanks! There are a few things I can use / didn't know
When I have some time I'll give it another try. Same as you, trying to fix what I encounter. Never used bazel before ;-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] eilandert edited a comment on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
eilandert edited a comment on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-962696232
@acachy
Thanks for your pointers, I used them in a slightly different approach but I am a lot further ahead than I was, but I am stuck on the same error now.
> /usr/bin/ld: doctype.cc:(.text+0x430): undefined reference to `google::LogMessageFatal::~LogMessageFatal()'
> collect2: error: ld returned 1 exit status
> ```
>
> Probably we should use other include-paths (for most files there are different copies in different dirs) or better ways to fix include-errors I have no significant knowledge here, just tried to fix everything that I've seen
After some digging I discovered that glog is build in objects but there is no .a library, so glog is not merged into the pagespeed library.
@oschaaf , perhaps we need a bazel/glog.bzl ?
Please note, I don't know what i am doing either ;-) Bazel is not my thing yet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] eilandert commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
eilandert commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-962696232
@acachy
Thanks for your pointers, I used them in a slightly different approach but I am a lot further ahead than I was, but I am stuck on the same error now.
> /usr/bin/ld: doctype.cc:(.text+0x430): undefined reference to `google::LogMessageFatal::~LogMessageFatal()'
> collect2: error: ld returned 1 exit status
> ```
>
> Probably we should use other include-paths (for most files there are different copies in different dirs) or better ways to fix include-errors I have no significant knowledge here, just tried to fix everything that I've seen
After some digging I discovered that glog is build in objects but there is no .a library, so glog is not merged into the pagespeed library.
@oschaaf , perhaps we need a bazel/glog.bzl ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] eilandert edited a comment on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
eilandert edited a comment on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-920379346
@acachy how did you manage to build?
I get a lot of errors like this in net_instaweb
pagespeed/kernel/base/string_util.cc: In function 'GoogleString net_instaweb::internal::StrCatNineOrMore(const StringPiece*, ...)':
pagespeed/kernel/base/string_util.cc:110:3: error: 'va_start' was not declared in this scope
edit: nevermind. dependency icu-dev was not installed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] acachy commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
acachy commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-966149681
> Or we need a @glog//:glog dependency
>
> bazel build -c fastbuild @glog//:glog seems to do something ;-)
Looks like your way is the good one
I've build a bunch of extra .a-libs with
```
bazel build -c fastbuild @//...
````
Now I have different error (but that build was not completed)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] oschaaf commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
oschaaf commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-856518249
As far as I know nobody has tried to build ngx_pagespeed yet. So we're in uncharted territory.
It should be possible though.
To obtain `pagespeed_automatic.a` this should suffice:
```
bazel build -c opt //pagespeed/automatic:automatic
```
If that yields the undefined reference you mention, then I think we're missing a dependency in a `BUILD` file on `glog`. Maybe adding it in https://github.com/apache/incubator-pagespeed-mod/blob/master/pagespeed/automatic/BUILD#L20 helps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] Lofesa commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
Lofesa commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-968332337
Hi
If I´m not wrong, chromium was changed by envoy in the bazel brach.
And I think nginx module is a wrapper around the psol lib to work with nginx, so if you have the pagespeed_automatic.a you can compile the nginx_pagespeed module.
Maybe @oschaaf can clarify if chromium headers are needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] Lofesa commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
Lofesa commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-968726076
I only intend to help as much as I can reading the code.
The files in the include, in fact all files under "base/debug", don´t exist anymore. I think is better to delete it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] acachy commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
acachy commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-962558016
> @acachy how did you manage to build?
>
> I get a lot of errors like this: pagespeed/kernel/base/string_util.cc: In function 'GoogleString net_instaweb::internal::StrCatNineOrMore(const StringPiece*, ...)': pagespeed/kernel/base/string_util.cc:110:3: error: 'va_start' was not declared in this scope
@eilandert, looks like you've already moved far enough in #2084
My previous attempts were forgotten because of fail at the end...
So I tried again on latest commit and here is a part of my deploy script for Debian, if you wanna try to repeate. Sorry for a lot of trash ways inside
1. Prepare dir with sources
```
echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" >> /etc/apt/sources.list
wget --no-check-certificate -qO - https://bazel.build/bazel-release.pub.gpg | apt-key add - 2>&1
apt-get install -y bazel-4.1.0
apt-get install -y ninja-build
mkdir -p /usr/share/man/man1
apt-get install -y openjdk-11-jre-headless
mkdir /ngx
cd /ngx
wget https://github.com/nginx/nginx/archive/release-1.19.10.tar.gz
tar xzf *gz
rm *gz
git clone https://github.com/apache/incubator-pagespeed-ngx.git
cd incubator-pagespeed-ngx
git reset --hard 9e70f6d
git config --global url.https://github.com/apache/.insteadOf git://git.apache.org/
git submodule update --init --recursive --jobs=16 --force
cd ..
git clone --recursive https://github.com/apache/incubator-pagespeed-mod.git
cd incubator-pagespeed-mod
git reset --hard 6199d78
```
2. Bazel-build with few hacks
```
sed -i '1s/^/#include <cstdarg>\n/' pagespeed/kernel/base/string_util.cc
bazel build -c fastbuild //pagespeed/kernel/... //pagespeed/automatic/... //pagespeed/system/... //pagespeed/controller/... //pagespeed/opt/... //base/... //net/instaweb/... //third_party/... mod_pagespeed
sed -i -r 's/sys_siglist\[signum\]/strsignal(signum)/g' ./bazel-bin/../../../../../external/apr/threadproc/unix/signals.c
bazel build -c fastbuild //pagespeed/kernel/... //pagespeed/automatic/... //pagespeed/system/... //pagespeed/controller/... //pagespeed/opt/... //base/... //net/instaweb/... //third_party/... mod_pagespeed
```
3. Update pagespeed/automatic/bazel_merge.sh
- Skip Envoy
- Change final destination path for lib
```
#!/bin/bash
ADIR=$(bazel info bazel-bin)
ALIST=$(find $ADIR | grep \\.a$ | grep -v main | grep -v copy | grep -v envoy | xargs echo)
for item in ${ALIST[*]}
do
echo $item
done
find $ADIR | grep \\.a$ | grep -v main | grep -v copy
set -e
echo "merging libs"
./merge_libraries.sh ~/pagespeed_automatic.a.dirty $ALIST > merge.log
echo "renaming symbols"
# XXX(oschaaf): objcopy 2.31 doesn't like clang-7's output unless we pass in -fno-addrsig
# https://github.com/travitch/whole-program-llvm/issues/75
# not passing this in will make the scripts that rename symbols fail
./rename_c_symbols.sh ~/pagespeed_automatic.a.dirty ~/pagespeed_automatic.a > symbol-rename.log
rm ~/pagespeed_automatic.a.dirty
cp ~/pagespeed_automatic.a ./../../bazel-out/k8-fastbuild/bin/pagespeed/automatic/
mv ~/pagespeed_automatic.a ./
echo "done"
```
4. Create pagespeed_automatic.a
```
cd pagespeed/automatic
bash bazel_merge.sh
```
5. Try to build nginx
```
cd /ngx/nginx-release-1.19.10
MOD_PAGESPEED_DIR="/ngx/incubator-pagespeed-mod" ./auto/configure --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --with-http_addition_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_secure_link_module --with-http_sub_module --w
ith-mail=dynamic --with-mail_ssl_module --with-stream=dynamic --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_perl_module=dynamic --add-module=/ngx/incubator-pagespeed-ngx
```
6. There are include-errors that can be seen in details at the end of ./objs/autoconf.err
I fix it step-by-step by searching paths with missing files and adding extra include-dirs to /ngx/incubator-pagespeed-ngx/config
After few steps I have include rule like this:
```
pagespeed_include="\
$mod_pagespeed_dir \
$mod_pagespeed_dir/third_party/chromium/src \
$mod_pagespeed_dir/third_party/google-sparsehash/src/src \
$mod_pagespeed_dir/third_party/google-sparsehash/gen/arch/$os_name/$arch_name/include \
$mod_pagespeed_dir/third_party/grpc/src/include \
$mod_pagespeed_dir/third_party/protobuf/src/src \
$mod_pagespeed_dir/third_party/re2/src \
$mod_pagespeed_dir/out/$buildtype/obj/gen \
$mod_pagespeed_dir/out/$buildtype/obj/gen/protoc_out/instaweb \
$mod_pagespeed_dir/third_party/apr/src/include \
$mod_pagespeed_dir/third_party/aprutil/src/include \
$mod_pagespeed_dir/third_party/apr/gen/arch/$os_name/$arch_name/include \
$mod_pagespeed_dir/third_party/aprutil/gen/arch/$os_name/$arch_name/include \
$mod_pagespeed_dir/bazel-out/../../../external/com_google_absl \
$mod_pagespeed_dir/bazel-out/k8-fastbuild/bin/external/com_github_gflags_gflags/_virtual_includes/gflags \
$mod_pagespeed_dir/bazel-out/k8-fastbuild/bin/external/glog/_virtual_includes/default_glog_headers \
$mod_pagespeed_dir/bazel-out/k8-fastbuild/bin/external/google_sparsehash/_virtual_includes/google_sparsehash \
$mod_pagespeed_dir/url"
```
And my last error on attempt to use auto/configure:
```
...
/usr/bin/ld: doctype.cc:(.text+0x430): undefined reference to `google::LogMessageFatal::~LogMessageFatal()'
collect2: error: ld returned 1 exit status
```
Probably we should use other include-paths (for most files there are different copies and different dirs) or better ways to fix include-errors
I have no significant knowledge here, just tried to fix everything that I've seen
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] acachy commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
acachy commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-858151770
> bazel build -c opt //pagespeed/automatic:automatic
As far as I understand - it just generates libautomatic.a (that I already listed as a part of my merged pagespeed_automatic.a). And implies to use broken bazel_merge.sh after
To be sure, I've tried to use just libautomatic.a as pagespeed_automatic.a and _ngx_ show errors like
```
/usr/bin/ld: /tmp/ccBVH139.o: warning: relocation against `_ZTVN12net_instaweb18NullMessageHandlerE' in read-only section `.text.startup'
/usr/bin/ld: /tmp/ccBVH139.o: in function `main':
autotest.cc:(.text.startup+0x37): undefined reference to `vtable for net_instaweb::StringWriter'
/usr/bin/ld: autotest.cc:(.text.startup+0x83): undefined reference to `net_instaweb::MessageHandler::MessageHandler()'
```
I've checked a bit how pagespeed_automatic is made. In "before-bazel" way it uses merge_libraries.sh to combine these libraries:
```
/build/libwebp_dsp.a
/build/libwebp_enc.a
/build/libwebp_mux.a
/build/libwebp_utils.a
/net/instaweb/libautomatic_util.a
/net/instaweb/libinstaweb_add_instrumentation_data2c.a
/net/instaweb/libinstaweb_add_instrumentation_opt_data2c.a
/net/instaweb/libinstaweb_admin_site_css_data2c.a
/net/instaweb/libinstaweb_apr.a
/net/instaweb/libinstaweb_automatic.a
/net/instaweb/libinstaweb_caches_css_data2c.a
/net/instaweb/libinstaweb_caches_js_data2c.a
/net/instaweb/libinstaweb_caches_js_opt_data2c.a
/net/instaweb/libinstaweb_client_domain_rewriter_data2c.a
/net/instaweb/libinstaweb_client_domain_rewriter_opt_data2c.a
/net/instaweb/libinstaweb_console_css_data2c.a
/net/instaweb/libinstaweb_console_js_data2c.a
/net/instaweb/libinstaweb_console_js_opt_data2c.a
/net/instaweb/libinstaweb_critical_css_beacon_data2c.a
/net/instaweb/libinstaweb_critical_css_beacon_opt_data2c.a
/net/instaweb/libinstaweb_critical_css_loader_data2c.a
/net/instaweb/libinstaweb_critical_css_loader_opt_data2c.a
/net/instaweb/libinstaweb_critical_images_beacon_data2c.a
/net/instaweb/libinstaweb_critical_images_beacon_opt_data2c.a
/net/instaweb/libinstaweb_critical_images_pb.a
/net/instaweb/libinstaweb_critical_keys_pb.a
/net/instaweb/libinstaweb_dedup_inlined_images_data2c.a
/net/instaweb/libinstaweb_dedup_inlined_images_opt_data2c.a
/net/instaweb/libinstaweb_defer_iframe_data2c.a
/net/instaweb/libinstaweb_defer_iframe_opt_data2c.a
/net/instaweb/libinstaweb_delay_images_data2c.a
/net/instaweb/libinstaweb_delay_images_inline_data2c.a
/net/instaweb/libinstaweb_delay_images_inline_opt_data2c.a
/net/instaweb/libinstaweb_delay_images_opt_data2c.a
/net/instaweb/libinstaweb_dependencies_pb.a
/net/instaweb/libinstaweb_deterministic_data2c.a
/net/instaweb/libinstaweb_deterministic_opt_data2c.a
/net/instaweb/libinstaweb_extended_instrumentation_data2c.a
/net/instaweb/libinstaweb_extended_instrumentation_opt_data2c.a
/net/instaweb/libinstaweb_flush_early_pb.a
/net/instaweb/libinstaweb_graphs_css_data2c.a
/net/instaweb/libinstaweb_graphs_js_data2c.a
/net/instaweb/libinstaweb_graphs_js_opt_data2c.a
/net/instaweb/libinstaweb_input_info_pb.a
/net/instaweb/libinstaweb_js_defer_data2c.a
/net/instaweb/libinstaweb_js_defer_opt_data2c.a
/net/instaweb/libinstaweb_lazyload_images_data2c.a
/net/instaweb/libinstaweb_lazyload_images_opt_data2c.a
/net/instaweb/libinstaweb_local_storage_cache_data2c.a
/net/instaweb/libinstaweb_local_storage_cache_opt_data2c.a
/net/instaweb/libinstaweb_messages_js_data2c.a
/net/instaweb/libinstaweb_messages_js_opt_data2c.a
/net/instaweb/libinstaweb_rendered_image_pb.a
/net/instaweb/libinstaweb_responsive_js_data2c.a
/net/instaweb/libinstaweb_responsive_js_opt_data2c.a
/net/instaweb/libinstaweb_rewriter.a
/net/instaweb/libinstaweb_rewriter_base.a
/net/instaweb/libinstaweb_rewriter_css.a
/net/instaweb/libinstaweb_rewriter_html.a
/net/instaweb/libinstaweb_rewriter_html_gperf.a
/net/instaweb/libinstaweb_rewriter_csp_gperf.a
/net/instaweb/libinstaweb_rewriter_image.a
/net/instaweb/libinstaweb_rewriter_javascript.a
/net/instaweb/libinstaweb_rewriter_pb.a
/net/instaweb/libinstaweb_spriter.a
/net/instaweb/libinstaweb_spriter_pb.a
/net/instaweb/libinstaweb_static_asset_config_pb.a
/net/instaweb/libinstaweb_statistics_css_data2c.a
/net/instaweb/libinstaweb_statistics_js_data2c.a
/net/instaweb/libinstaweb_statistics_js_opt_data2c.a
/net/instaweb/libinstaweb_system.a
/net/instaweb/libinstaweb_util.a
/net/instaweb/libprocess_context.a
/net/instaweb/libtest_infrastructure.a
/pagespeed/libjs_tokenizer.a
/pagespeed/libjsminify.a
/pagespeed/libmem_lock.a
/pagespeed/libpagespeed_ads_util.a
/pagespeed/libpagespeed_base.a
/pagespeed/libpagespeed_base_core.a
/pagespeed/libpagespeed_base_test_infrastructure.a
/pagespeed/libpagespeed_cache.a
/pagespeed/libpagespeed_controller.a
/pagespeed/libpagespeed_controller_proto.a
/pagespeed/libpagespeed_html.a
/pagespeed/libpagespeed_html_gperf.a
/pagespeed/libpagespeed_http.a
/pagespeed/libpagespeed_http_core.a
/pagespeed/libpagespeed_http_gperf.a
/pagespeed/libpagespeed_http_pb.a
/pagespeed/libpagespeed_image_processing.a
/pagespeed/libpagespeed_image_types_pb.a
/pagespeed/libpagespeed_javascript_gperf.a
/pagespeed/libpagespeed_logging.a
/pagespeed/libpagespeed_logging_enums_pb.a
/pagespeed/libpagespeed_logging_pb.a
/pagespeed/libpagespeed_opt_http.a
/pagespeed/libpagespeed_property_cache_pb.a
/pagespeed/libpagespeed_sharedmem.a
/pagespeed/libpagespeed_sharedmem_pb.a
/pagespeed/libpagespeed_thread.a
/pagespeed/libpthread_system.a
/pagespeed/libutil.a
/third_party/apr/libapr.a
/third_party/aprutil/libaprutil.a
/third_party/base64/libbase64.a
/third_party/chromium/src/base/third_party/dynamic_annotations/libdynamic_annotations.a
/third_party/css_parser/libcss_parser.a
/third_party/css_parser/libcss_parser_gperf.a
/third_party/css_parser/libutf.a
/third_party/domain_registry_provider/src/domain_registry/libassert_lib.a
/third_party/domain_registry_provider/src/domain_registry/libdomain_registry_lib.a
/third_party/domain_registry_provider/src/domain_registry/libinit_registry_tables_lib.a
/third_party/gflags/libgflags.a
/third_party/giflib/libdgiflib.a
/third_party/giflib/libgiflib_core.a
/third_party/grpc/libgpr.a
/third_party/grpc/libgrpc_core.a
/third_party/grpc/libgrpc_cpp.a
/third_party/hiredis/libhiredis.a
/third_party/icu/libicudata.a
/third_party/icu/libicuuc.a
/third_party/jsoncpp/libjsoncpp.a
/third_party/libjpeg_turbo/libjpeg.a
/third_party/libjpeg_turbo/src/libjpeg_turbo.a
/third_party/libpng/libpng.a
/third_party/optipng/libopngreduc.a
/third_party/protobuf/libprotobuf_full_do_not_use.a
/third_party/re2/libre2.a
/third_party/serf/libopenssl.a
/third_party/serf/libserf.a
/third_party/zlib/libzlib.a
/url/liburl_lib.a
```
But now there are almost totally different names, so I've just get all "*.a" libs except few copies and ones that related to envoy and tests.
At least, errors are different after
> If that yields the undefined reference you mention, then I think we're missing a dependency in a BUILD file on glog. Maybe adding it in https://github.com/apache/incubator-pagespeed-mod/blob/master/pagespeed/automatic/BUILD#L20 helps.
At the moment I didn't succeed with search for correct dependency name
However
- File "base/logging.h" contains "include glog/logging.h"
- I had error about "glog" until I've added path to "glog/logging.h" to list of "include"-paths for CC-command in ngx/config:$pagespeed_include
- When pagespeed_automatic.a is not complete - it also triggers "undefined reference"
So, maybe it needs to include some extra ".a" library?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] acachy edited a comment on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
acachy edited a comment on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-962558016
> @acachy how did you manage to build?
>
> I get a lot of errors like this: pagespeed/kernel/base/string_util.cc: In function 'GoogleString net_instaweb::internal::StrCatNineOrMore(const StringPiece*, ...)': pagespeed/kernel/base/string_util.cc:110:3: error: 'va_start' was not declared in this scope
@eilandert, looks like you've already moved far enough in #2084
My previous attempts were forgotten because of fail at the end...
So I tried again on latest commit and here is a part of my deploy script for Debian, if you wanna try to repeate. Sorry for a lot of trash ways inside
1. Prepare dir with sources
```
echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" >> /etc/apt/sources.list
wget --no-check-certificate -qO - https://bazel.build/bazel-release.pub.gpg | apt-key add - 2>&1
apt-get install -y bazel-4.1.0
apt-get install -y ninja-build
mkdir -p /usr/share/man/man1
apt-get install -y openjdk-11-jre-headless
mkdir /ngx
cd /ngx
wget https://github.com/nginx/nginx/archive/release-1.19.10.tar.gz
tar xzf *gz
rm *gz
git clone https://github.com/apache/incubator-pagespeed-ngx.git
cd incubator-pagespeed-ngx
git reset --hard 9e70f6d
git config --global url.https://github.com/apache/.insteadOf git://git.apache.org/
git submodule update --init --recursive --jobs=16 --force
cd ..
git clone --recursive https://github.com/apache/incubator-pagespeed-mod.git
cd incubator-pagespeed-mod
git reset --hard 6199d78
```
2. Bazel-build with few hacks
```
sed -i '1s/^/#include <cstdarg>\n/' pagespeed/kernel/base/string_util.cc
bazel build -c fastbuild //pagespeed/kernel/... //pagespeed/automatic/... //pagespeed/system/... //pagespeed/controller/... //pagespeed/opt/... //base/... //net/instaweb/... //third_party/... mod_pagespeed
sed -i -r 's/sys_siglist\[signum\]/strsignal(signum)/g' ./bazel-bin/../../../../../external/apr/threadproc/unix/signals.c
bazel build -c fastbuild //pagespeed/kernel/... //pagespeed/automatic/... //pagespeed/system/... //pagespeed/controller/... //pagespeed/opt/... //base/... //net/instaweb/... //third_party/... mod_pagespeed
```
3. Update pagespeed/automatic/bazel_merge.sh
- Skip Envoy
- Change final destination path for lib
```
#!/bin/bash
ADIR=$(bazel info bazel-bin)
ALIST=$(find $ADIR | grep \\.a$ | grep -v main | grep -v copy | grep -v envoy | xargs echo)
for item in ${ALIST[*]}
do
echo $item
done
find $ADIR | grep \\.a$ | grep -v main | grep -v copy
set -e
echo "merging libs"
./merge_libraries.sh ~/pagespeed_automatic.a.dirty $ALIST > merge.log
echo "renaming symbols"
# XXX(oschaaf): objcopy 2.31 doesn't like clang-7's output unless we pass in -fno-addrsig
# https://github.com/travitch/whole-program-llvm/issues/75
# not passing this in will make the scripts that rename symbols fail
./rename_c_symbols.sh ~/pagespeed_automatic.a.dirty ~/pagespeed_automatic.a > symbol-rename.log
rm ~/pagespeed_automatic.a.dirty
cp ~/pagespeed_automatic.a ./../../bazel-out/k8-fastbuild/bin/pagespeed/automatic/
mv ~/pagespeed_automatic.a ./
echo "done"
```
4. Create pagespeed_automatic.a
```
cd pagespeed/automatic
bash bazel_merge.sh
```
5. Try to build nginx
```
cd /ngx/nginx-release-1.19.10
MOD_PAGESPEED_DIR="/ngx/incubator-pagespeed-mod" ./auto/configure --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --modules-path=/usr/lib/nginx/modules --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-pcre-jit --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_v2_module --with-http_dav_module --with-http_slice_module --with-threads --with-http_addition_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_secure_link_module --with-http_sub_module --w
ith-mail=dynamic --with-mail_ssl_module --with-stream=dynamic --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_perl_module=dynamic --add-module=/ngx/incubator-pagespeed-ngx
```
6. There are include-errors that can be seen in details at the end of ./objs/autoconf.err
I fix it step-by-step by searching paths with missing files and adding extra include-dirs to /ngx/incubator-pagespeed-ngx/config
After few steps I have include rule like this:
```
pagespeed_include="\
$mod_pagespeed_dir \
$mod_pagespeed_dir/third_party/chromium/src \
$mod_pagespeed_dir/third_party/google-sparsehash/src/src \
$mod_pagespeed_dir/third_party/google-sparsehash/gen/arch/$os_name/$arch_name/include \
$mod_pagespeed_dir/third_party/grpc/src/include \
$mod_pagespeed_dir/third_party/protobuf/src/src \
$mod_pagespeed_dir/third_party/re2/src \
$mod_pagespeed_dir/out/$buildtype/obj/gen \
$mod_pagespeed_dir/out/$buildtype/obj/gen/protoc_out/instaweb \
$mod_pagespeed_dir/third_party/apr/src/include \
$mod_pagespeed_dir/third_party/aprutil/src/include \
$mod_pagespeed_dir/third_party/apr/gen/arch/$os_name/$arch_name/include \
$mod_pagespeed_dir/third_party/aprutil/gen/arch/$os_name/$arch_name/include \
$mod_pagespeed_dir/bazel-out/../../../external/com_google_absl \
$mod_pagespeed_dir/bazel-out/k8-fastbuild/bin/external/com_github_gflags_gflags/_virtual_includes/gflags \
$mod_pagespeed_dir/bazel-out/k8-fastbuild/bin/external/glog/_virtual_includes/default_glog_headers \
$mod_pagespeed_dir/bazel-out/k8-fastbuild/bin/external/google_sparsehash/_virtual_includes/google_sparsehash \
$mod_pagespeed_dir/url"
```
And my last error on attempt to use auto/configure:
```
...
/usr/bin/ld: doctype.cc:(.text+0x430): undefined reference to `google::LogMessageFatal::~LogMessageFatal()'
collect2: error: ld returned 1 exit status
```
Probably we should use other include-paths (for most files there are different copies in different dirs) or better ways to fix include-errors
I have no significant knowledge here, just tried to fix everything that I've seen
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] acachy commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
acachy commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-971381360
Same as last errors, but more clear example
```
/ngx/incubator-pagespeed-ngx/src/log_message_handler.cc: In function ‘void net_instaweb::log_message_handler::Install(ngx_log_t*)’:
/ngx/incubator-pagespeed-ngx/src/log_message_handler.cc:106:12: error: ‘SetLogMessageHandler’ is not a member of ‘logging’
106 | logging::SetLogMessageHandler(&LogMessageHandler);
```
Function "SetLogMessageHandler" not exists in mod_pagespeed at all, it can be seen only in _/incubator-pagespeed-ngx/testing-dependencies/mod_pagespeed/third_party/chromium/src/base/logging.h_
As far as I understand - it's not just build problem further...
1. Or we restore chronium parts around
I think it's "not meant to be"
Even if we build it (can use before-bazel commit for test) it should be difficult enough since "logging" and other things are exists in new places\ways
2. Or we update included-modules to make them fit current pagespeed-ngx
3. Or we fix pagespeed-ngx
As @Lofesa suggested I've deleted "base/debug" includes (together with deleting my funny chromium include). However, there are still this logging and other problems...
4. Or worse
I think some hybrid of №2+№3 is the best way
Not good feeling for non-dev approach thought...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-pagespeed-mod] eilandert commented on issue #2069: PSOL with bazel
Posted by GitBox <gi...@apache.org>.
eilandert commented on issue #2069:
URL: https://github.com/apache/incubator-pagespeed-mod/issues/2069#issuecomment-973403740
@oschaaf That would be cool, I hope to have time in a couple of weeks too.
In the meantime I've created https://github.com/eilandert/psol-bazel , you can clone this, run ./build.sh and docker does all the magic until the first unfixed error in https://github.com/eilandert/pagespeed-ngx-bazel ;-)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@pagespeed.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org