You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@madlib.apache.org by "orhankislal (via GitHub)" <gi...@apache.org> on 2023/05/17 19:50:17 UTC
[GitHub] [madlib] orhankislal opened a new pull request, #602: Build: Add PG13 on Ubuntu 20 support for Jenkins
orhankislal opened a new pull request, #602:
URL: https://github.com/apache/madlib/pull/602
This commit fixes the Jenkins pipeline for PG13. It disables some tests (with FIXME tag) since they had docker specific memory issues. PMML is also fixed for py3.
Finally, it adds PG14 and PG15 support for future convenience.
<!--
Thanks for sending a pull request! Here are some tips for you:
1. Refer to this link for contribution guidelines https://cwiki.apache.org/confluence/display/MADLIB/Contribution+Guidelines
2. Please Provide the Module Name, a JIRA Number and a short description about your changes.
-->
- [ ] Add the module name, JIRA# to PR/commit and description.
- [ ] Add tests for the change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@madlib.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [madlib] orhankislal commented on a diff in pull request #602: Build: Add PG13 on Ubuntu 20 support for Jenkins
Posted by "orhankislal (via GitHub)" <gi...@apache.org>.
orhankislal commented on code in PR #602:
URL: https://github.com/apache/madlib/pull/602#discussion_r1198142494
##########
tool/jenkins/jenkins_build.sh:
##########
@@ -51,35 +51,46 @@ docker pull madlib/postgres_13:jenkins
# Launch docker container with volume mounted from workdir
echo "-------------------------------"
cat <<EOF
-docker run -d -e POSTGRES_PASSWORD=postgres --name madlib -v "${workdir}":/madlib madlib/postgres_13:jenkins | tee logs/docker_setup.log
+docker run -d -t -e POSTGRES_PASSWORD=postgres --name madlib --ulimit core=-1 --privileged -v "${workdir}":/madlib madlib/postgres_13:jenkins | tee logs/docker_setup.log
EOF
-docker run -d -e POSTGRES_PASSWORD=postgres --name madlib -v "${workdir}":/madlib madlib/postgres_13:jenkins | tee logs/docker_setup.log
+docker run -d -t -e POSTGRES_PASSWORD=postgres --name madlib --ulimit core=-1 --privileged -v "${workdir}":/madlib madlib/postgres_13:jenkins | tee logs/docker_setup.log
+# -c log_statement=all -c logging_collector=on -c log_directory=logs -c log_filename=postgresql.log
Review Comment:
They were useful while I was trying to collect postgres logs. I'll add a comment to mention that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@madlib.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [madlib] khannaekta commented on a diff in pull request #602: Build: Add PG13 on Ubuntu 20 support for Jenkins
Posted by "khannaekta (via GitHub)" <gi...@apache.org>.
khannaekta commented on code in PR #602:
URL: https://github.com/apache/madlib/pull/602#discussion_r1198076326
##########
tool/jenkins/jenkins_build.sh:
##########
@@ -51,35 +51,46 @@ docker pull madlib/postgres_13:jenkins
# Launch docker container with volume mounted from workdir
echo "-------------------------------"
cat <<EOF
-docker run -d -e POSTGRES_PASSWORD=postgres --name madlib -v "${workdir}":/madlib madlib/postgres_13:jenkins | tee logs/docker_setup.log
+docker run -d -t -e POSTGRES_PASSWORD=postgres --name madlib --ulimit core=-1 --privileged -v "${workdir}":/madlib madlib/postgres_13:jenkins | tee logs/docker_setup.log
EOF
-docker run -d -e POSTGRES_PASSWORD=postgres --name madlib -v "${workdir}":/madlib madlib/postgres_13:jenkins | tee logs/docker_setup.log
+docker run -d -t -e POSTGRES_PASSWORD=postgres --name madlib --ulimit core=-1 --privileged -v "${workdir}":/madlib madlib/postgres_13:jenkins | tee logs/docker_setup.log
+# -c log_statement=all -c logging_collector=on -c log_directory=logs -c log_filename=postgresql.log
Review Comment:
we can probably remove this commented line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@madlib.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [madlib] orhankislal commented on a diff in pull request #602: Build: Add PG13 on Ubuntu 20 support for Jenkins
Posted by "orhankislal (via GitHub)" <gi...@apache.org>.
orhankislal commented on code in PR #602:
URL: https://github.com/apache/madlib/pull/602#discussion_r1198141227
##########
tool/docker/base/Dockerfile_postgres_11_Jenkins:
##########
@@ -16,25 +16,26 @@
# specific language governing permissions and limitations
# under the License.
-FROM postgres:11
+FROM postgres:11-bullseye
### Get postgres specific add-ons
-RUN apt-get update && apt-get install -y wget \
+RUN apt-get update
+RUN apt-get install -y wget \
build-essential \
postgresql-server-dev-11 \
- postgresql-plpython-11 \
+ postgresql-plpython3-11 \
openssl \
libssl-dev \
libboost-all-dev \
m4 \
rpm \
- python-pip \
- python-dev \
+ python3-pip \
+ python3-dev \
build-essential \
cmake \
libspatialindex-dev
-RUN pip install tensorflow==1.14 dill "rtree>=0.8,<0.9"
+RUN python3 -m pip install tensorflow dill rtree xgboost mock pandas numpy scikit-learn
Review Comment:
With python3 the latest versions work as expected. If we start to get any issues, we should get an error and try to fix it, or at least pin it to a different version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@madlib.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [madlib] orhankislal merged pull request #602: Build: Add PG13 on Ubuntu 20 support for Jenkins
Posted by "orhankislal (via GitHub)" <gi...@apache.org>.
orhankislal merged PR #602:
URL: https://github.com/apache/madlib/pull/602
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@madlib.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [madlib] khannaekta commented on a diff in pull request #602: Build: Add PG13 on Ubuntu 20 support for Jenkins
Posted by "khannaekta (via GitHub)" <gi...@apache.org>.
khannaekta commented on code in PR #602:
URL: https://github.com/apache/madlib/pull/602#discussion_r1198074476
##########
tool/docker/base/Dockerfile_postgres_11_Jenkins:
##########
@@ -16,25 +16,26 @@
# specific language governing permissions and limitations
# under the License.
-FROM postgres:11
+FROM postgres:11-bullseye
### Get postgres specific add-ons
-RUN apt-get update && apt-get install -y wget \
+RUN apt-get update
+RUN apt-get install -y wget \
build-essential \
postgresql-server-dev-11 \
- postgresql-plpython-11 \
+ postgresql-plpython3-11 \
openssl \
libssl-dev \
libboost-all-dev \
m4 \
rpm \
- python-pip \
- python-dev \
+ python3-pip \
+ python3-dev \
build-essential \
cmake \
libspatialindex-dev
-RUN pip install tensorflow==1.14 dill "rtree>=0.8,<0.9"
+RUN python3 -m pip install tensorflow dill rtree xgboost mock pandas numpy scikit-learn
Review Comment:
don't we need the specific versions anymore?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@madlib.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org