You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@whimsical.apache.org by Sam Ruby <ru...@intertwingly.net> on 2019/11/19 03:17:43 UTC
Docker beta testers wanted (was: Mac OSX read only file system)
On Sat, Oct 19, 2019 at 12:43 AM Dave Fisher <wa...@comcast.net> wrote:
>
> Maybe a docker image is the way to go. I’ll test it if you point it out.
Beta testers wanted: https://github.com/apache/whimsy/blob/master/DOCKER.md
- Sam Ruby
Re: Docker beta testers wanted (was: Mac OSX read only file system)
Posted by Roy Lenferink <rl...@apache.org>.
Thanks for the reply Sam!
I think I learned another thing today, don't just trust 'docker run
hello-world' to verify your docker configuration ;) In the end it was a
network problem with Docker on my side and strange enough a reboot solved
it.
The image is still building but I'll keep this list updated.
Op di 19 nov. 2019 om 22:15 schreef Sam Ruby <ru...@intertwingly.net>:
> On Tue, Nov 19, 2019 at 2:13 PM Roy Lenferink <rl...@apache.org>
> wrote:
> >
> > I just tried the Docker instructions, however the rake docker:update
> > command is failing for me with the error shown below. Am I maybe missing
> > something?
>
> [snip]
>
> > Err:1 http://security.ubuntu.com/ubuntu bionic-security InRelease
> > Temporary failure resolving 'security.ubuntu.com'
>
> It looks like you might be missing a working network connection?
>
> Does ping security.ubuntu.com work for you?
>
> - Sam Ruby
>
Re: Docker beta testers wanted (was: Mac OSX read only file system)
Posted by Sam Ruby <ru...@intertwingly.net>.
On Tue, Nov 19, 2019 at 2:13 PM Roy Lenferink <rl...@apache.org> wrote:
>
> I just tried the Docker instructions, however the rake docker:update
> command is failing for me with the error shown below. Am I maybe missing
> something?
[snip]
> Err:1 http://security.ubuntu.com/ubuntu bionic-security InRelease
> Temporary failure resolving 'security.ubuntu.com'
It looks like you might be missing a working network connection?
Does ping security.ubuntu.com work for you?
- Sam Ruby
Re: Docker beta testers wanted (was: Mac OSX read only file system)
Posted by Roy Lenferink <rl...@apache.org>.
I just tried the Docker instructions, however the rake docker:update
command is failing for me with the error shown below. Am I maybe missing
something?
I am on CentOS 8.0.1905 with Docker version 19.03.5, build 633a0ea.
I also tested using podman with podman-compose (with alias docker='podman'
etc..), however this doesn't even start with the rake docker:update command.
Regards,
Roy
[root@centos8 whimsy]# rake docker:update
docker-compose build web
Building web
Step 1/8 : FROM ubuntu:18.04
18.04: Pulling from library/ubuntu
7ddbc47eeb70: Pull complete
c1bbdc448b72: Pull complete
8c3b70e39044: Pull complete
45d437916d57: Pull complete
Digest:
sha256:6e9f67fa63b0323e9a1e587fd71c561ba48a034504fb804fd26fd8800039835d
Status: Downloaded newer image for ubuntu:18.04
---> 775349758637
Step 2/8 : ENV GEM_HOME="/srv/gems" LANG=C.UTF-8 LC_ALL=C.UTF-8
---> Running in 185fda5837b4
Removing intermediate container 185fda5837b4
---> 65ccad352b15
Step 3/8 : RUN apt-get update && apt-get install -y curl
software-properties-common apt-utils && curl -sL
https://deb.nodesource.com/setup_12.x | bash - && echo "deb
http://opensource.wandisco.com/ubuntu bionic svn110" >
/etc/apt/sources.list.d/subversion.list && curl -sL
http://opensource.wandisco.com/wandisco-debian-new.gpg | apt-key add
- && apt-get update && DEBIAN_FRONTEND='noninteractive' apt-get
install -y apache2 subversion git build-essential
libgmp3-dev libldap2-dev libsasl2-dev python3-pip
ruby-dev zlib1g-dev imagemagick img2pdf nodejs
procmail poppler-utils texlive-extra-utils gnupg2
libcurl4-openssl-dev libssl-dev apache2-dev libapr1-dev
libaprutil1-dev && gem install bundler passenger
--install_dir=/var/lib/gems/2.5.0 && passenger-install-apache2-module
--auto && passenger-install-apache2-module --snippet >
/etc/apache2/conf-enabled/passenger.conf && pip3 install img2pdf &&
a2enmod cgi && a2enmod headers && a2enmod rewrite && a2enmod
authnz_ldap && a2enmod speling && a2enmod remoteip && a2enmod
expires && a2enmod proxy_wstunnel && echo "ServerName whimsy.local"
> /etc/apache2/conf-enabled/servername.conf
---> Running in ef2b4a44e5db
Err:1 http://security.ubuntu.com/ubuntu bionic-security InRelease
Temporary failure resolving 'security.ubuntu.com'
Err:2 http://archive.ubuntu.com/ubuntu bionic InRelease
Temporary failure resolving 'archive.ubuntu.com'
Err:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
Temporary failure resolving 'archive.ubuntu.com'
Err:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
Temporary failure resolving 'archive.ubuntu.com'
Reading package lists...
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic/InRelease
Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch
http://archive.ubuntu.com/ubuntu/dists/bionic-updates/InRelease Temporary
failure resolving 'archive.ubuntu.com'
W: Failed to fetch
http://archive.ubuntu.com/ubuntu/dists/bionic-backports/InRelease
Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch
http://security.ubuntu.com/ubuntu/dists/bionic-security/InRelease
Temporary failure resolving 'security.ubuntu.com'
W: Some index files failed to download. They have been ignored, or old ones
used instead.
Reading package lists...
Building dependency tree...
Reading state information...
Package apt-utils is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
apt
E: Unable to locate package curl
E: Unable to locate package software-properties-common
E: Package 'apt-utils' has no installation candidate
ERROR: Service 'web' failed to build: The command '/bin/sh -c apt-get
update && apt-get install -y curl software-properties-common apt-utils
&& curl -sL https://deb.nodesource.com/setup_12.x | bash - && echo
"deb http://opensource.wandisco.com/ubuntu bionic svn110" >
/etc/apt/sources.list.d/subversion.list && curl -sL
http://opensource.wandisco.com/wandisco-debian-new.gpg | apt-key add
- && apt-get update && DEBIAN_FRONTEND='noninteractive' apt-get
install -y apache2 subversion git build-essential
libgmp3-dev libldap2-dev libsasl2-dev python3-pip
ruby-dev zlib1g-dev imagemagick img2pdf nodejs
procmail poppler-utils texlive-extra-utils gnupg2
libcurl4-openssl-dev libssl-dev apache2-dev libapr1-dev
libaprutil1-dev && gem install bundler passenger
--install_dir=/var/lib/gems/2.5.0 && passenger-install-apache2-module
--auto && passenger-install-apache2-module --snippet >
/etc/apache2/conf-enabled/passenger.conf && pip3 install img2pdf &&
a2enmod cgi && a2enmod headers && a2enmod rewrite && a2enmod
authnz_ldap && a2enmod speling && a2enmod remoteip && a2enmod
expires && a2enmod proxy_wstunnel && echo "ServerName whimsy.local"
> /etc/apache2/conf-enabled/servername.conf' returned a non-zero code: 100
rake aborted!
Command failed with status (1): [docker-compose build web...]
/root/whimsy/whimsy/Rakefile:262:in `block (3 levels) in <top (required)>'
/root/whimsy/whimsy/Rakefile:261:in `chdir'
/root/whimsy/whimsy/Rakefile:261:in `block (2 levels) in <top (required)>'
/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
Tasks: TOP => docker:update => docker:build
(See full trace by running task with --trace)
[root@centos8 whimsy]#
Op di 19 nov. 2019 om 18:49 schreef Sam Ruby <ru...@intertwingly.net>:
> On Tue, Nov 19, 2019 at 9:15 AM sebb <se...@gmail.com> wrote:
> >
> > On Tue, 19 Nov 2019 at 13:34, Sam Ruby <ru...@intertwingly.net> wrote:
> >
> > > On Tue, Nov 19, 2019 at 8:17 AM sebb <se...@gmail.com> wrote:
> > > >
> > > > On Tue, 19 Nov 2019 at 12:26, sebb <se...@gmail.com> wrote:
> > > >
> > > > > docker:update had issues pulling down various svn repos, e.g.
> > > > >
> > > > > svn: E170013: Unable to connect to a repository at URL ...
> > > > > svn: E125009: Invalid config: unable to load certificate file
> > > > > '/Users/sebb/.subversion/Thawte_Premium_Server_CA.pem'
> > > > >
> > > > > I was able to remove the Invalid config error by changing
> /Users/sebb
> > > =>
> > > > > /root
> > > > > However I then got the error:
> > > > >
> > > > > svn: E000111: Error running context: Connection refused
> > > > >
> > > > > I assume that is because my SVN is set to grab passwords from the
> macOS
> > > > > keychain
> > > > >
> > > > >
> > > > Turns out that SVN was down at the time.
> > > >
> > > > Once I copied the cert to the expected place it worked OK.
> > > >
> > > > I've just realised that the /srv/svn directory is mounted from the
> host
> > > OS.
> > > > So checkouts in the image actually update the host working directory.
> > > > Given that, it seems to me it would make more sense to do the
> check-out
> > > on
> > > > the host under /srv, and mount that instead.
> > >
> > > Two issues:
> > >
> > > 1) From time to time subversion makes changes to the file format. It
> > > could be the case that the version of subversion you have on the host
> > > is not compatible with the version of subversion in the container.
> > >
> >
> > In which case having the checkout on the host filing system is risky.
> > It would be safer to have the data all within the container, and not
> > accessible from the host.
> > AFAICT it should not take any more space overall.
>
> It's complicated, and a lot depends on what the use cases we intend to
> support are.
>
> For now, I've updated the Dockerfile to install a compatible version
> of subversion, and updated DOCKER.md to explain the constraints.
>
> I actually developed these files using the /srv directory from the
> MACOSX instructions.
>
> My thoughts are that most people would feel more comfortable doing
> their browsing and updates in the host environment, so a shared mount
> would be in order.
>
> I definitely would not want to put repository data in images as those
> are frozen (i.e., updates would be done in separate layers and would
> take up more space). Similarly, I would not want to put repository
> data in container space as those go away when the container exits.
> Putting this data into a docker volume would work (and may be faster),
> but may be less convenient for people that want to access the data
> from their host.
>
> These constraints are particularly true for the whimsy directory. I'm
> comfortable exec'ing into the container and editing the files using
> vim, but others may want to use their Mac IDEs.
>
> > But for those who can cope with the possible issue, there could be
> another
> > build that uses host data under /svn
>
> I do most of my whimsy work on my Ubuntu desktop machine, but I
> recently updated the MACOSX instructions and have this working on my
> laptop. I'm more likely to use those instructions than the container
> instructions, but being able to share a /srv directory would be
> convenient for me, allowing me to switch between the two.
>
> > 2) Doing the checkout on the host only defers the problem. A number
> > > of whimsy processes access svn (checkouts and commits).
> > >
> > >
> > It might be useful to be able to disable commits for testing purposes.
> > This is also true of LDAP.
>
> Agreed, though for testing purposes I have defined a locally hosted
> subversion repository for the board agenda tests, allowing commits to
> be tested.
>
> > Regarding the SVN checkouts, maybe it would make sense to start by
> checking
> > out only the top-level files/directories for some of the less-used/larger
> > trees.
>
> The largest are iclas, cclas, grants, and member-apps. I believe that
> these are only used by one tool, and primarily to determine if there
> is a collision. In these days of terabyte hard drives, it isn't a
> huge issue for me, but may be for others.
>
> > > Also there does not seem to be a visual editor included in the docker
> > > build.
> > > > > (I had to use sed)
> > > > > It would be useful to have vi(m) or such.
> > >
> > > I've been doing an "apt install vim" when needed, but, yes, it would
> > > be nice to have vim already installed. There probably are a lot of
> > > affordances that would be helpful. At one point I tried an "apachectl
> > > status" and that failed as there wasn't a browser installed that it
> > > could use to fetch the status. I no longer recall what other commands
> > > I tried to use to check network status and the like only to find that
> > > they weren't installed.
> > >
> > > - Sam Ruby
> > >
> > > P.S. Thanks for helping work through this!
>
> - Sam Ruby
>
Re: Docker beta testers wanted (was: Mac OSX read only file system)
Posted by Matt Sicker <bo...@gmail.com>.
For the /srv checkouts, if you make them volumes, you can override that at
runtime to use a bound directory instead of a generic Docker volume (which
itself just goes in Docker’s data directory otherwise). Using a volume is
important regardless for file system performance on that data.
As for keeping the data format in sync, if that changes again in
the future, update the checkout instructions to use the appropriate feature
flag to use the old file format.
On Tue, Nov 19, 2019 at 14:35 sebb <se...@gmail.com> wrote:
> On Tue, 19 Nov 2019 at 17:49, Sam Ruby <ru...@intertwingly.net> wrote:
>
> > On Tue, Nov 19, 2019 at 9:15 AM sebb <se...@gmail.com> wrote:
> > >
> > > On Tue, 19 Nov 2019 at 13:34, Sam Ruby <ru...@intertwingly.net> wrote:
> > >
> > > > On Tue, Nov 19, 2019 at 8:17 AM sebb <se...@gmail.com> wrote:
> > > > >
> > > > > On Tue, 19 Nov 2019 at 12:26, sebb <se...@gmail.com> wrote:
> > > > >
> > > > > > docker:update had issues pulling down various svn repos, e.g.
> > > > > >
> > > > > > svn: E170013: Unable to connect to a repository at URL ...
> > > > > > svn: E125009: Invalid config: unable to load certificate file
> > > > > > '/Users/sebb/.subversion/Thawte_Premium_Server_CA.pem'
> > > > > >
> > > > > > I was able to remove the Invalid config error by changing
> > /Users/sebb
> > > > =>
> > > > > > /root
> > > > > > However I then got the error:
> > > > > >
> > > > > > svn: E000111: Error running context: Connection refused
> > > > > >
> > > > > > I assume that is because my SVN is set to grab passwords from the
> > macOS
> > > > > > keychain
> > > > > >
> > > > > >
> > > > > Turns out that SVN was down at the time.
> > > > >
> > > > > Once I copied the cert to the expected place it worked OK.
> > > > >
> > > > > I've just realised that the /srv/svn directory is mounted from the
> > host
> > > > OS.
> > > > > So checkouts in the image actually update the host working
> directory.
> > > > > Given that, it seems to me it would make more sense to do the
> > check-out
> > > > on
> > > > > the host under /srv, and mount that instead.
> > > >
> > > > Two issues:
> > > >
> > > > 1) From time to time subversion makes changes to the file format. It
> > > > could be the case that the version of subversion you have on the host
> > > > is not compatible with the version of subversion in the container.
> > > >
> > >
> > > In which case having the checkout on the host filing system is risky.
> > > It would be safer to have the data all within the container, and not
> > > accessible from the host.
> > > AFAICT it should not take any more space overall.
> >
> > It's complicated, and a lot depends on what the use cases we intend to
> > support are.
> >
> > For now, I've updated the Dockerfile to install a compatible version
> > of subversion, and updated DOCKER.md to explain the constraints.
> >
> > I actually developed these files using the /srv directory from the
> > MACOSX instructions.
> >
> > My thoughts are that most people would feel more comfortable doing
> > their browsing and updates in the host environment, so a shared mount
> > would be in order.
> >
> >
> OK
>
>
> > I definitely would not want to put repository data in images as those
> > are frozen (i.e., updates would be done in separate layers and would
> > take up more space). Similarly, I would not want to put repository
> > data in container space as those go away when the container exits.
> >
>
> I see.
>
>
> > Putting this data into a docker volume would work (and may be faster),
> > but may be less convenient for people that want to access the data
> > from their host.
> >
> > These constraints are particularly true for the whimsy directory. I'm
> > comfortable exec'ing into the container and editing the files using
> > vim, but others may want to use their Mac IDEs.
> >
> >
> Good point.
>
>
> > > But for those who can cope with the possible issue, there could be
> > another
> > > build that uses host data under /svn
> >
> > I do most of my whimsy work on my Ubuntu desktop machine, but I
> > recently updated the MACOSX instructions and have this working on my
> > laptop. I'm more likely to use those instructions than the container
> > instructions, but being able to share a /srv directory would be
> > convenient for me, allowing me to switch between the two.
> >
> > > 2) Doing the checkout on the host only defers the problem. A number
> > > > of whimsy processes access svn (checkouts and commits).
> > > >
> > > >
> > > It might be useful to be able to disable commits for testing purposes.
> > > This is also true of LDAP.
> >
> > Agreed, though for testing purposes I have defined a locally hosted
> > subversion repository for the board agenda tests, allowing commits to
> > be tested.
> >
> > > Regarding the SVN checkouts, maybe it would make sense to start by
> > checking
> > > out only the top-level files/directories for some of the
> less-used/larger
> > > trees.
> >
> > The largest are iclas, cclas, grants, and member-apps. I believe that
> > these are only used by one tool, and primarily to determine if there
> > is a collision. In these days of terabyte hard drives, it isn't a
> > huge issue for me, but may be for others.
> >
> >
> It's also a lot of data to download, and there might need to be a separate
> copy if the host version cannot be shared.
>
> In the case of iclas, I wrote a script to list the SVN repo contents and
> create empty files/directories locally.
> This works well with an empty checkout.
>
>
> > > Also there does not seem to be a visual editor included in the docker
> > > > build.
> > > > > > (I had to use sed)
> > > > > > It would be useful to have vi(m) or such.
> > > >
> > > > I've been doing an "apt install vim" when needed, but, yes, it would
> > > > be nice to have vim already installed. There probably are a lot of
> > > > affordances that would be helpful. At one point I tried an
> "apachectl
> > > > status" and that failed as there wasn't a browser installed that it
> > > > could use to fetch the status. I no longer recall what other
> commands
> > > > I tried to use to check network status and the like only to find that
> > > > they weren't installed.
> > > >
> > > > - Sam Ruby
> > > >
> > > > P.S. Thanks for helping work through this!
> >
> > - Sam Ruby
> >
>
--
Matt Sicker <bo...@gmail.com>
Re: Docker beta testers wanted (was: Mac OSX read only file system)
Posted by sebb <se...@gmail.com>.
On Tue, 19 Nov 2019 at 17:49, Sam Ruby <ru...@intertwingly.net> wrote:
> On Tue, Nov 19, 2019 at 9:15 AM sebb <se...@gmail.com> wrote:
> >
> > On Tue, 19 Nov 2019 at 13:34, Sam Ruby <ru...@intertwingly.net> wrote:
> >
> > > On Tue, Nov 19, 2019 at 8:17 AM sebb <se...@gmail.com> wrote:
> > > >
> > > > On Tue, 19 Nov 2019 at 12:26, sebb <se...@gmail.com> wrote:
> > > >
> > > > > docker:update had issues pulling down various svn repos, e.g.
> > > > >
> > > > > svn: E170013: Unable to connect to a repository at URL ...
> > > > > svn: E125009: Invalid config: unable to load certificate file
> > > > > '/Users/sebb/.subversion/Thawte_Premium_Server_CA.pem'
> > > > >
> > > > > I was able to remove the Invalid config error by changing
> /Users/sebb
> > > =>
> > > > > /root
> > > > > However I then got the error:
> > > > >
> > > > > svn: E000111: Error running context: Connection refused
> > > > >
> > > > > I assume that is because my SVN is set to grab passwords from the
> macOS
> > > > > keychain
> > > > >
> > > > >
> > > > Turns out that SVN was down at the time.
> > > >
> > > > Once I copied the cert to the expected place it worked OK.
> > > >
> > > > I've just realised that the /srv/svn directory is mounted from the
> host
> > > OS.
> > > > So checkouts in the image actually update the host working directory.
> > > > Given that, it seems to me it would make more sense to do the
> check-out
> > > on
> > > > the host under /srv, and mount that instead.
> > >
> > > Two issues:
> > >
> > > 1) From time to time subversion makes changes to the file format. It
> > > could be the case that the version of subversion you have on the host
> > > is not compatible with the version of subversion in the container.
> > >
> >
> > In which case having the checkout on the host filing system is risky.
> > It would be safer to have the data all within the container, and not
> > accessible from the host.
> > AFAICT it should not take any more space overall.
>
> It's complicated, and a lot depends on what the use cases we intend to
> support are.
>
> For now, I've updated the Dockerfile to install a compatible version
> of subversion, and updated DOCKER.md to explain the constraints.
>
> I actually developed these files using the /srv directory from the
> MACOSX instructions.
>
> My thoughts are that most people would feel more comfortable doing
> their browsing and updates in the host environment, so a shared mount
> would be in order.
>
>
OK
> I definitely would not want to put repository data in images as those
> are frozen (i.e., updates would be done in separate layers and would
> take up more space). Similarly, I would not want to put repository
> data in container space as those go away when the container exits.
>
I see.
> Putting this data into a docker volume would work (and may be faster),
> but may be less convenient for people that want to access the data
> from their host.
>
> These constraints are particularly true for the whimsy directory. I'm
> comfortable exec'ing into the container and editing the files using
> vim, but others may want to use their Mac IDEs.
>
>
Good point.
> > But for those who can cope with the possible issue, there could be
> another
> > build that uses host data under /svn
>
> I do most of my whimsy work on my Ubuntu desktop machine, but I
> recently updated the MACOSX instructions and have this working on my
> laptop. I'm more likely to use those instructions than the container
> instructions, but being able to share a /srv directory would be
> convenient for me, allowing me to switch between the two.
>
> > 2) Doing the checkout on the host only defers the problem. A number
> > > of whimsy processes access svn (checkouts and commits).
> > >
> > >
> > It might be useful to be able to disable commits for testing purposes.
> > This is also true of LDAP.
>
> Agreed, though for testing purposes I have defined a locally hosted
> subversion repository for the board agenda tests, allowing commits to
> be tested.
>
> > Regarding the SVN checkouts, maybe it would make sense to start by
> checking
> > out only the top-level files/directories for some of the less-used/larger
> > trees.
>
> The largest are iclas, cclas, grants, and member-apps. I believe that
> these are only used by one tool, and primarily to determine if there
> is a collision. In these days of terabyte hard drives, it isn't a
> huge issue for me, but may be for others.
>
>
It's also a lot of data to download, and there might need to be a separate
copy if the host version cannot be shared.
In the case of iclas, I wrote a script to list the SVN repo contents and
create empty files/directories locally.
This works well with an empty checkout.
> > Also there does not seem to be a visual editor included in the docker
> > > build.
> > > > > (I had to use sed)
> > > > > It would be useful to have vi(m) or such.
> > >
> > > I've been doing an "apt install vim" when needed, but, yes, it would
> > > be nice to have vim already installed. There probably are a lot of
> > > affordances that would be helpful. At one point I tried an "apachectl
> > > status" and that failed as there wasn't a browser installed that it
> > > could use to fetch the status. I no longer recall what other commands
> > > I tried to use to check network status and the like only to find that
> > > they weren't installed.
> > >
> > > - Sam Ruby
> > >
> > > P.S. Thanks for helping work through this!
>
> - Sam Ruby
>
Re: Docker beta testers wanted (was: Mac OSX read only file system)
Posted by Sam Ruby <ru...@intertwingly.net>.
On Tue, Nov 19, 2019 at 9:15 AM sebb <se...@gmail.com> wrote:
>
> On Tue, 19 Nov 2019 at 13:34, Sam Ruby <ru...@intertwingly.net> wrote:
>
> > On Tue, Nov 19, 2019 at 8:17 AM sebb <se...@gmail.com> wrote:
> > >
> > > On Tue, 19 Nov 2019 at 12:26, sebb <se...@gmail.com> wrote:
> > >
> > > > docker:update had issues pulling down various svn repos, e.g.
> > > >
> > > > svn: E170013: Unable to connect to a repository at URL ...
> > > > svn: E125009: Invalid config: unable to load certificate file
> > > > '/Users/sebb/.subversion/Thawte_Premium_Server_CA.pem'
> > > >
> > > > I was able to remove the Invalid config error by changing /Users/sebb
> > =>
> > > > /root
> > > > However I then got the error:
> > > >
> > > > svn: E000111: Error running context: Connection refused
> > > >
> > > > I assume that is because my SVN is set to grab passwords from the macOS
> > > > keychain
> > > >
> > > >
> > > Turns out that SVN was down at the time.
> > >
> > > Once I copied the cert to the expected place it worked OK.
> > >
> > > I've just realised that the /srv/svn directory is mounted from the host
> > OS.
> > > So checkouts in the image actually update the host working directory.
> > > Given that, it seems to me it would make more sense to do the check-out
> > on
> > > the host under /srv, and mount that instead.
> >
> > Two issues:
> >
> > 1) From time to time subversion makes changes to the file format. It
> > could be the case that the version of subversion you have on the host
> > is not compatible with the version of subversion in the container.
> >
>
> In which case having the checkout on the host filing system is risky.
> It would be safer to have the data all within the container, and not
> accessible from the host.
> AFAICT it should not take any more space overall.
It's complicated, and a lot depends on what the use cases we intend to
support are.
For now, I've updated the Dockerfile to install a compatible version
of subversion, and updated DOCKER.md to explain the constraints.
I actually developed these files using the /srv directory from the
MACOSX instructions.
My thoughts are that most people would feel more comfortable doing
their browsing and updates in the host environment, so a shared mount
would be in order.
I definitely would not want to put repository data in images as those
are frozen (i.e., updates would be done in separate layers and would
take up more space). Similarly, I would not want to put repository
data in container space as those go away when the container exits.
Putting this data into a docker volume would work (and may be faster),
but may be less convenient for people that want to access the data
from their host.
These constraints are particularly true for the whimsy directory. I'm
comfortable exec'ing into the container and editing the files using
vim, but others may want to use their Mac IDEs.
> But for those who can cope with the possible issue, there could be another
> build that uses host data under /svn
I do most of my whimsy work on my Ubuntu desktop machine, but I
recently updated the MACOSX instructions and have this working on my
laptop. I'm more likely to use those instructions than the container
instructions, but being able to share a /srv directory would be
convenient for me, allowing me to switch between the two.
> 2) Doing the checkout on the host only defers the problem. A number
> > of whimsy processes access svn (checkouts and commits).
> >
> >
> It might be useful to be able to disable commits for testing purposes.
> This is also true of LDAP.
Agreed, though for testing purposes I have defined a locally hosted
subversion repository for the board agenda tests, allowing commits to
be tested.
> Regarding the SVN checkouts, maybe it would make sense to start by checking
> out only the top-level files/directories for some of the less-used/larger
> trees.
The largest are iclas, cclas, grants, and member-apps. I believe that
these are only used by one tool, and primarily to determine if there
is a collision. In these days of terabyte hard drives, it isn't a
huge issue for me, but may be for others.
> > Also there does not seem to be a visual editor included in the docker
> > build.
> > > > (I had to use sed)
> > > > It would be useful to have vi(m) or such.
> >
> > I've been doing an "apt install vim" when needed, but, yes, it would
> > be nice to have vim already installed. There probably are a lot of
> > affordances that would be helpful. At one point I tried an "apachectl
> > status" and that failed as there wasn't a browser installed that it
> > could use to fetch the status. I no longer recall what other commands
> > I tried to use to check network status and the like only to find that
> > they weren't installed.
> >
> > - Sam Ruby
> >
> > P.S. Thanks for helping work through this!
- Sam Ruby
Re: Docker beta testers wanted (was: Mac OSX read only file system)
Posted by sebb <se...@gmail.com>.
On Tue, 19 Nov 2019 at 13:34, Sam Ruby <ru...@intertwingly.net> wrote:
> On Tue, Nov 19, 2019 at 8:17 AM sebb <se...@gmail.com> wrote:
> >
> > On Tue, 19 Nov 2019 at 12:26, sebb <se...@gmail.com> wrote:
> >
> > > docker:update had issues pulling down various svn repos, e.g.
> > >
> > > svn: E170013: Unable to connect to a repository at URL ...
> > > svn: E125009: Invalid config: unable to load certificate file
> > > '/Users/sebb/.subversion/Thawte_Premium_Server_CA.pem'
> > >
> > > I was able to remove the Invalid config error by changing /Users/sebb
> =>
> > > /root
> > > However I then got the error:
> > >
> > > svn: E000111: Error running context: Connection refused
> > >
> > > I assume that is because my SVN is set to grab passwords from the macOS
> > > keychain
> > >
> > >
> > Turns out that SVN was down at the time.
> >
> > Once I copied the cert to the expected place it worked OK.
> >
> > I've just realised that the /srv/svn directory is mounted from the host
> OS.
> > So checkouts in the image actually update the host working directory.
> > Given that, it seems to me it would make more sense to do the check-out
> on
> > the host under /srv, and mount that instead.
>
> Two issues:
>
> 1) From time to time subversion makes changes to the file format. It
> could be the case that the version of subversion you have on the host
> is not compatible with the version of subversion in the container.
>
In which case having the checkout on the host filing system is risky.
It would be safer to have the data all within the container, and not
accessible from the host.
AFAICT it should not take any more space overall.
But for those who can cope with the possible issue, there could be another
build that uses host data under /svn
2) Doing the checkout on the host only defers the problem. A number
> of whimsy processes access svn (checkouts and commits).
>
>
It might be useful to be able to disable commits for testing purposes.
This is also true of LDAP.
Regarding the SVN checkouts, maybe it would make sense to start by checking
out only the top-level files/directories for some of the less-used/larger
trees.
> Also there does not seem to be a visual editor included in the docker
> build.
> > > (I had to use sed)
> > > It would be useful to have vi(m) or such.
>
> I've been doing an "apt install vim" when needed, but, yes, it would
> be nice to have vim already installed. There probably are a lot of
> affordances that would be helpful. At one point I tried an "apachectl
> status" and that failed as there wasn't a browser installed that it
> could use to fetch the status. I no longer recall what other commands
> I tried to use to check network status and the like only to find that
> they weren't installed.
>
> - Sam Ruby
>
> P.S. Thanks for helping work through this!
>
Re: Docker beta testers wanted (was: Mac OSX read only file system)
Posted by Sam Ruby <ru...@intertwingly.net>.
On Tue, Nov 19, 2019 at 8:17 AM sebb <se...@gmail.com> wrote:
>
> On Tue, 19 Nov 2019 at 12:26, sebb <se...@gmail.com> wrote:
>
> > docker:update had issues pulling down various svn repos, e.g.
> >
> > svn: E170013: Unable to connect to a repository at URL ...
> > svn: E125009: Invalid config: unable to load certificate file
> > '/Users/sebb/.subversion/Thawte_Premium_Server_CA.pem'
> >
> > I was able to remove the Invalid config error by changing /Users/sebb =>
> > /root
> > However I then got the error:
> >
> > svn: E000111: Error running context: Connection refused
> >
> > I assume that is because my SVN is set to grab passwords from the macOS
> > keychain
> >
> >
> Turns out that SVN was down at the time.
>
> Once I copied the cert to the expected place it worked OK.
>
> I've just realised that the /srv/svn directory is mounted from the host OS.
> So checkouts in the image actually update the host working directory.
> Given that, it seems to me it would make more sense to do the check-out on
> the host under /srv, and mount that instead.
Two issues:
1) From time to time subversion makes changes to the file format. It
could be the case that the version of subversion you have on the host
is not compatible with the version of subversion in the container.
2) Doing the checkout on the host only defers the problem. A number
of whimsy processes access svn (checkouts and commits).
> Also there does not seem to be a visual editor included in the docker build.
> > (I had to use sed)
> > It would be useful to have vi(m) or such.
I've been doing an "apt install vim" when needed, but, yes, it would
be nice to have vim already installed. There probably are a lot of
affordances that would be helpful. At one point I tried an "apachectl
status" and that failed as there wasn't a browser installed that it
could use to fetch the status. I no longer recall what other commands
I tried to use to check network status and the like only to find that
they weren't installed.
- Sam Ruby
P.S. Thanks for helping work through this!
Re: Docker beta testers wanted (was: Mac OSX read only file system)
Posted by sebb <se...@gmail.com>.
On Tue, 19 Nov 2019 at 12:26, sebb <se...@gmail.com> wrote:
> docker:update had issues pulling down various svn repos, e.g.
>
> svn: E170013: Unable to connect to a repository at URL ...
> svn: E125009: Invalid config: unable to load certificate file
> '/Users/sebb/.subversion/Thawte_Premium_Server_CA.pem'
>
> I was able to remove the Invalid config error by changing /Users/sebb =>
> /root
> However I then got the error:
>
> svn: E000111: Error running context: Connection refused
>
> I assume that is because my SVN is set to grab passwords from the macOS
> keychain
>
>
Turns out that SVN was down at the time.
Once I copied the cert to the expected place it worked OK.
I've just realised that the /srv/svn directory is mounted from the host OS.
So checkouts in the image actually update the host working directory.
Given that, it seems to me it would make more sense to do the check-out on
the host under /srv, and mount that instead.
Also there does not seem to be a visual editor included in the docker build.
> (I had to use sed)
> It would be useful to have vi(m) or such.
>
> S.
> On Tue, 19 Nov 2019 at 03:24, Sam Ruby <ru...@intertwingly.net> wrote:
>
>> On Sat, Oct 19, 2019 at 12:43 AM Dave Fisher <wa...@comcast.net>
>> wrote:
>> >
>> > Maybe a docker image is the way to go. I’ll test it if you point it out.
>>
>> Beta testers wanted:
>> https://github.com/apache/whimsy/blob/master/DOCKER.md
>>
>> - Sam Ruby
>>
>
Re: Docker beta testers wanted (was: Mac OSX read only file system)
Posted by sebb <se...@gmail.com>.
docker:update had issues pulling down various svn repos, e.g.
svn: E170013: Unable to connect to a repository at URL ...
svn: E125009: Invalid config: unable to load certificate file
'/Users/sebb/.subversion/Thawte_Premium_Server_CA.pem'
I was able to remove the Invalid config error by changing /Users/sebb =>
/root
However I then got the error:
svn: E000111: Error running context: Connection refused
I assume that is because my SVN is set to grab passwords from the macOS
keychain
Also there does not seem to be a visual editor included in the docker build.
(I had to use sed)
It would be useful to have vi(m) or such.
S.
On Tue, 19 Nov 2019 at 03:24, Sam Ruby <ru...@intertwingly.net> wrote:
> On Sat, Oct 19, 2019 at 12:43 AM Dave Fisher <wa...@comcast.net>
> wrote:
> >
> > Maybe a docker image is the way to go. I’ll test it if you point it out.
>
> Beta testers wanted:
> https://github.com/apache/whimsy/blob/master/DOCKER.md
>
> - Sam Ruby
>