Difference between revisions of "Pymediawikidocker"
Jump to navigation
Jump to search
| Line 250: | Line 250: | ||
/usr/bin/docker-compose | /usr/bin/docker-compose | ||
</source> | </source> | ||
| − | + | === create docker compose files === | |
| + | <source lang='bash' highlight='1'> | ||
python mwdocker/mwcluster.py --forceRebuild --versionList 1.35.5 --basePort 8480 --sqlBasePort 10306 --mariaDBVersion 10.5 | python mwdocker/mwcluster.py --forceRebuild --versionList 1.35.5 --basePort 8480 --sqlBasePort 10306 --mariaDBVersion 10.5 | ||
| + | </source> | ||
| + | === debug docker compose results - ports and other configuration parts === | ||
| + | <source lang='bash' highlight='1-3'> | ||
| + | cd ~/.pymediawikidocker/mw1_35_5 | ||
| + | docker-compose up | ||
| + | # in another terminal session: | ||
| + | docker ps | ||
| + | docker ps | ||
| + | CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES | ||
| + | 682b81b36fc7 mw1_35_5_mw "docker-php-entrypoi…" 2 minutes ago Up 2 minutes 0.0.0.0:8480->80/tcp, :::8480->80/tcp mw1_35_5_mw_1 | ||
| + | 3017fbc569d3 mariadb:10.5 "docker-entrypoint.s…" 2 minutes ago Up 2 minutes 0.0.0.0:10306->3306/tcp, :::10306->3306/tcp mw1_35_5_db_1 | ||
| + | </source> | ||
== Semantic MediaWiki cluster with WikiUser creation == | == Semantic MediaWiki cluster with WikiUser creation == | ||
Revision as of 09:35, 2 July 2022
OsProject
| OsProject | |
|---|---|
 edit
| |
| id | pymediawikidocker |
| state | |
| owner | WolfgangFahl |
| title | Python controlled mediawiki docker image installation |
| url | https://github.com/WolfgangFahl/pymediawikidocker |
| version | 0.2.0 |
| description | |
| date | 2021-12-19 |
| since | |
| until | |
Motivation
Official Mediawiki docker images are available for the different Mediawiki versions.
These images need additional infrastructure to create useable Mediawiki environments. The goal of this project is to semi-automatically create such environments and test the functionality. Python was choosen as a development environment for the libraries that are needed and available to perform the tasks:
- https://github.com/gabrieldemarmiesse/python-on-whales - to automate the docker configuration and start
python on whales was chosen over:
Since it works on the command line interface and not via the SDK and supports docker compose.
How it works
The core concept is a "MediawikiCluster" that is a set of docker applications that run MediaWiki instances with varying versions of MediaWiki and the underlying database.
Installation
Prerequisites
Docker on Windows, Mac or Linux suppporting "docker compose up" using the Docker composer V2 CLI
Compatibility test
docker 2>&1 | grep compose
compose* Docker Compose (Docker Inc., 2.0.0-beta.1)
Installation of Docker Compose V2 CLI
see https://github.com/WolfgangFahl/pymediawikidocker/blob/main/scripts/install
docker --version
cliplugindir=$HOME/.docker/cli-plugins
mkdir -p $cliplugindir
cd $cliplugindir
wget https://github.com/docker/compose-cli/releases/download/v2.0.0-beta.3/docker-compose-linux-amd64
mv docker-compose-linux-amd64 docker-compose
chmod +x docker-compose
# check result
ls -l
docker
docker 2>&1 | grep compose
via pip
pip install pymediawikidocker
# alternatively if your pip is not a python3 pip
pip3 install pymediawikidocker
upgrade
pip install pymediawikidocker -U
# alternatively if your pip is not a python3 pip
pip3 install pymediawikidocker -U
Usage
mwcluster -h
usage: mwcluster [-h] [-d] [-V] [-vl [VERSIONS ...]] [-wl [WIKIIDLIST ...]] [-el [EXTENSIONNAMELIST ...]] [-ej EXTENSIONJSONFILE]
[-u USER] [-p PASSWORD] [-bp BASEPORT] [-sp SQLPORT] [-smw SMWVERSION] [-mv MARIADBVERSION] [-f] [--logo LOGO]
mwcluster Created by Wolfgang Fahl on 2021-06-21. Copyright 2021 Wolfgang Fahl. All rights reserved. Licensed under the Apache License
2.0 http://www.apache.org/licenses/LICENSE-2.0 Distributed on an "AS IS" basis without warranties or conditions of any kind, either
express or implied.
optional arguments:
-h, --help show this help message and exit
-d, --debug set debug level [default: False]
-V, --version show program's version number and exit
-vl [VERSIONS ...], --versionList [VERSIONS ...]
mediawiki versions to create docker applications for [default: ['1.27.7', '1.31.15', '1.35.3', '1.36.1']]
-wl [WIKIIDLIST ...], --wikiIdList [WIKIIDLIST ...]
list of wikiIDs to be used for for py-3rdparty-mediawiki wikiuser quick access (default: None)
-el [EXTENSIONNAMELIST ...], --extensionList [EXTENSIONNAMELIST ...]
list of extensions to be installed [default: ['Admin Links', 'Header Tabs', 'SyntaxHighlight', 'Variables']]
-ej EXTENSIONJSONFILE, --extensionJson EXTENSIONJSONFILE
additional extension descriptions default: None (default: None)
-u USER, --user USER set username of initial user with sysop rights [default: Sysop]
-p PASSWORD, --password PASSWORD
set password for initial user [default: sysop-1234!]
-bp BASEPORT, --basePort BASEPORT
set how base html port 80 to be exposed - incrementing by one for each version [default: 9080]
-sp SQLPORT, --sqlBasePort SQLPORT
set base mySql port 3306 to be exposed - incrementing by one for each version [default: 9306]
-smw SMWVERSION, --smwVersion SMWVERSION
set SemanticMediaWiki Version to be installed default is None - no installation of SMW (default: None)
-mv MARIADBVERSION, --mariaDBVersion MARIADBVERSION
mariaDB Version to be installed [default: 10.5]
-f, --forceRebuild shall the applications rebuild be forced (with stop and remove of existing containers) (default: False)
--logo LOGO set Logo [default: $wgResourceBasePath/resources/assets/wiki.png]
Examples
Default settings
Starts a cluster with versions 1.27.7, 1.31.14 and 1.35.2 using MariaDB 10.5
mwcluster -f
starting mediawiki 1.27.7 docker application ...
stopping and removing container mw1_27_7_db_1
stopping and removing container mw1_27_7_mw_1
[+] Building 0.5s (12/12) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.10kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/mediawiki:1.27.7 0.0s
=> [1/7] FROM docker.io/library/mediawiki:1.27.7 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 61.98kB 0.0s
=> CACHED [2/7] RUN apt-get -y update && apt-get -y --no-install-recommends --fix-missing install maria 0.0s
=> [3/7] COPY LocalSettings.php /var/www/html 0.0s
=> [4/7] COPY phpinfo.php /var/www/html 0.0s
=> [5/7] COPY wiki.sql /tmp 0.0s
=> [6/7] COPY initdb.sh /tmp 0.0s
=> [7/7] RUN chmod +x /tmp/initdb.sh 0.2s
=> exporting to image 0.1s
=> => exporting layers 0.0s
=> => writing image sha256:acee6c878b9a8dde738b38c8d73812ae92b175bd1ddefe1f2493d26af7e7bbdb 0.0s
=> => naming to docker.io/library/mw1_27_7_mw 0.0s
Initializing MediaWiki SQL tables
Trying DB-Connection to wiki on localhost port 9306 with user wikiuser with max 6 tries and 10s timeout per try - initial sleep 2.5s
Connection to wiki on localhost with user wikiuser established database returns: ('wiki',)
starting mediawiki 1.31.14 docker application ...
stopping and removing container mw1_31_14_db_1
stopping and removing container mw1_31_14_mw_1
[+] Building 0.5s (12/12) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.10kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/mediawiki:1.31.14 0.0s
=> [1/7] FROM docker.io/library/mediawiki:1.31.14 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 72.20kB 0.0s
=> CACHED [2/7] RUN apt-get -y update && apt-get -y --no-install-recommends --fix-missing install maria 0.0s
=> [3/7] COPY LocalSettings.php /var/www/html 0.0s
=> [4/7] COPY phpinfo.php /var/www/html 0.0s
=> [5/7] COPY wiki.sql /tmp 0.0s
=> [6/7] COPY initdb.sh /tmp 0.0s
=> [7/7] RUN chmod +x /tmp/initdb.sh 0.2s
=> exporting to image 0.1s
=> => exporting layers 0.0s
=> => writing image sha256:a478ee76ef95f5c6eafc88828058a1d67f4033fcb110abc1b0085729d682c613 0.0s
=> => naming to docker.io/library/mw1_31_14_mw 0.0s
Initializing MediaWiki SQL tables
Trying DB-Connection to wiki on localhost port 9307 with user wikiuser with max 6 tries and 10s timeout per try - initial sleep 2.5s
Connection to wiki on localhost with user wikiuser established database returns: ('wiki',)
starting mediawiki 1.35.2 docker application ...
stopping and removing container mw1_35_2_db_1
stopping and removing container mw1_35_2_mw_1
[+] Building 0.5s (12/12) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.10kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/mediawiki:1.35.2 0.0s
=> [1/7] FROM docker.io/library/mediawiki:1.35.2 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 69.71kB 0.0s
=> CACHED [2/7] RUN apt-get -y update && apt-get -y --no-install-recommends --fix-missing install maria 0.0s
=> [3/7] COPY LocalSettings.php /var/www/html 0.0s
=> [4/7] COPY phpinfo.php /var/www/html 0.0s
=> [5/7] COPY wiki.sql /tmp 0.0s
=> [6/7] COPY initdb.sh /tmp 0.0s
=> [7/7] RUN chmod +x /tmp/initdb.sh 0.2s
=> exporting to image 0.1s
=> => exporting layers 0.1s
=> => writing image sha256:a90cfd37ad0fc08329db5b499437f2539364c6b9d0c4a40cfbadcb9b51debd29 0.0s
=> => naming to docker.io/library/mw1_35_2_mw 0.0s
Initializing MediaWiki SQL tables
Trying DB-Connection to wiki on localhost port 9308 with user wikiuser with max 6 tries and 10s timeout per try - initial sleep 2.5s
Connection to wiki on localhost with user wikiuser established database returns: ('wiki',)
Mediawiki applications will be available at
Single 1.27.5
python mwdocker/mwcluster.py --forceRebuild --versionList 1.27.5 --basePort 8480 --sqlBasePort 10306 --mariaDBVersion 10.5
Mediawiki application will be available at
Single 1.35
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.4 LTS
Release: 20.04
Codename: focal
follow procedure in https://nextgentips.com/2022/05/06/how-to-install-docker-compose-v2-on-ubuntu-22-04/
docker compose version
Docker Compose version v2.5.0
sudo apt install docker-compose
docker-compose version
docker-compose version 1.25.0, build unknown
docker-py version: 4.1.0
CPython version: 3.8.10
OpenSSL version: OpenSSL 1.1.1f 31 Mar 2020
which docker-compose
/usr/bin/docker-compose
create docker compose files
python mwdocker/mwcluster.py --forceRebuild --versionList 1.35.5 --basePort 8480 --sqlBasePort 10306 --mariaDBVersion 10.5
debug docker compose results - ports and other configuration parts
cd ~/.pymediawikidocker/mw1_35_5
docker-compose up
# in another terminal session:
docker ps
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
682b81b36fc7 mw1_35_5_mw "docker-php-entrypoi…" 2 minutes ago Up 2 minutes 0.0.0.0:8480->80/tcp, :::8480->80/tcp mw1_35_5_mw_1
3017fbc569d3 mariadb:10.5 "docker-entrypoint.s…" 2 minutes ago Up 2 minutes 0.0.0.0:10306->3306/tcp, :::10306->3306/tcp mw1_35_5_db_1
Semantic MediaWiki cluster with WikiUser creation
mwcluster -f -smw 3.2.3 -vl 1.31.14 1.35.2 1.36.0 -wl mw31test mw35test mw36test
creating docker applications for mediawiki versions ['1.31.14', '1.35.2', '1.36.0']
Debugging
Docker files
You'll find the files created by pymediawikidocker in ".pymediawikidocker" in your homedirectory.
pwd
/Users/***/.pymediawikidocker/mw1_35_2
ls -l
total 176
-rw-r--r-- 1 wf staff 1057 22 Jun 07:03 Dockerfile
-rw-r--r-- 1 wf staff 4580 22 Jun 07:03 LocalSettings.php
-rw-r--r-- 1 wf staff 1161 22 Jun 07:03 docker-compose.yml
-rw-r--r-- 1 wf staff 317 22 Jun 07:03 initdb.sh
-rw-r--r-- 1 wf staff 125 22 Jun 07:03 phpinfo.php
-rw-r--r-- 1 wf staff 64515 22 Jun 07:03 wiki.sql
Bash into docker containers
# Mediawiki container
docker exec -it mw1_36_0_mw_1 /bin/bash
# MariaDB container
docker exec -it mw1_36_0_db_1 /bin/bash
No space left on device issue
e.g. when starting mariadb container see https://stackoverflow.com/questions/37645879/how-can-i-fix-docker-mac-no-space-left-on-device-error
Wiki Backup
The passwords are available from the docker files e.g.
grep PASS docker-compose.yml
MYSQL_PASSWORD: "TnbKf2zGRmZPjUvz7B"
MYSQL_ROOT_PASSWORD: "QWehnR9yLMY7YAb9pw"
In a docker bash a dump can be created
mysqldump wiki -u wikiuser --password=TnbKf2zGRmZPjUvz7B > /tmp/wiki.sql
and then copied to your local machine
docker cp mw1_36_0_db_1:/tmp/wiki.sql /tmp
Stop and Remove all running containers
These commands will stop and remove all running docker containers (not only the ones created pymediawikidocker/the mwcluster command)
docker stop $(docker ps -aq)
docker rm $(docker ps -aq)