Pymediawikidocker
Revision as of 11:53, 23 June 2021 by Wf (talk | contribs) (→Semantic MediaWiki Cluster with WikiUser)
OsProject
| OsProject | |
|---|---|
 edit
| |
| id | pymediawikidocker |
| state | |
| owner | WolfgangFahl |
| title | Python controlled mediawiki docker image installation |
| url | https://github.com/WolfgangFahl/pymediawikidocker |
| version | 0.0.13 |
| description | |
| date | 2021-06-23 |
| since | |
| until | |
Motivation
Official Mediawiki docker images are available for the different Mediawiki versions.
These images need additional infrastructure to create useable Mediawiki environments. The goal of this project is to semi-automatically create such environments and test the functionality. Python was choosen as a development environment for the libraries that are needed and available to perform the tasks:
- https://github.com/gabrieldemarmiesse/python-on-whales - to automate the docker configuration and start
python on whales was chosen over:
Since it works on the command line interface and not via the SDK and supports docker compose.
How it works
The core concept is a "MediawikiCluster" that is a set of docker applications that run MediaWiki instances with varying versions of MediaWiki and the underlying database.
Installation
Prerequisites
Docker on Windows, Mac or Linux suppporting "docker compose up" using the Docker composer V2 CLI
Compatibility test
docker 2>&1 | grep compose
compose* Docker Compose (Docker Inc., 2.0.0-beta.1)
Installation of Docker Compose V2 CLI
see https://github.com/WolfgangFahl/pymediawikidocker/blob/main/scripts/install
docker --version
cliplugindir=$HOME/.docker/cli-plugins
mkdir -p $cliplugindir
cd $cliplugindir
wget https://github.com/docker/compose-cli/releases/download/v2.0.0-beta.3/docker-compose-linux-amd64
mv docker-compose-linux-amd64 docker-compose
chmod +x docker-compose
# check result
ls -l
docker
docker 2>&1 | grep compose
via pip
pip install pymediawikidocker
# alternatively if your pip is not a python3 pip
pip3 install pymediawikidocker
upgrade
pip install pymediawikidocker -U
# alternatively if your pip is not a python3 pip
pip3 install pymediawikidocker -U
Usage
usage: mwcluster [-h] [-d] [-V] [-vl [VERSIONS ...]] [-wl [WIKIIDLIST ...]]
[-u USER] [-p PASSWORD] [-bp BASEPORT] [-sp SQLPORT]
[-smw SMWVERSION] [-mv MARIADBVERSION] [-f]
mwcluster Created by Wolfgang Fahl on 2021-06-21. Copyright 2021 Wolfgang
Fahl. All rights reserved. Licensed under the Apache License 2.0
http://www.apache.org/licenses/LICENSE-2.0 Distributed on an "AS IS" basis
without warranties or conditions of any kind, either express or implied.
optional arguments:
-h, --help show this help message and exit
-d, --debug set debug level [default: False]
-V, --version show program s version number and exit
-vl [VERSIONS ...], --versionList [VERSIONS ...]
mediawiki versions to create docker applications for
[default: ['1.27.7', '1.31.14', '1.35.2', '1.36.0']]
-wl [WIKIIDLIST ...], --wikiIdList [WIKIIDLIST ...]
list of wikiIDs to be used for for py-3rdparty-
mediawiki wikiuser quick access (default: None)
-u USER, --user USER set username of initial user with sysop rights
[default: Sysop]
-p PASSWORD, --password PASSWORD
set password for initial user [default: sysop-1234!]
-bp BASEPORT, --basePort BASEPORT
set how base html port 80 to be exposed - incrementing
by one for each version [default: 9080]
-sp SQLPORT, --sqlBasePort SQLPORT
set base mySql port 3306 to be exposed - incrementing
by one for each version [default: 9306]
-smw SMWVERSION, --smwVersion SMWVERSION
set SemanticMediaWiki Version to be installed default
is None - no installation of SMW (default: None)
-mv MARIADBVERSION, --mariaDBVersion MARIADBVERSION
mariaDB Version to be installed [default: 10.5]
-f, --forceRebuild shall the applications rebuild be forced (with stop
and remove of existing containers) (default: False)
Examples
Default settings
Starts a cluster with versions 1.27.7, 1.31.14 and 1.35.2 using MariaDB 10.5
mwcluster -f
starting mediawiki 1.27.7 docker application ...
stopping and removing container mw1_27_7_db_1
stopping and removing container mw1_27_7_mw_1
[+] Building 0.5s (12/12) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.10kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/mediawiki:1.27.7 0.0s
=> [1/7] FROM docker.io/library/mediawiki:1.27.7 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 61.98kB 0.0s
=> CACHED [2/7] RUN apt-get -y update && apt-get -y --no-install-recommends --fix-missing install maria 0.0s
=> [3/7] COPY LocalSettings.php /var/www/html 0.0s
=> [4/7] COPY phpinfo.php /var/www/html 0.0s
=> [5/7] COPY wiki.sql /tmp 0.0s
=> [6/7] COPY initdb.sh /tmp 0.0s
=> [7/7] RUN chmod +x /tmp/initdb.sh 0.2s
=> exporting to image 0.1s
=> => exporting layers 0.0s
=> => writing image sha256:acee6c878b9a8dde738b38c8d73812ae92b175bd1ddefe1f2493d26af7e7bbdb 0.0s
=> => naming to docker.io/library/mw1_27_7_mw 0.0s
Initializing MediaWiki SQL tables
Trying DB-Connection to wiki on localhost port 9306 with user wikiuser with max 6 tries and 10s timeout per try - initial sleep 2.5s
Connection to wiki on localhost with user wikiuser established database returns: ('wiki',)
starting mediawiki 1.31.14 docker application ...
stopping and removing container mw1_31_14_db_1
stopping and removing container mw1_31_14_mw_1
[+] Building 0.5s (12/12) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.10kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/mediawiki:1.31.14 0.0s
=> [1/7] FROM docker.io/library/mediawiki:1.31.14 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 72.20kB 0.0s
=> CACHED [2/7] RUN apt-get -y update && apt-get -y --no-install-recommends --fix-missing install maria 0.0s
=> [3/7] COPY LocalSettings.php /var/www/html 0.0s
=> [4/7] COPY phpinfo.php /var/www/html 0.0s
=> [5/7] COPY wiki.sql /tmp 0.0s
=> [6/7] COPY initdb.sh /tmp 0.0s
=> [7/7] RUN chmod +x /tmp/initdb.sh 0.2s
=> exporting to image 0.1s
=> => exporting layers 0.0s
=> => writing image sha256:a478ee76ef95f5c6eafc88828058a1d67f4033fcb110abc1b0085729d682c613 0.0s
=> => naming to docker.io/library/mw1_31_14_mw 0.0s
Initializing MediaWiki SQL tables
Trying DB-Connection to wiki on localhost port 9307 with user wikiuser with max 6 tries and 10s timeout per try - initial sleep 2.5s
Connection to wiki on localhost with user wikiuser established database returns: ('wiki',)
starting mediawiki 1.35.2 docker application ...
stopping and removing container mw1_35_2_db_1
stopping and removing container mw1_35_2_mw_1
[+] Building 0.5s (12/12) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.10kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/mediawiki:1.35.2 0.0s
=> [1/7] FROM docker.io/library/mediawiki:1.35.2 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 69.71kB 0.0s
=> CACHED [2/7] RUN apt-get -y update && apt-get -y --no-install-recommends --fix-missing install maria 0.0s
=> [3/7] COPY LocalSettings.php /var/www/html 0.0s
=> [4/7] COPY phpinfo.php /var/www/html 0.0s
=> [5/7] COPY wiki.sql /tmp 0.0s
=> [6/7] COPY initdb.sh /tmp 0.0s
=> [7/7] RUN chmod +x /tmp/initdb.sh 0.2s
=> exporting to image 0.1s
=> => exporting layers 0.1s
=> => writing image sha256:a90cfd37ad0fc08329db5b499437f2539364c6b9d0c4a40cfbadcb9b51debd29 0.0s
=> => naming to docker.io/library/mw1_35_2_mw 0.0s
Initializing MediaWiki SQL tables
Trying DB-Connection to wiki on localhost port 9308 with user wikiuser with max 6 tries and 10s timeout per try - initial sleep 2.5s
Connection to wiki on localhost with user wikiuser established database returns: ('wiki',)
Mediawiki applications will be available at
Single 1.27.5
python mwdocker/mwcluster.py --forceRebuild --versionList 1.27.5 --basePort 8480 --sqlBasePort 10306 --mariaDBVersion 10.5
Mediawiki application will be available at
Semantic MediaWiki cluster with WikiUser creation
mwcluster -f -smw 3.2.3 -wl mw27test mw31test mw35test mw36test
creating docker applications for mediawiki versions ['1.27.7', '1.31.14', '1.35.2', '1.36.0']
Debugging
Docker files
You'll find the files created by pymediawikidocker in ".pymediawikidocker" in your homedirectory.
pwd
/Users/***/.pymediawikidocker/mw1_35_2
ls -l
total 176
-rw-r--r-- 1 wf staff 1057 22 Jun 07:03 Dockerfile
-rw-r--r-- 1 wf staff 4580 22 Jun 07:03 LocalSettings.php
-rw-r--r-- 1 wf staff 1161 22 Jun 07:03 docker-compose.yml
-rw-r--r-- 1 wf staff 317 22 Jun 07:03 initdb.sh
-rw-r--r-- 1 wf staff 125 22 Jun 07:03 phpinfo.php
-rw-r--r-- 1 wf staff 64515 22 Jun 07:03 wiki.sql
Bash into docker containers
# Mediawiki container
docker exec -it mw1_36_0_mw_1 /bin/bash
# MariaDB container
docker exec -it mw1_36_0_db_1 /bin/bash
Wiki Backup
The passwords are available from the docker files e.g.
grep PASS docker-compose.yml
MYSQL_PASSWORD: "TnbKf2zGRmZPjUvz7B"
MYSQL_ROOT_PASSWORD: "QWehnR9yLMY7YAb9pw"
In a docker bash a dump can be created
mysqldump wiki -u wikiuser --password=TnbKf2zGRmZPjUvz7B > /tmp/wiki.sql
and then copied to your local machine
docker cp mw1_36_0_db_1:/tmp/wiki.sql /tmp
Stop and Remove all running containers
These commands will stop and remove all running docker containers (not only the ones created pymediawikidocker/the mwcluster command)
docker stop $(docker ps -aq)
docker rm $(docker ps -aq)