Pymediawikidocker

OsProject

OsProject
edit
id	pymediawikidocker
state
owner	WolfgangFahl
title	Python controlled mediawiki docker image installation
url	https://github.com/WolfgangFahl/pymediawikidocker
version	0.4.0
description
date	2022-10-23
since
until

Motivation

Official Mediawiki docker images are available for the different Mediawiki versions.

These images need additional infrastructure to create useable Mediawiki environments. The goal of this project is to semi-automatically create such environments and test the functionality. Python was choosen as a development environment for the libraries that are needed and available to perform the tasks:

• https://github.com/gabrieldemarmiesse/python-on-whales - to automate the docker configuration and start

python on whales was chosen over:

• https://pypi.org/project/docker/

Since it works on the command line interface and not via the SDK and supports docker compose.

How it works

The core concept is a "MediawikiCluster" that is a set of docker applications that run MediaWiki instances with varying versions of MediaWiki and the underlying database.

Installation

Prerequisites

Docker on Windows, Mac or Linux suppporting "docker compose up" using the Docker composer V2 CLI

Ubuntu 22.04 LTS installation

see https://docs.docker.com/engine/install/ubuntu/ on how to setup the docker repositorysudo

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
docker --version
docker run hello-world
Docker version 20.10.20, build 9fdeb9c

Compatibility test

# test on a Mac OS machine 2022-07-02
docker 2>&1 | grep compose
  compose*    Docker Compose (Docker Inc., v2.4.1)
# test on an Ubuntu machine 2022-10-23
docker 2>&1 | grep compose
  compose*    Docker Compose (Docker Inc., v2.12.0)

Installation of Docker Compose V2 CLI

see https://github.com/WolfgangFahl/pymediawikidocker/blob/main/scripts/install

   docker --version
   cliplugindir=$HOME/.docker/cli-plugins
   mkdir -p $cliplugindir
   cd $cliplugindir
   wget https://github.com/docker/compose-cli/releases/download/v2.0.0-beta.3/docker-compose-linux-amd64
   mv  docker-compose-linux-amd64 docker-compose

   chmod +x docker-compose
# check result
   ls -l
   docker
   docker  2>&1 | grep compose

via pip

pip install pymediawikidocker
# alternatively if your pip is not a python3 pip
pip3 install pymediawikidocker

upgrade

pip install pymediawikidocker -U
# alternatively if your pip is not a python3 pip
pip3 install pymediawikidocker -U

Usage

mwcluster -h
usage: mwcluster [-h] [-d] [-V] [-vl [VERSIONS ...]] [-wl [WIKIIDLIST ...]] [-el [EXTENSIONNAMELIST ...]] [-ej EXTENSIONJSONFILE]
                 [-u USER] [-p PASSWORD] [-bp BASEPORT] [-sp SQLPORT] [-smw SMWVERSION] [-mv MARIADBVERSION] [-f] [--logo LOGO]

mwcluster Created by Wolfgang Fahl on 2021-06-21. Copyright 2021 Wolfgang Fahl. All rights reserved. Licensed under the Apache License
2.0 http://www.apache.org/licenses/LICENSE-2.0 Distributed on an "AS IS" basis without warranties or conditions of any kind, either
express or implied.

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           set debug level [default: False]
  -V, --version         show program's version number and exit
  -vl [VERSIONS ...], --versionList [VERSIONS ...]
                        mediawiki versions to create docker applications for [default: ['1.27.7', '1.31.15', '1.35.3', '1.36.1']]
  -wl [WIKIIDLIST ...], --wikiIdList [WIKIIDLIST ...]
                        list of wikiIDs to be used for for py-3rdparty-mediawiki wikiuser quick access (default: None)
  -el [EXTENSIONNAMELIST ...], --extensionList [EXTENSIONNAMELIST ...]
                        list of extensions to be installed [default: ['Admin Links', 'Header Tabs', 'SyntaxHighlight', 'Variables']]
  -ej EXTENSIONJSONFILE, --extensionJson EXTENSIONJSONFILE
                        additional extension descriptions default: None (default: None)
  -u USER, --user USER  set username of initial user with sysop rights [default: Sysop]
  -p PASSWORD, --password PASSWORD
                        set password for initial user [default: sysop-1234!]
  -bp BASEPORT, --basePort BASEPORT
                        set how base html port 80 to be exposed - incrementing by one for each version [default: 9080]
  -sp SQLPORT, --sqlBasePort SQLPORT
                        set base mySql port 3306 to be exposed - incrementing by one for each version [default: 9306]
  -smw SMWVERSION, --smwVersion SMWVERSION
                        set SemanticMediaWiki Version to be installed default is None - no installation of SMW (default: None)
  -mv MARIADBVERSION, --mariaDBVersion MARIADBVERSION
                        mariaDB Version to be installed [default: 10.5]
  -f, --forceRebuild    shall the applications rebuild be forced (with stop and remove of existing containers) (default: False)
  --logo LOGO           set Logo [default: $wgResourceBasePath/resources/assets/wiki.png]

Examples

Default settings

Starts a cluster with versions 1.27.7, 1.31.16, 1.35.7, 1.37.3 and 1.38.2 using MariaDB 10.8

mwcluster -f
creating docker applications for mediawiki versions ['1.27.7', '1.31.16', '1.35.7', '1.37.3', '1.38.2']
starting mediawiki 1.27.7 docker application ...
..
Initializing MediaWiki SQL tables
Trying DB-Connection to wiki on localhost port 9308 with user wikiuser with max 6 tries and 10s timeout per try - initial sleep 2.5s
Connection to wiki on localhost with user wikiuser established database returns: ('wiki',)

Mediawiki applications will be available at

1. http://localhost:9080
2. http://localhost:9081
3. http://localhost:9082
4. http://localhost:9083
5. http://localhost:9084

Single 1.27.5

python mwdocker/mwcluster.py --forceRebuild --versionList 1.27.5 --basePort 8480 --sqlBasePort 10306 --mariaDBVersion 10.5

Mediawiki application will be available at

1. http://localhost:8480

Single 1.35 with Semantic Mediawiki

lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.1 LTS
Release:	22.04
Codename:	jammy

docker compose version
Docker Compose version v2.12.0

sudo apt  install docker-compose
docker-compose version
docker-compose version 1.29.2, build unknown
docker-py version: 5.0.3
CPython version: 3.10.6
OpenSSL version: OpenSSL 3.0.2 15 Mar 2022
which docker-compose
/usr/bin/docker-compose

create docker compose files

mwcluster --forceRebuild --versionList 1.35.7 --basePort 8480 --sqlBasePort 10306 --mariaDBVersion 10.5 -smw 3.2.3

debug docker compose results - ports and other configuration parts

cd ~/.pymediawikidocker/mw1_35_7
docker-compose up
# in another terminal session:
docker ps
CONTAINER ID   IMAGE          COMMAND                  CREATED         STATUS         PORTS                     NAMES
15281793ba90   mw1_35_7_mw    "docker-php-entrypoi…"   2 minutes ago   Up 2 minutes   0.0.0.0:8480->80/tcp      mw1_35_7-mw-1
2e6decc79be0   mariadb:10.5   "docker-entrypoint.s…"   2 minutes ago   Up 2 minutes   0.0.0.0:10306->3306/tcp   mw1_35_7-db-1

open bash in docker container

docker exec -it mw1_35_7-mw-1 /bin/bash
/tmp/installExtensions.sh

Semantic MediaWiki cluster with WikiUser creation

mwcluster -f -smw 3.2.3 -vl 1.31.14 1.35.2 1.36.0 -wl  mw31test mw35test mw36test
creating docker applications for mediawiki versions ['1.31.14', '1.35.2', '1.36.0']

Debugging

Docker files

You'll find the files created by pymediawikidocker in ".pymediawikidocker" in your homedirectory.

pwd
/Users/***/.pymediawikidocker/mw1_35_2
ls -l
total 176
-rw-r--r--  1 wf  staff   1057 22 Jun 07:03 Dockerfile
-rw-r--r--  1 wf  staff   4580 22 Jun 07:03 LocalSettings.php
-rw-r--r--  1 wf  staff   1161 22 Jun 07:03 docker-compose.yml
-rw-r--r--  1 wf  staff    317 22 Jun 07:03 initdb.sh
-rw-r--r--  1 wf  staff    125 22 Jun 07:03 phpinfo.php
-rw-r--r--  1 wf  staff  64515 22 Jun 07:03 wiki.sql

Bash into docker containers

# Mediawiki container
docker exec -it mw1_36_0_mw_1 /bin/bash
# MariaDB container
docker exec -it mw1_36_0_db_1 /bin/bash

No space left on device issue

e.g. when starting mariadb container see https://stackoverflow.com/questions/37645879/how-can-i-fix-docker-mac-no-space-left-on-device-error

Wiki Backup

The passwords are available from the docker files e.g.

grep PASS docker-compose.yml 
      MYSQL_PASSWORD: "TnbKf2zGRmZPjUvz7B"
      MYSQL_ROOT_PASSWORD: "QWehnR9yLMY7YAb9pw"

In a docker bash a dump can be created

mysqldump wiki -u wikiuser --password=TnbKf2zGRmZPjUvz7B > /tmp/wiki.sql

and then copied to your local machine

docker cp mw1_36_0_db_1:/tmp/wiki.sql /tmp

Stop and Remove all running containers

These commands will stop and remove all running docker containers (not only the ones created pymediawikidocker/the mwcluster command)

docker stop $(docker ps -aq)
docker rm $(docker ps -aq)

Links

• Wikidata entry for Mediawiki software