Difference between revisions of "Pymediawikidocker"
Line 244: | Line 244: | ||
</source> | </source> | ||
== Bash into docker containers == | == Bash into docker containers == | ||
+ | Please note that the naming convention for containers is environment specific e.g. mw1_27_7-mw-1 might have the name mw1_27_7_mw_1 | ||
<source lang='bash' highlight='2,4'> | <source lang='bash' highlight='2,4'> | ||
# Mediawiki container | # Mediawiki container |
Revision as of 08:29, 25 October 2022
OsProject
OsProject | |
---|---|
id | pymediawikidocker |
state | |
owner | WolfgangFahl |
title | Python controlled mediawiki docker image installation |
url | https://github.com/WolfgangFahl/pymediawikidocker |
version | 0.4.1 |
description | |
date | 2022-10-24 |
since | |
until |
Motivation
Official Mediawiki docker images are available for the different Mediawiki versions.
These images need additional infrastructure to create useable Mediawiki environments. The goal of this project is to semi-automatically create such environments and test the functionality. Python was choosen as a development environment for the libraries that are needed and available to perform the tasks:
- https://github.com/gabrieldemarmiesse/python-on-whales - to automate the docker configuration and start
python on whales was chosen over:
Since it works on the command line interface and not via the SDK and supports docker compose.
How it works
The core concept is a "MediawikiCluster" that is a set of docker applications that run MediaWiki instances with varying versions of MediaWiki and the underlying database.
![](/images/wiki/diagrams/Diagrams_a9d182e63271ff006d8d7402503e06e5.png)
Installation
Prerequisites
Docker on Windows, Mac or Linux suppporting "docker compose up" using the Docker composer V2 CLI
Ubuntu 22.04 LTS installation
see https://docs.docker.com/engine/install/ubuntu/ on how to setup the docker repositorysudo
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
docker --version
docker run hello-world
Docker version 20.10.20, build 9fdeb9c
Compatibility test
# test on a Mac OS machine 2022-07-02
docker 2>&1 | grep compose
compose* Docker Compose (Docker Inc., v2.4.1)
# test on an Ubuntu machine 2022-10-23
docker 2>&1 | grep compose
compose* Docker Compose (Docker Inc., v2.12.0)
Installation of Docker Compose V2 CLI
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.1 LTS
Release: 22.04
Codename: jammy
docker compose version
Docker Compose version v2.12.0
sudo apt install docker-compose
docker-compose version
docker-compose version 1.29.2, build unknown
docker-py version: 5.0.3
CPython version: 3.10.6
OpenSSL version: OpenSSL 3.0.2 15 Mar 2022
which docker-compose
/usr/bin/docker-compose
pymediawikidocker installation via pip
pip install pymediawikidocker
# alternatively if your pip is not a python3 pip
pip3 install pymediawikidocker
upgrade
pip install pymediawikidocker -U
# alternatively if your pip is not a python3 pip
pip3 install pymediawikidocker -U
check version
mwcluster -V
mwcluster v0.4.2 (2022-10-25)
Usage
mwcluster -h
usage: mwcluster [-h] [-d] [-V] [-vl [VERSIONS ...]] [-wl [WIKIIDLIST ...]]
[-el [EXTENSIONNAMELIST ...]] [-ej EXTENSIONJSONFILE] [-u USER]
[-p PASSWORD] [-bp BASEPORT] [-sp SQLPORT] [-smw SMWVERSION]
[-mv MARIADBVERSION] [-f] [--logo LOGO]
mwcluster Created by Wolfgang Fahl on 2021-06-21. Copyright 2021 Wolfgang Fahl. All rights
reserved. Licensed under the Apache License 2.0 http://www.apache.org/licenses/LICENSE-2.0
Distributed on an "AS IS" basis without warranties or conditions of any kind, either express
or implied.
options:
-h, --help show this help message and exit
-d, --debug set debug level [default: False]
-V, --version show program's version number and exit
-vl [VERSIONS ...], --versionList [VERSIONS ...]
mediawiki versions to create docker applications for [default:
['1.27.7', '1.31.16', '1.35.8', '1.37.6', '1.38.4']]
-wl [WIKIIDLIST ...], --wikiIdList [WIKIIDLIST ...]
list of wikiIDs to be used for for py-3rdparty-mediawiki wikiuser
quick access (default: None)
-el [EXTENSIONNAMELIST ...], --extensionList [EXTENSIONNAMELIST ...]
list of extensions to be installed [default: ['Admin Links', 'Header
Tabs', 'SyntaxHighlight', 'Variables']]
-ej EXTENSIONJSONFILE, --extensionJson EXTENSIONJSONFILE
additional extension descriptions default: None (default: None)
-u USER, --user USER set username of initial user with sysop rights [default: Sysop]
-p PASSWORD, --password PASSWORD
set password for initial user [default: sysop-1234!]
-bp BASEPORT, --basePort BASEPORT
set how base html port 80 to be exposed - incrementing by one for
each version [default: 9080]
-sp SQLPORT, --sqlBasePort SQLPORT
set base mySql port 3306 to be exposed - incrementing by one for
each version [default: 9306]
-smw SMWVERSION, --smwVersion SMWVERSION
set SemanticMediaWiki Version to be installed default is None - no
installation of SMW (default: None)
-mv MARIADBVERSION, --mariaDBVersion MARIADBVERSION
mariaDB Version to be installed [default: 10.9]
-f, --forceRebuild shall the applications rebuild be forced (with stop and remove of
existing containers) (default: False)
--logo LOGO set Logo [default: $wgResourceBasePath/resources/assets/wiki.png]
Examples
Default settings
Starts a cluster with versions 1.27.7, 1.31.16, 1.35.8, 1.37.6 and 1.38.4 using MariaDB 10.9
mwcluster -f
creating docker applications for mediawiki versions ['1.27.7', '1.31.16', '1.35.8', '1.37.6', '1.38.4']
...
starting mediawiki 1.27.7 docker application ...
..
Initializing MediaWiki SQL tables
Trying DB-Connection to wiki on localhost port 9308 with user wikiuser with max 6 tries and 10s timeout per try - initial sleep 2.5s
Connection to wiki on localhost with user wikiuser established database returns: ('wiki',)
Mediawiki applications will be available at
http://localhost:9080 http://localhost:9080/index.php/Special:Version
http://localhost:9081 http://localhost:9081/index.php/Special:Version
http://localhost:9082 http://localhost:9082/index.php/Special:Version
http://localhost:9083 http://localhost:9083/index.php/Special:Version
http://localhost:9084 http://localhost:9084/index.php/Special:Version
Single 1.27.5
# python mwdocker/mwcluster.py can be used instead of mwcluster if you test from your locally cloned environment
mwcluster --forceRebuild --versionList 1.27.5 --basePort 8480 --sqlBasePort 10306 --mariaDBVersion 10.5
Mediawiki application will be available at
http://localhost:8480 http://localhost:8480/index.php/Special:Version
Single 1.35 with Semantic Mediawiki
create docker compose files
mwcluster --forceRebuild --versionList 1.35.7 --basePort 8480 --sqlBasePort 10306 --mariaDBVersion 10.5 -smw 3.2.3
debug docker compose results - ports and other configuration parts
cd ~/.pymediawikidocker/mw1_35_7
docker-compose up
# in another terminal session:
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
15281793ba90 mw1_35_7_mw "docker-php-entrypoi…" 2 minutes ago Up 2 minutes 0.0.0.0:8480->80/tcp mw1_35_7-mw-1
2e6decc79be0 mariadb:10.5 "docker-entrypoint.s…" 2 minutes ago Up 2 minutes 0.0.0.0:10306->3306/tcp mw1_35_7-db-1
open bash in docker container
docker exec -it mw1_35_7-mw-1 /bin/bash
/tmp/installExtensions.sh
Semantic MediaWiki cluster with WikiUser creation
mwcluster -f -smw 3.2.3 -vl 1.31.14 1.35.2 1.36.0 -wl mw31test mw35test mw36test
creating docker applications for mediawiki versions ['1.31.14', '1.35.2', '1.36.0']
Debugging
Docker files
You'll find the files created by pymediawikidocker in ".pymediawikidocker" in your homedirectory.
pwd
/Users/***/.pymediawikidocker/mw1_35_2
ls -l
total 176
-rw-r--r-- 1 wf staff 1057 22 Jun 07:03 Dockerfile
-rw-r--r-- 1 wf staff 4580 22 Jun 07:03 LocalSettings.php
-rw-r--r-- 1 wf staff 1161 22 Jun 07:03 docker-compose.yml
-rw-r--r-- 1 wf staff 317 22 Jun 07:03 initdb.sh
-rw-r--r-- 1 wf staff 125 22 Jun 07:03 phpinfo.php
-rw-r--r-- 1 wf staff 64515 22 Jun 07:03 wiki.sql
Bash into docker containers
Please note that the naming convention for containers is environment specific e.g. mw1_27_7-mw-1 might have the name mw1_27_7_mw_1
# Mediawiki container
docker exec -it mw1_27_7-mw-1 /bin/bash
docker exec -it mw1_36_0_mw_1 /bin/bash
# MariaDB container
docker exec -it mw1_36_0_db_1 /bin/bash
No space left on device issue
e.g. when starting mariadb container see https://stackoverflow.com/questions/37645879/how-can-i-fix-docker-mac-no-space-left-on-device-error
Wiki Backup
The passwords are available from the docker files e.g.
grep PASS docker-compose.yml
MYSQL_PASSWORD: "TnbKf2zGRmZPjUvz7B"
MYSQL_ROOT_PASSWORD: "QWehnR9yLMY7YAb9pw"
In a docker bash a dump can be created
mysqldump wiki -u wikiuser --password=TnbKf2zGRmZPjUvz7B > /tmp/wiki.sql
and then copied to your local machine
docker cp mw1_36_0_db_1:/tmp/wiki.sql /tmp
Stop and Remove all running containers
These commands will stop and remove all running docker containers (not only the ones created pymediawikidocker/the mwcluster command)
# kill/stop all running containers
docker kill $(docker ps -q)
#docker stop $(docker ps -aq)
# remove all docker containers
docker rm $(docker ps -aq)
Recreate Clean docker state
# remove all images
docker rmi $(docker images -q)