Difference between revisions of "Pymediawikidocker"

From BITPlan Wiki
Jump to navigation Jump to search
 
(165 intermediate revisions by the same user not shown)
Line 3: Line 3:
 
{{OsProject
 
{{OsProject
 
|id=pymediawikidocker
 
|id=pymediawikidocker
 +
|state=active
 
|owner=WolfgangFahl
 
|owner=WolfgangFahl
−
|title=Python controlled mediawiki docker image installation
+
|title=pymediawikidocker
 
|url=https://github.com/WolfgangFahl/pymediawikidocker
 
|url=https://github.com/WolfgangFahl/pymediawikidocker
−
|version=0.2.0
+
|version=0.13.4
−
|date=2021-12-19
+
|description=Python controlled mediawiki docker image installation
 +
|date=2024-10-24
 +
|since=2021-06-08
 
|storemode=property
 
|storemode=property
 
}}
 
}}
  
 +
=tickets=
 
= Motivation =
 
= Motivation =
 
[https://hub.docker.com/_/mediawiki Official Mediawiki docker images] are available for the different Mediawiki versions.
 
[https://hub.docker.com/_/mediawiki Official Mediawiki docker images] are available for the different Mediawiki versions.
  
−
These images need additional infrastructure to create useable Mediawiki environments. The goal of this project is to semi-automatically create
+
These images need additional infrastructure to create useable Mediawiki environments.
 +
<source lang='bash'>
 +
docker run --name local-mediawiki -p 8080:80 -d mediawiki
 +
</source>
 +
 
 +
will only install part of a LAMP environment to install Mediawiki and make it available via http://localhost:8080/. E.g. the database needs to be installed separately.
 +
 +
The goal of this project is to semi-automatically create
 
such environments and test the functionality. Python was choosen as a development environment for the [https://github.com/WolfgangFahl/pymediawikidocker/blob/main/requirements.txt libraries that are needed and available] to perform the tasks:
 
such environments and test the functionality. Python was choosen as a development environment for the [https://github.com/WolfgangFahl/pymediawikidocker/blob/main/requirements.txt libraries that are needed and available] to perform the tasks:
 
* https://github.com/gabrieldemarmiesse/python-on-whales - to automate the docker configuration and start
 
* https://github.com/gabrieldemarmiesse/python-on-whales - to automate the docker configuration and start
Line 23: Line 34:
 
= How it works =
 
= How it works =
 
The core concept is a "MediawikiCluster" that is a set of docker applications that run MediaWiki instances with varying versions of MediaWiki and the underlying database.
 
The core concept is a "MediawikiCluster" that is a set of docker applications that run MediaWiki instances with varying versions of MediaWiki and the underlying database.
 +
The graphic belows shows the docker containers which are created for different versions of Mediawiki
  
 
<graphviz>
 
<graphviz>
 +
// generated by /Users/wf/py-workspace/pymediawikidocker/tests/test_install.py on 2024-10-24T19:31:07.227612
 
digraph mwcluster {
 
digraph mwcluster {
−
   mew0 [label="Mediawiki 1.27.7\nport 9080"
+
  rankdir="RL"
−
   mdb0 [ label="MariaDB 10.5\nport 9306"
+
   mew0 [ label="mw-135-mw\nMediaWiki 1.35.13\nport 9080" ]
 +
   mdb0 [ label="mw-135-db\nMariaDB 10.11\nport 9306" ]
 
   subgraph cluster_0{
 
   subgraph cluster_0{
−
     label="mw1_27_7"
+
     label="mw-135"
 
     mew0->mdb0
 
     mew0->mdb0
 
   }
 
   }
−
   mew1 [label="Mediawiki 1.31.16\nport 9081"
+
   mew1 [ label="mw-138-mw\nMediaWiki 1.38.6\nport 9081" ]
−
   mdb1 [ label="MariaDB 10.5\nport 9307"
+
   mdb1 [ label="mw-138-db\nMariaDB 10.11\nport 9307" ]
 
   subgraph cluster_1{
 
   subgraph cluster_1{
−
     label="mw1_31_16"
+
     label="mw-138"
 
     mew1->mdb1
 
     mew1->mdb1
 
   }
 
   }
−
   mew2 [label="Mediawiki 1.35.5\nport 9082"
+
   mew2 [ label="mw-139-mw\nMediaWiki 1.39.10\nport 9082" ]
−
   mdb2 [ label="MariaDB 10.5\nport 9308"
+
   mdb2 [ label="mw-139-db\nMariaDB 10.11\nport 9308" ]
 
   subgraph cluster_2{
 
   subgraph cluster_2{
−
     label="mw1_35_5"
+
     label="mw-139"
 
     mew2->mdb2
 
     mew2->mdb2
 
   }
 
   }
−
   mew3 [label="Mediawiki 1.36.3\nport 9083"
+
   mew3 [ label="mw-140-mw\nMediaWiki 1.40.4\nport 9083" ]
−
   mdb3 [ label="MariaDB 10.5\nport 9309"
+
   mdb3 [ label="mw-140-db\nMariaDB 10.11\nport 9309" ]
 
   subgraph cluster_3{
 
   subgraph cluster_3{
−
     label="mw1_36_3"
+
     label="mw-140"
 
     mew3->mdb3
 
     mew3->mdb3
 
   }
 
   }
−
   mew4 [label="Mediawiki 1.37.1\nport 9084"
+
   mew4 [ label="mw-141-mw\nMediaWiki 1.41.4\nport 9084" ]
−
   mdb4 [ label="MariaDB 10.5\nport 9310"
+
   mdb4 [ label="mw-141-db\nMariaDB 10.11\nport 9310" ]
 
   subgraph cluster_4{
 
   subgraph cluster_4{
−
     label="mw1_37_1"
+
     label="mw-141"
 
     mew4->mdb4
 
     mew4->mdb4
 +
  }
 +
  mew5 [ label="mw-142-mw\nMediaWiki 1.42.3\nport 9085" ]
 +
  mdb5 [ label="mw-142-db\nMariaDB 10.11\nport 9311" ]
 +
  subgraph cluster_5{
 +
    label="mw-142"
 +
    mew5->mdb5
 
   }
 
   }
 
}
 
}
Line 62: Line 82:
 
== Prerequisites ==
 
== Prerequisites ==
 
Docker on Windows, Mac or Linux suppporting "docker compose up" using the Docker composer V2 CLI
 
Docker on Windows, Mac or Linux suppporting "docker compose up" using the Docker composer V2 CLI
 +
=== Ubuntu 22.04 LTS installation ===
 +
see https://docs.docker.com/engine/install/ubuntu/ on how to setup the docker repositorysudo
 +
<source lang='bash' highlight='1-2,4'>
 +
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
 +
docker --version
 +
Docker version 23.0.2, build 569dd73
 +
docker run hello-world
 +
Hello from Docker!
 +
This message shows that your installation appears to be working correctly.
 +
 +
...
 +
</source>
 +
 
=== Compatibility test ===
 
=== Compatibility test ===
 
<source lang='bash' highlight='1'>
 
<source lang='bash' highlight='1'>
 +
# test on a Mac OS machine 2022-07-02
 
docker 2>&1 | grep compose
 
docker 2>&1 | grep compose
 
   compose*    Docker Compose (Docker Inc., v2.4.1)
 
   compose*    Docker Compose (Docker Inc., v2.4.1)
 +
# test on an Ubuntu machine 2023-04-07
 +
docker 2>&1 | grep compose
 +
compose*    Docker Compose (Docker Inc., v2.17.2)
 
</source>
 
</source>
  
 
=== Installation of Docker Compose V2 CLI ===
 
=== Installation of Docker Compose V2 CLI ===
−
see https://github.com/WolfgangFahl/pymediawikidocker/blob/main/scripts/install
+
<source lang='bash' highlight='1'>
 +
lsb_release -a
 +
No LSB modules are available.
 +
Distributor ID: Ubuntu
 +
Description: Ubuntu 22.04.e LTS
 +
Release: 22.04
 +
Codename: jammy
 +
</source>
 +
==== docker-ce from docker repo ====
 
<source lang='bash'>
 
<source lang='bash'>
−
  docker --version
+
# Update existing list of packages
−
  cliplugindir=$HOME/.docker/cli-plugins
+
sudo apt update
−
  mkdir -p $cliplugindir
+
 
−
  cd $cliplugindir
+
# Install prerequisite packages which let apt use packages over HTTPS
−
  wget https://github.com/docker/compose-cli/releases/download/v2.0.0-beta.3/docker-compose-linux-amd64
+
sudo apt install apt-transport-https ca-certificates curl software-properties-common
−
  mv  docker-compose-linux-amd64 docker-compose
+
 
 +
# Add Docker’s official GPG key
 +
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
  
−
  chmod +x docker-compose
+
# Add Docker repository to APT sources
−
# check result
+
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
−
  ls -l
+
 
−
  docker
+
# Update the package database with Docker packages from the newly added repo
−
  docker 2>&1 | grep compose
+
sudo apt update
 +
 
 +
# Make sure you are about to install from the Docker repo instead of the default Ubuntu repo
 +
apt-cache policy docker-ce
 +
 
 +
# Install Docker CE
 +
sudo apt install docker-ce
 +
</source>
 +
==== docker compose (not docker-compose) ====
 +
<source lang='bash' highlight='1'>
 +
docker compose version
 +
Docker Compose version v2.26.1
 +
</source>
 +
<source lang='bash' highlight='1-2,7'>
 +
sudo apt  install docker-compose
 +
docker-compose version
 +
docker-compose version 1.29.2, build unknown
 +
docker-py version: 5.0.3
 +
CPython version: 3.10.12
 +
OpenSSL version: OpenSSL 3.0.2 15 Mar 2022
 +
which docker-compose
 +
/usr/bin/docker-compose
 
</source>
 
</source>
  
−
== via pip ==
+
== pymediawikidocker installation via pip ==
 
<source lang='bash'>
 
<source lang='bash'>
 
pip install pymediawikidocker
 
pip install pymediawikidocker
Line 96: Line 164:
 
# alternatively if your pip is not a python3 pip
 
# alternatively if your pip is not a python3 pip
 
pip3 install pymediawikidocker -U
 
pip3 install pymediawikidocker -U
 +
</source>
 +
=== check version ===
 +
<source lang='bash' highlight='1'>
 +
mwcluster -V
 +
mwcluster (v0.13.0,2024-08-02)
 
</source>
 
</source>
  
 
= Usage =
 
= Usage =
−
<source lang='bash'>
+
<source lang='bash' highlight='1'>
 
mwcluster -h
 
mwcluster -h
−
usage: mwcluster [-h] [-d] [-V] [-vl [VERSIONS ...]] [-wl [WIKIIDLIST ...]] [-el [EXTENSIONNAMELIST ...]] [-ej EXTENSIONJSONFILE]
+
usage: mwcluster [-h] [-cn CONTAINER_NAME] [-d] [-el [EXTENSIONNAMELIST ...]]
−
                 [-u USER] [-p PASSWORD] [-bp BASEPORT] [-sp SQLPORT] [-smw SMWVERSION] [-mv MARIADBVERSION] [-f] [--logo LOGO]
+
                [-ej EXTENSIONJSONFILE] [-f] [-fu] [--host HOST]
 +
                [-dp DOCKER_PATH] [--logo LOGO] [-mv MARIADBVERSION]
 +
                 [--mysqlPassword MYSQLPASSWORD] [-rp] [-p PASSWORD]
 +
                [-pl PASSWORD_LENGTH] [--prefix PREFIX] [--prot PROT]
 +
                [--script_path SCRIPT_PATH] [--url URL] [-sp SQL_PORT]
 +
                [-smw SMW_VERSION] [-u USER] [-q] [-bp BASE_PORT]
 +
                [-vl [VERSIONS ...]] [--about] [--create] [--down] [--check]
 +
                [--list] [-V]
  
−
mwcluster Created by Wolfgang Fahl on 2021-06-21. Copyright 2021 Wolfgang Fahl. All rights reserved. Licensed under the Apache License
+
Copyright 2020-2023 contributors. All rights reserved. Licensed under the
−
2.0 http://www.apache.org/licenses/LICENSE-2.0 Distributed on an "AS IS" basis without warranties or conditions of any kind, either
+
Apache License 2.0 http://www.apache.org/licenses/LICENSE-2.0 Distributed on
−
express or implied.
+
an "AS IS" basis without warranties or conditions of any kind, either express
 +
or implied.
  
−
optional arguments:
+
options:
 
   -h, --help            show this help message and exit
 
   -h, --help            show this help message and exit
−
   -d, --debug          set debug level [default: False]
+
   -cn CONTAINER_NAME, --container_name CONTAINER_NAME
−
  -V, --version        show program's version number and exit
+
                        set container name (only valid and recommended for
−
  -vl [VERSIONS ...], --versionList [VERSIONS ...]
+
                         single version call) (default: mw-139)
−
                         mediawiki versions to create docker applications for [default: ['1.27.7', '1.31.15', '1.35.3', '1.36.1']]
+
   -d, --debug          enable debug mode [default: False]
−
   -wl [WIKIIDLIST ...], --wikiIdList [WIKIIDLIST ...]
 
−
                        list of wikiIDs to be used for for py-3rdparty-mediawiki wikiuser quick access (default: None)
 
 
   -el [EXTENSIONNAMELIST ...], --extensionList [EXTENSIONNAMELIST ...]
 
   -el [EXTENSIONNAMELIST ...], --extensionList [EXTENSIONNAMELIST ...]
−
                         list of extensions to be installed [default: ['Admin Links', 'Header Tabs', 'SyntaxHighlight', 'Variables']]
+
                         list of extensions to be installed [default: ['Admin
 +
                        Links', 'Header Tabs', 'SyntaxHighlight',
 +
                        'Variables']]
 
   -ej EXTENSIONJSONFILE, --extensionJson EXTENSIONJSONFILE
 
   -ej EXTENSIONJSONFILE, --extensionJson EXTENSIONJSONFILE
−
                         additional extension descriptions default: None (default: None)
+
                         additional extension descriptions default: [default:
−
   -u USER, --user USER  set username of initial user with sysop rights [default: Sysop]
+
                        None]
 +
  -f, --forceRebuild    force rebuilding [default: False]
 +
  -fu, --force_user    force overwrite of wikiuser (default: False)
 +
   --host HOST          the host to serve / listen from [default:
 +
                        fix.bitplan.com]
 +
  -dp DOCKER_PATH, --docker_path DOCKER_PATH
 +
                        the base directory to store docker and jinja template
 +
                        files [default: /Users/wf/.pymediawikidocker]
 +
  --logo LOGO          set Logo [default:
 +
                        $wgResourceBasePath/resources/assets/wiki.png]
 +
  -mv MARIADBVERSION, --mariaDBVersion MARIADBVERSION
 +
                        mariaDB Version to be installed [default: 10.11]
 +
  --mysqlPassword MYSQLPASSWORD
 +
                        set sqlRootPassword [default: None] - random password
 +
                        if None
 +
  -rp, --random_password
 +
                        create random password and create wikiuser while at it
 +
                        (default: False)
 
   -p PASSWORD, --password PASSWORD
 
   -p PASSWORD, --password PASSWORD
 
                         set password for initial user [default: sysop-1234!]
 
                         set password for initial user [default: sysop-1234!]
−
   -bp BASEPORT, --basePort BASEPORT
+
   -pl PASSWORD_LENGTH, --password_length PASSWORD_LENGTH
−
                         set how base html port 80 to be exposed - incrementing by one for each version [default: 9080]
+
                         set the password length for random passwords[default:
−
   -sp SQLPORT, --sqlBasePort SQLPORT
+
                        15]
−
                         set base mySql port 3306 to be exposed - incrementing by one for each version [default: 9306]
+
  --prefix PREFIX      the container name prefix to use [default: mw]
−
   -smw SMWVERSION, --smwVersion SMWVERSION
+
  --prot PROT          change to https in case [default: http]
−
                         set SemanticMediaWiki Version to be installed default is None - no installation of SMW (default: None)
+
  --script_path SCRIPT_PATH
−
   -mv MARIADBVERSION, --mariaDBVersion MARIADBVERSION
+
                        change to any script_path you might need to set
−
                         mariaDB Version to be installed [default: 10.5]
+
                        [default: ]
−
   -f, --forceRebuild    shall the applications rebuild be forced (with stop and remove of existing containers) (default: False)
+
  --url URL            will set prot host,script_path, and optionally port
−
   --logo LOGO          set Logo [default: $wgResourceBasePath/resources/assets/wiki.png]
+
                        based on the url given [default: None]
 +
   -sp SQL_PORT, --sql_base_port SQL_PORT
 +
                         set base mySql port 3306 to be exposed - incrementing
 +
                        by one for each version [default: 9306]
 +
   -smw SMW_VERSION, --smw_version SMW_VERSION
 +
                         set SemanticMediaWiki Version to be installed default
 +
                        is None - no installation of SMW (default: None)
 +
   -u USER, --user USER  set username of initial user with sysop rights
 +
                        [default: Sysop]
 +
  -q, --quiet          not verbose [default: False]
 +
  -bp BASE_PORT, --base_port BASE_PORT
 +
                         set how base html port 80 to be exposed - incrementing
 +
                        by one for each version [default: 9080]
 +
  -vl [VERSIONS ...], --version_list [VERSIONS ...]
 +
                        mediawiki versions to create docker applications for
 +
                        [default: ['1.35.13', '1.38.6', '1.39.7', '1.40.3',
 +
                        '1.41.1']]
 +
  --about              show about info [default: False]
 +
   --create              create wikis [default: False]
 +
  --down                shutdown wikis [default: False]
 +
  --check              check the wikis [default: False]
 +
   --list                list the wikis [default: False]
 +
  -V, --version        show program's version number and exit
 
</source>
 
</source>
  
 
= Examples =
 
= Examples =
−
== Default settings ==
+
{{Link|target=Pymediawikidocker_example_2022-10-26}}
−
Starts a cluster with versions 1.27.7, 1.31.14 and 1.35.2 using MariaDB 10.5
+
== create with default settings ==
 +
Starts a cluster with versions 1.35.13,1.38.6 and 1.39.8, 1.40.4, 1.41.2 and 1.42.1 using MariaDB 10.11.
 +
Essentially this is automatic generation of the configuration with a docker compose up for all versions asked for.
 
<source lang='bash' highlight='1'>
 
<source lang='bash' highlight='1'>
−
mwcluster -f
+
mwcluster --create -f
−
starting mediawiki 1.27.7 docker application ...
+
creating docker compose applications for mediawiki versions ['1.35.13', '1.38.6', '1.39.8', '1.40.4', '1.41.2', '1.42.1']
−
stopping and removing container mw1_27_7_db_1
+
...
−
stopping and removing container mw1_27_7_mw_1
 
−
[+] Building 0.5s (12/12) FINISHED                                                                                   
 
−
=> [internal] load build definition from Dockerfile                                                              0.0s
 
−
=> => transferring dockerfile: 1.10kB                                                                            0.0s
 
−
=> [internal] load .dockerignore                                                                                0.0s
 
−
=> => transferring context: 2B                                                                                  0.0s
 
−
=> [internal] load metadata for docker.io/library/mediawiki:1.27.7                                              0.0s
 
−
=> [1/7] FROM docker.io/library/mediawiki:1.27.7                                                                0.0s
 
−
=> [internal] load build context                                                                                0.0s
 
−
=> => transferring context: 61.98kB                                                                              0.0s
 
−
=> CACHED [2/7] RUN apt-get -y update &&    apt-get -y --no-install-recommends --fix-missing install    maria  0.0s
 
−
=> [3/7] COPY LocalSettings.php /var/www/html                                                                    0.0s
 
−
=> [4/7] COPY phpinfo.php /var/www/html                                                                          0.0s
 
−
=> [5/7] COPY wiki.sql /tmp                                                                                      0.0s
 
−
=> [6/7] COPY initdb.sh /tmp                                                                                    0.0s
 
−
=> [7/7] RUN chmod +x /tmp/initdb.sh                                                                            0.2s
 
−
=> exporting to image                                                                                            0.1s
 
−
=> => exporting layers                                                                                          0.0s
 
−
=> => writing image sha256:acee6c878b9a8dde738b38c8d73812ae92b175bd1ddefe1f2493d26af7e7bbdb                      0.0s
 
−
=> => naming to docker.io/library/mw1_27_7_mw                                                                    0.0s
 
−
Initializing MediaWiki SQL tables
 
−
Trying DB-Connection to wiki on localhost port 9306 with user wikiuser with max 6 tries and 10s timeout per try - initial sleep 2.5s
 
−
Connection to wiki on localhost with user wikiuser established database returns: ('wiki',)
 
−
starting mediawiki 1.31.14 docker application ...
 
−
stopping and removing container mw1_31_14_db_1
 
−
stopping and removing container mw1_31_14_mw_1
 
−
[+] Building 0.5s (12/12) FINISHED                                                                                   
 
−
=> [internal] load build definition from Dockerfile                                                              0.0s
 
−
=> => transferring dockerfile: 1.10kB                                                                            0.0s
 
−
=> [internal] load .dockerignore                                                                                0.0s
 
−
=> => transferring context: 2B                                                                                  0.0s
 
−
=> [internal] load metadata for docker.io/library/mediawiki:1.31.14                                              0.0s
 
−
=> [1/7] FROM docker.io/library/mediawiki:1.31.14                                                                0.0s
 
−
=> [internal] load build context                                                                                0.0s
 
−
=> => transferring context: 72.20kB                                                                              0.0s
 
−
=> CACHED [2/7] RUN apt-get -y update &&    apt-get -y --no-install-recommends --fix-missing install    maria  0.0s
 
−
=> [3/7] COPY LocalSettings.php /var/www/html                                                                    0.0s
 
−
=> [4/7] COPY phpinfo.php /var/www/html                                                                          0.0s
 
−
=> [5/7] COPY wiki.sql /tmp                                                                                      0.0s
 
−
=> [6/7] COPY initdb.sh /tmp                                                                                    0.0s
 
−
=> [7/7] RUN chmod +x /tmp/initdb.sh                                                                            0.2s
 
−
=> exporting to image                                                                                            0.1s
 
−
=> => exporting layers                                                                                          0.0s
 
−
=> => writing image sha256:a478ee76ef95f5c6eafc88828058a1d67f4033fcb110abc1b0085729d682c613                      0.0s
 
−
=> => naming to docker.io/library/mw1_31_14_mw                                                                  0.0s
 
−
Initializing MediaWiki SQL tables
 
−
Trying DB-Connection to wiki on localhost port 9307 with user wikiuser with max 6 tries and 10s timeout per try - initial sleep 2.5s
 
−
Connection to wiki on localhost with user wikiuser established database returns: ('wiki',)
 
−
starting mediawiki 1.35.2 docker application ...
 
−
stopping and removing container mw1_35_2_db_1
 
−
stopping and removing container mw1_35_2_mw_1
 
−
[+] Building 0.5s (12/12) FINISHED                                                                                   
 
−
=> [internal] load build definition from Dockerfile                                                              0.0s
 
−
=> => transferring dockerfile: 1.10kB                                                                            0.0s
 
−
=> [internal] load .dockerignore                                                                                0.0s
 
−
=> => transferring context: 2B                                                                                  0.0s
 
−
=> [internal] load metadata for docker.io/library/mediawiki:1.35.2                                              0.0s
 
−
=> [1/7] FROM docker.io/library/mediawiki:1.35.2                                                                0.0s
 
−
=> [internal] load build context                                                                                0.0s
 
−
=> => transferring context: 69.71kB                                                                              0.0s
 
−
=> CACHED [2/7] RUN apt-get -y update &&    apt-get -y --no-install-recommends --fix-missing install    maria  0.0s
 
−
=> [3/7] COPY LocalSettings.php /var/www/html                                                                    0.0s
 
−
=> [4/7] COPY phpinfo.php /var/www/html                                                                          0.0s
 
−
=> [5/7] COPY wiki.sql /tmp                                                                                      0.0s
 
−
=> [6/7] COPY initdb.sh /tmp                                                                                    0.0s
 
−
=> [7/7] RUN chmod +x /tmp/initdb.sh                                                                            0.2s
 
−
=> exporting to image                                                                                            0.1s
 
−
=> => exporting layers                                                                                          0.1s
 
−
=> => writing image sha256:a90cfd37ad0fc08329db5b499437f2539364c6b9d0c4a40cfbadcb9b51debd29                      0.0s
 
−
=> => naming to docker.io/library/mw1_35_2_mw                                                                    0.0s
 
 
Initializing MediaWiki SQL tables
 
Initializing MediaWiki SQL tables
−
Trying DB-Connection to wiki on localhost port 9308 with user wikiuser with max 6 tries and 10s timeout per try - initial sleep 2.5s
+
Executing docker command /root/installExtensions.sh
−
Connection to wiki on localhost with user wikiuser established database returns: ('wiki',)
+
Executing docker command /root/initdb.sh
 +
Executing docker command /root/update.sh
 +
Executing docker command /root/addSysopUser.sh
 +
Executing docker command /root/addCronTabEntry.sh
 +
...
 
</source>
 
</source>
 
Mediawiki applications will be available at
 
Mediawiki applications will be available at
−
# http://localhost:9080
+
# {{mwlink|port=9080}}
−
# http://localhost:9081
+
# {{mwlink|port=9081}}
−
# http://localhost:9082
+
# {{mwlink|port=9082}}
 +
# {{mwlink|port=9083}}
 +
# {{mwlink|port=9084}}
 +
# {{mwlink|port=9085}}
  
−
== Single 1.27.5 ==
+
=== check ===
−
<source lang='bash' highlight="1">
 
−
python mwdocker/mwcluster.py --forceRebuild --versionList 1.27.5 --basePort 8480 --sqlBasePort 10306 --mariaDBVersion 10.5
 
−
</source>
 
−
Mediawiki application will be available at
 
−
# http://localhost:8480
 
−
== Single 1.35 ==
 
 
<source lang='bash' highlight='1'>
 
<source lang='bash' highlight='1'>
−
lsb_release -a
+
mwcluster --check -f
−
No LSB modules are available.
+
checking docker access for mediawiki versions ['1.35.13', '1.38.6', '1.39.8', '1.40.4', '1.41.2', '1.42.1']
−
Distributor ID: Ubuntu
+
1:checking 1.35.13 ...
−
Description: Ubuntu 20.04.4 LTS
+
mediawiki webserver container mw-135-mw:✅
−
Release: 20.04
+
mediawiki database container mw-135-db:✅
−
Codename: focal
+
port binding 9080= expected  port 9080?:✅
 +
Checking http://fixit.bitplan.com:9080/index.php/Special:Version ...
 +
Special Version accessible ...:✅
 +
Mediawiki Version 1.35.13= expected  1.35.13?:✅
 +
Maria DB Version 10.11 fitting expected 10.11?:✅
 +
2:checking 1.38.6 ...
 +
mediawiki webserver container mw-138-mw:✅
 +
mediawiki database container mw-138-db:✅
 +
port binding 9081= expected  port 9081?:✅
 +
Checking http://fixit.bitplan.com:9081/index.php/Special:Version ...
 +
Special Version accessible ...:✅
 +
Mediawiki Version 1.38.6= expected  1.38.6?:✅
 +
Maria DB Version 10.11 fitting expected 10.11?:✅
 +
3:checking 1.39.8 ...
 +
mediawiki webserver container mw-139-mw:✅
 +
mediawiki database container mw-139-db:✅
 +
port binding 9082= expected  port 9082?:✅
 +
Checking http://fixit.bitplan.com:9082/index.php/Special:Version ...
 +
Special Version accessible ...:✅
 +
Mediawiki Version 1.39.8= expected  1.39.8?:✅
 +
Maria DB Version 10.11 fitting expected 10.11?:✅
 +
4:checking 1.40.4 ...
 +
mediawiki webserver container mw-140-mw:✅
 +
mediawiki database container mw-140-db:✅
 +
port binding 9083= expected  port 9083?:✅
 +
Checking http://fixit.bitplan.com:9083/index.php/Special:Version ...
 +
Special Version accessible ...:✅
 +
Mediawiki Version 1.40.4= expected  1.40.4?:✅
 +
Maria DB Version 10.11 fitting expected 10.11?:✅
 +
5:checking 1.41.2 ...
 +
mediawiki webserver container mw-141-mw:✅
 +
mediawiki database container mw-141-db:✅
 +
port binding 9084= expected  port 9084?:✅
 +
Checking http://fixit.bitplan.com:9084/index.php/Special:Version ...
 +
Special Version accessible ...:✅
 +
Mediawiki Version 1.41.2= expected  1.41.2?:✅
 +
Maria DB Version 10.11 fitting expected 10.11?:✅
 +
6:checking 1.42.1 ...
 +
mediawiki webserver container mw-142-mw:✅
 +
mediawiki database container mw-142-db:✅
 +
port binding 9085= expected  port 9085?:✅
 +
Checking http://fixit.bitplan.com:9085/index.php/Special:Version ...
 +
Special Version accessible ...:✅
 +
Mediawiki Version 1.42.1= expected  1.42.1?:✅
 +
Maria DB Version 10.11 fitting expected 10.11?:✅
 
</source>
 
</source>
−
follow procedure in https://nextgentips.com/2022/05/06/how-to-install-docker-compose-v2-on-ubuntu-22-04/
+
 
 +
== down ==
 
<source lang='bash' highlight='1'>
 
<source lang='bash' highlight='1'>
−
docker compose version
+
mwcluster --down -f
−
Docker Compose version v2.5.0
+
running docker compose down for mediawiki versions ['1.35.13', '1.38.6', '1.39.8', '1.40.4', '1.41.2', '1.42.1']
 +
running docker compose down for mw-135 1.35.13 docker application ...
 +
[+] Running 6/6
 +
✔ Container mw-135-mw      Removed                                      1.1s
 +
✔ Container mw-135-db      Removed                                      0.5s
 +
✔ Volume mw-135_wiki-etc    Removed                                      0.3s
 +
✔ Volume mw-135_mysql-data  Removed                                      0.3s
 +
✔ Volume mw-135_wiki-www    Removed                                      0.3s
 +
✔ Network mw-135_default    Removed                                      0.1s
 +
running docker compose down for mw-138 1.38.6 docker application ...
 +
[+] Running 6/6
 +
✔ Container mw-138-mw      Removed                                      1.2s
 +
✔ Container mw-138-db      Removed                                      0.5s
 +
✔ Volume mw-138_wiki-etc    Removed                                      0.0s
 +
✔ Volume mw-138_mysql-data  Removed                                      0.0s
 +
✔ Volume mw-138_wiki-www    Removed                                      0.4s
 +
✔ Network mw-138_default    Removed                                      0.1s
 +
running docker compose down for mw-139 1.39.8 docker application ...
 +
[+] Running 6/6
 +
✔ Container mw-139-mw      Removed                                      1.2s
 +
✔ Container mw-139-db      Removed                                      0.5s
 +
✔ Volume mw-139_wiki-www    Removed                                      0.3s
 +
✔ Volume mw-139_wiki-etc    Removed                                      0.3s
 +
✔ Volume mw-139_mysql-data  Removed                                      0.3s
 +
✔ Network mw-139_default    Removed                                      0.1s
 +
running docker compose down for mw-140 1.40.4 docker application ...
 +
[+] Running 6/6
 +
✔ Container mw-140-mw      Removed                                      1.2s
 +
✔ Container mw-140-db      Removed                                      0.5s
 +
✔ Volume mw-140_wiki-etc    Removed                                      0.0s
 +
✔ Volume mw-140_mysql-data  Removed                                      0.0s
 +
✔ Volume mw-140_wiki-www    Removed                                      0.4s
 +
✔ Network mw-140_default    Removed                                      0.1s
 +
running docker compose down for mw-141 1.41.2 docker application ...
 +
[+] Running 6/6
 +
✔ Container mw-141-mw      Removed                                      1.2s
 +
✔ Container mw-141-db      Removed                                      0.5s
 +
✔ Volume mw-141_mysql-data  Removed                                      0.4s
 +
✔ Volume mw-141_wiki-etc    Removed                                      0.4s
 +
✔ Volume mw-141_wiki-www    Removed                                      0.4s
 +
✔ Network mw-141_default    Removed                                      0.1s
 +
running docker compose down for mw-142 1.42.1 docker application ...
 +
[+] Running 6/6
 +
✔ Container mw-142-mw      Removed                                      1.2s
 +
✔ Container mw-142-db      Removed                                      0.5s
 +
✔ Volume mw-142_wiki-www    Removed                                      0.4s
 +
✔ Volume mw-142_mysql-data  Removed                                      0.0s
 +
✔ Volume mw-142_wiki-etc    Removed                                      0.0s
 +
✔ Network mw-142_default    Removed                                      0.1s
 
</source>
 
</source>
−
<source lang='bash' highlight='1-2,7'>
+
 
−
sudo apt install docker-compose
+
== Single 1.35.13 LTS ==
−
docker-compose version
+
<source lang='bash' highlight="2">
−
docker-compose version 1.25.0, build unknown
+
# python mwdocker/mwcluster.py can be used instead of mwcluster if you test from your locally cloned environment
−
docker-py version: 4.1.0
+
mwcluster \
−
CPython version: 3.8.10
+
  --create \
−
OpenSSL version: OpenSSL 1.1.1f  31 Mar 2020
+
  --forceRebuild\
−
which docker-compose
+
  --version_list 1.35.13\
−
/usr/bin/docker-compose
+
  --base_port 8480\
 +
  --sql_base_port 10306\
 +
  --mariaDBVersion 10.11\
 +
  --container_name mw135lts\
 +
  --logo https://upload.wikimedia.org/wikipedia/commons/thumb/f/f9/Bigsmile_smiley_yellow_simple.svg/128px-Bigsmile_smiley_yellow_simple.svg.png
 +
creating docker compose applications for mediawiki versions ['1.35.13']
 +
starting mw135lts 1.35.13 docker application ..
 
</source>
 
</source>
−
=== create docker compose files ===
+
Mediawiki application will be available at
−
<source lang='bash' highlight='1'>
+
# {{Mwlink|port=8480}}
−
python mwdocker/mwcluster.py --forceRebuild --versionList 1.35.5 --basePort 8480 --sqlBasePort 10306 --mariaDBVersion 10.5
+
[[File:pyMediaWikiDocker-ExampleMW135WithLogo.png|600px]]
 +
 
 +
== Single 1.35 with Semantic Mediawiki ==
 +
There might be more current versions of SMW available see https://www.semantic-mediawiki.org/wiki/Semantic_MediaWiki_Version_History and adapt
 +
the example below if you see the need to do so.
 +
=== Version 4.1.2 ===
 +
<source lang='bash' highlight='1-9'>
 +
mwcluster \
 +
  --create \
 +
  --container_name smw4\
 +
  --forceRebuild\
 +
  --version_list 1.35.13\
 +
  --base_port 8481\
 +
  --sql_base_port 10307\
 +
  --mariaDBVersion 10.11\
 +
  --smw_version 4.1.2
 +
creating docker compose applications for mediawiki versions ['1.35.13']
 +
starting smw4 1.35.13 docker application ...
 
</source>
 
</source>
−
=== debug docker compose results - ports and other configuration parts ===
+
 
 +
==== debug docker compose results - ports and other configuration parts ====
 
<source lang='bash' highlight='1-2,4'>
 
<source lang='bash' highlight='1-2,4'>
−
cd ~/.pymediawikidocker/mw1_35_5
+
cd ~/.pymediawikidocker/smw4
 
docker-compose up
 
docker-compose up
 
# in another terminal session:
 
# in another terminal session:
 
docker ps
 
docker ps
−
CONTAINER ID  IMAGE         COMMAND                  CREATED         STATUS        PORTS                                         NAMES
+
CONTAINER ID  IMAGE                   COMMAND                  CREATED             STATUS        PORTS                     NAMES
−
682b81b36fc7   mw1_35_5_mw    "docker-php-entrypoi…"  2 minutes ago  Up 2 minutes   0.0.0.0:8480->80/tcp, :::8480->80/tcp        mw1_35_5_mw_1
+
87a795d444eb   smw4_mw                "docker-php-entrypoi…"  About a minute ago  Up 7 seconds   0.0.0.0:8481->80/tcp     smw4-mw
−
3017fbc569d3   mariadb:10.5  "docker-entrypoint.s…"  2 minutes ago  Up 2 minutes   0.0.0.0:10306->3306/tcp, :::10306->3306/tcp  mw1_35_5_db_1
+
7ca3a0434d31   mariadb:10.11          "docker-entrypoint.s…"  About a minute ago  Up 7 seconds   0.0.0.0:10307->3306/tcp   smw4-db
 
</source>
 
</source>
 
==== open bash in docker container ====
 
==== open bash in docker container ====
−
<source lang='bash'>
+
<source lang='bash' highlight='1'>
−
docker exec -it mw1_35_5_mw_1 /bin/bash
+
docker exec -it smw4-mw /bin/bash
 +
root@87a795d444eb:/var/www/html#
 +
</source>
 +
'''check installExtensions.sh script'''
 +
<source lang='bash' highlight='1'>
 +
cat /root/installExtensions.sh
 +
#!/bin/bash
 +
# install the required extensions
 +
# WF 2021-06-23
 +
cd /var/www/html/extensions
 +
 
 +
# Admin Links
 +
# https://www.mediawiki.org/wiki/Extension:Admin_Links
 +
git clone https://github.com/wikimedia/mediawiki-extensions-AdminLinks --single-branch --branch REL1_35 AdminLinks
 +
 
 +
# Header Tabs
 +
# https://www.mediawiki.org/wiki/Extension:Header_Tabs
 +
git clone https://github.com/wikimedia/mediawiki-extensions-HeaderTabs --single-branch --branch REL1_35 HeaderTabs
 +
 
 +
# SyntaxHighlight
 +
# https://www.mediawiki.org/wiki/Extension:SyntaxHighlight
 +
# no installation script command specified
 +
 
 +
# Variables
 +
# https://www.mediawiki.org/wiki/Extension:Variables
 +
git clone https://github.com/wikimedia/mediawiki-extensions-Variables --single-branch --branch REL1_35 Variables
 +
</source>
 +
'''check Semantic MediaWiki settings in LocalSettings.php'''
 +
<source lang='bash' highlight='1'>
 +
grep -i semantic LocalSettings.php
 +
# enable Support for Semantic MediaWiki
 +
# see https://www.semantic-mediawiki.org/wiki/Help:EnableSemantics
 +
# Version of SemanticMediaWiki at install time: 4.1.1
 +
wfLoadExtension( 'SemanticMediaWiki' );
 +
enableSemantics();
 +
# https://www.semantic-mediawiki.org/wiki/Help:$smwgQMaxInlineLimit
 +
</source>
 +
=== SMW Version 3.2.3 ===
 +
==== Installation ====
 +
<source lang='bash' highlight='1-10'>
 +
mwcluster \
 +
  --create \
 +
  --container_name smw3\
 +
  --forceRebuild\
 +
  --version_list 1.35.13\
 +
  --base_port 8482\
 +
  --sql_base_port 10308\
 +
  --mariaDBVersion 10.11\
 +
  --logo https://www.semantic-mediawiki.org/w/images/4/4e/SMW_logo_180.png\
 +
  --smw_version 3.2.3
 +
</source>
 +
 
 +
==== Check ====
 +
<source lang='bash' highlight='1'>
 +
mwcluster --check --container_name smw3 --version_list 1.35.13
 
</source>
 
</source>
  
−
== Semantic MediaWiki cluster with WikiUser creation ==
+
==== Down ====
−
<source lang='bash' highlight="1">
+
<source lang='bash' highlight='1'>
−
mwcluster -f -smw 3.2.3 -vl 1.31.14 1.35.2 1.36.0 -wl  mw31test mw35test mw36test
+
mwcluster --down --container_name smw3 --version_list 1.35.13 -f
−
creating docker applications for mediawiki versions ['1.31.14', '1.35.2', '1.36.0']
+
running docker compose down for mediawiki versions ['1.35.13']
 +
running docker compose down for smw3 1.35.10 docker application ...
 +
[+] Running 3/3
 +
⠿ Volume smw3_wiki-etc    Removed                                0.0s
 +
⠿ Volume smw3_mysql-data  Removed                                0.1s
 +
⠿ Volume smw3_wiki-www    Removed                                1.2s
 
</source>
 
</source>
  
 +
= Extensions =
 +
see {{Link|target=Pymediawikidocker/Extensions}}
 
= Debugging =
 
= Debugging =
 
== Docker files ==
 
== Docker files ==
Line 291: Line 522:
 
-rw-r--r--  1 wf  staff  64515 22 Jun 07:03 wiki.sql
 
-rw-r--r--  1 wf  staff  64515 22 Jun 07:03 wiki.sql
 
</source>
 
</source>
 +
== Check installed docker images ==
 +
<source lang='bash' highlight='1'>
 +
docker images | grep mw
 +
mw-142-mw    latest    13e469c7d859  54 minutes ago      1.17GB
 +
mw-141-mw    latest    99578ea06019  55 minutes ago      1.2GB
 +
mw-140-mw    latest    d84a2410dbea  56 minutes ago      1.13GB
 +
mw-139-mw    latest    f83c36dee73d  58 minutes ago      1.06GB
 +
mw-138-mw    latest    8bfb68343139  59 minutes ago      1.03GB
 +
mw-135-mw    latest    bf7ae8fe499c  About an hour ago  1.03GB
 +
</source>
 +
 
== Bash into docker containers ==
 
== Bash into docker containers ==
−
<source lang='bash' highlight='2,4'>
+
 
 +
<source lang='bash' highlight='2-6,9-13'>
 
# Mediawiki container
 
# Mediawiki container
−
docker exec -it mw1_36_0_mw_1 /bin/bash
+
docker exec -it mw-142-mw /bin/bash
 +
docker exec -it mw-141-mw /bin/bash
 +
docker exec -it mw-140-mw /bin/bash
 +
docker exec -it mw-139-mw /bin/bash
 +
docker exec -it mw-138-mw /bin/bash
 +
docker exec -it mw-135-mw /bin/bash
 +
 
 
# MariaDB container
 
# MariaDB container
−
docker exec -it mw1_36_0_db_1 /bin/bash
+
docker exec -it mw-142-db /bin/bash
 +
docker exec -it mw-141-db /bin/bash
 +
docker exec -it mw-140-db /bin/bash
 +
docker exec -it mw-139-db /bin/bash
 +
docker exec -it mw-138-db /bin/bash
 +
docker exec -it mw-135-db /bin/bash
 +
</source>
 +
 
 +
== Monkey patching ==
 +
<source lang='bash' highlight='2'>
 +
# Enable raw html in for the wiki
 +
docker exec  mw1_35_8-mw sh -c "echo '\$wgRawHtml = true;\n' >> /var/www/html/LocalSettings.php"
 
</source>
 
</source>
  
Line 318: Line 578:
 
docker cp mw1_36_0_db_1:/tmp/wiki.sql /tmp
 
docker cp mw1_36_0_db_1:/tmp/wiki.sql /tmp
 
</source>
 
</source>
−
== Stop and Remove all running containers ==
+
== Docker cleanup procedure ==
−
These commands will stop and remove '''all''' running docker containers (not only the ones created pymediawikidocker/the mwcluster command)
 
 
<source lang='bash'>
 
<source lang='bash'>
−
docker stop $(docker ps -aq)
+
#!/bin/bash
 +
# WF 2023-04-08
 +
#
 +
# ⚠️ ⚠️ ⚠️
 +
#
 +
#  kids  don't do this at home ...
 +
#
 +
# ⚠️ These commands will stop and remove all running docker containers (not only the ones created pymediawikidocker/the mwcluster command)
 +
# kill/stop all running containers
 +
docker kill $(docker ps -q)
 +
#docker stop $(docker ps -aq)
 +
# remove all docker containers
 
docker rm $(docker ps -aq)
 
docker rm $(docker ps -aq)
 +
# remove all images
 +
# ⚠️ This command will remove all your images not only the ones created by pymediawikidocker (docker will hopefully refuse to do this without the -f option)
 +
docker rmi $(docker images -q)
 +
# ⚠️ these commands will remove all your volumes - you might loose valuable data if you don't check what your volumes contain before starting this command
 +
# see https://stackoverflow.com/a/46091568/1497139
 +
docker volume rm $(docker volume ls -qf dangling=true | xargs)
 +
#⚠️ WARNING! This will remove:
 +
#  - all stopped containers
 +
#  - all networks not used by at least one container
 +
#  - all dangling images
 +
#  - all dangling build cache
 +
docker system prune --force
 
</source>
 
</source>
  
 
= Links =
 
= Links =
 
* [https://www.wikidata.org/wiki/Q83 Wikidata entry for Mediawiki software]
 
* [https://www.wikidata.org/wiki/Q83 Wikidata entry for Mediawiki software]

Latest revision as of 19:44, 24 October 2024

OsProject

OsProject
edit
id  pymediawikidocker
state  active
owner  WolfgangFahl
title  pymediawikidocker
url  https://github.com/WolfgangFahl/pymediawikidocker
version  0.13.4
description  Python controlled mediawiki docker image installation
date  2024-10-24
since  2021-06-08
until  

tickets

Motivation

Official Mediawiki docker images are available for the different Mediawiki versions.

These images need additional infrastructure to create useable Mediawiki environments. 

docker run --name local-mediawiki -p 8080:80 -d mediawiki

will only install part of a LAMP environment to install Mediawiki and make it available via http://localhost:8080/. E.g. the database needs to be installed separately.

The goal of this project is to semi-automatically create such environments and test the functionality. Python was choosen as a development environment for the libraries that are needed and available to perform the tasks:

python on whales was chosen over:

Since it works on the command line interface and not via the SDK and supports docker compose.

How it works

The core concept is a "MediawikiCluster" that is a set of docker applications that run MediaWiki instances with varying versions of MediaWiki and the underlying database. The graphic belows shows the docker containers which are created for different versions of Mediawiki

Installation

Prerequisites

Docker on Windows, Mac or Linux suppporting "docker compose up" using the Docker composer V2 CLI

Ubuntu 22.04 LTS installation

see https://docs.docker.com/engine/install/ubuntu/ on how to setup the docker repositorysudo 

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
docker --version
Docker version 23.0.2, build 569dd73
docker run hello-world
Hello from Docker!
This message shows that your installation appears to be working correctly.

...

Compatibility test

# test on a Mac OS machine 2022-07-02
docker 2>&1 | grep compose
  compose*    Docker Compose (Docker Inc., v2.4.1)
# test on an Ubuntu machine 2023-04-07
docker 2>&1 | grep compose
 compose*    Docker Compose (Docker Inc., v2.17.2)

Installation of Docker Compose V2 CLI

lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.e LTS
Release:	22.04
Codename:	jammy

docker-ce from docker repo

# Update existing list of packages
sudo apt update

# Install prerequisite packages which let apt use packages over HTTPS
sudo apt install apt-transport-https ca-certificates curl software-properties-common

# Add Docker’s official GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

# Add Docker repository to APT sources
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

# Update the package database with Docker packages from the newly added repo
sudo apt update

# Make sure you are about to install from the Docker repo instead of the default Ubuntu repo
apt-cache policy docker-ce

# Install Docker CE
sudo apt install docker-ce

docker compose (not docker-compose)

docker compose version
Docker Compose version v2.26.1

sudo apt  install docker-compose
docker-compose version
docker-compose version 1.29.2, build unknown
docker-py version: 5.0.3
CPython version: 3.10.12
OpenSSL version: OpenSSL 3.0.2 15 Mar 2022
which docker-compose
/usr/bin/docker-compose

pymediawikidocker installation via pip

pip install pymediawikidocker
# alternatively if your pip is not a python3 pip
pip3 install pymediawikidocker

upgrade

pip install pymediawikidocker -U
# alternatively if your pip is not a python3 pip
pip3 install pymediawikidocker -U

check version

mwcluster -V
mwcluster (v0.13.0,2024-08-02)

Usage

mwcluster -h
usage: mwcluster [-h] [-cn CONTAINER_NAME] [-d] [-el [EXTENSIONNAMELIST ...]]
                 [-ej EXTENSIONJSONFILE] [-f] [-fu] [--host HOST]
                 [-dp DOCKER_PATH] [--logo LOGO] [-mv MARIADBVERSION]
                 [--mysqlPassword MYSQLPASSWORD] [-rp] [-p PASSWORD]
                 [-pl PASSWORD_LENGTH] [--prefix PREFIX] [--prot PROT]
                 [--script_path SCRIPT_PATH] [--url URL] [-sp SQL_PORT]
                 [-smw SMW_VERSION] [-u USER] [-q] [-bp BASE_PORT]
                 [-vl [VERSIONS ...]] [--about] [--create] [--down] [--check]
                 [--list] [-V]

Copyright 2020-2023 contributors. All rights reserved. Licensed under the
Apache License 2.0 http://www.apache.org/licenses/LICENSE-2.0 Distributed on
an "AS IS" basis without warranties or conditions of any kind, either express
or implied.

options:
  -h, --help            show this help message and exit
  -cn CONTAINER_NAME, --container_name CONTAINER_NAME
                        set container name (only valid and recommended for
                        single version call) (default: mw-139)
  -d, --debug           enable debug mode [default: False]
  -el [EXTENSIONNAMELIST ...], --extensionList [EXTENSIONNAMELIST ...]
                        list of extensions to be installed [default: ['Admin
                        Links', 'Header Tabs', 'SyntaxHighlight',
                        'Variables']]
  -ej EXTENSIONJSONFILE, --extensionJson EXTENSIONJSONFILE
                        additional extension descriptions default: [default:
                        None]
  -f, --forceRebuild    force rebuilding [default: False]
  -fu, --force_user     force overwrite of wikiuser (default: False)
  --host HOST           the host to serve / listen from [default:
                        fix.bitplan.com]
  -dp DOCKER_PATH, --docker_path DOCKER_PATH
                        the base directory to store docker and jinja template
                        files [default: /Users/wf/.pymediawikidocker]
  --logo LOGO           set Logo [default:
                        $wgResourceBasePath/resources/assets/wiki.png]
  -mv MARIADBVERSION, --mariaDBVersion MARIADBVERSION
                        mariaDB Version to be installed [default: 10.11]
  --mysqlPassword MYSQLPASSWORD
                        set sqlRootPassword [default: None] - random password
                        if None
  -rp, --random_password
                        create random password and create wikiuser while at it
                        (default: False)
  -p PASSWORD, --password PASSWORD
                        set password for initial user [default: sysop-1234!]
  -pl PASSWORD_LENGTH, --password_length PASSWORD_LENGTH
                        set the password length for random passwords[default:
                        15]
  --prefix PREFIX       the container name prefix to use [default: mw]
  --prot PROT           change to https in case [default: http]
  --script_path SCRIPT_PATH
                        change to any script_path you might need to set
                        [default: ]
  --url URL             will set prot host,script_path, and optionally port
                        based on the url given [default: None]
  -sp SQL_PORT, --sql_base_port SQL_PORT
                        set base mySql port 3306 to be exposed - incrementing
                        by one for each version [default: 9306]
  -smw SMW_VERSION, --smw_version SMW_VERSION
                        set SemanticMediaWiki Version to be installed default
                        is None - no installation of SMW (default: None)
  -u USER, --user USER  set username of initial user with sysop rights
                        [default: Sysop]
  -q, --quiet           not verbose [default: False]
  -bp BASE_PORT, --base_port BASE_PORT
                        set how base html port 80 to be exposed - incrementing
                        by one for each version [default: 9080]
  -vl [VERSIONS ...], --version_list [VERSIONS ...]
                        mediawiki versions to create docker applications for
                        [default: ['1.35.13', '1.38.6', '1.39.7', '1.40.3',
                        '1.41.1']]
  --about               show about info [default: False]
  --create              create wikis [default: False]
  --down                shutdown wikis [default: False]
  --check               check the wikis [default: False]
  --list                list the wikis [default: False]
  -V, --version         show program's version number and exit

Examples

Pymediawikidocker_example_2022-10-26

create with default settings

Starts a cluster with versions 1.35.13,1.38.6 and 1.39.8, 1.40.4, 1.41.2 and 1.42.1 using MariaDB 10.11. Essentially this is automatic generation of the configuration with a docker compose up for all versions asked for. 

mwcluster --create -f
creating docker compose applications for mediawiki versions ['1.35.13', '1.38.6', '1.39.8', '1.40.4', '1.41.2', '1.42.1']
...
Initializing MediaWiki SQL tables
Executing docker command /root/installExtensions.sh
Executing docker command /root/initdb.sh
Executing docker command /root/update.sh
Executing docker command /root/addSysopUser.sh
Executing docker command /root/addCronTabEntry.sh
...

Mediawiki applications will be available at

http://localhost:9080 http://localhost:9080/index.php/Special:Version

http://localhost:9081 http://localhost:9081/index.php/Special:Version

http://localhost:9082 http://localhost:9082/index.php/Special:Version

http://localhost:9083 http://localhost:9083/index.php/Special:Version

http://localhost:9084 http://localhost:9084/index.php/Special:Version

http://localhost:9085 http://localhost:9085/index.php/Special:Version

check

mwcluster --check -f
checking docker access for mediawiki versions ['1.35.13', '1.38.6', '1.39.8', '1.40.4', '1.41.2', '1.42.1']
1:checking 1.35.13 ...
mediawiki webserver container mw-135-mw:✅
mediawiki database container mw-135-db:✅
port binding 9080= expected  port 9080?:✅
Checking http://fixit.bitplan.com:9080/index.php/Special:Version ...
Special Version accessible ...:✅
Mediawiki Version 1.35.13= expected  1.35.13?:✅
Maria DB Version 10.11 fitting expected 10.11?:✅
2:checking 1.38.6 ...
mediawiki webserver container mw-138-mw:✅
mediawiki database container mw-138-db:✅
port binding 9081= expected  port 9081?:✅
Checking http://fixit.bitplan.com:9081/index.php/Special:Version ...
Special Version accessible ...:✅
Mediawiki Version 1.38.6= expected  1.38.6?:✅
Maria DB Version 10.11 fitting expected 10.11?:✅
3:checking 1.39.8 ...
mediawiki webserver container mw-139-mw:✅
mediawiki database container mw-139-db:✅
port binding 9082= expected  port 9082?:✅
Checking http://fixit.bitplan.com:9082/index.php/Special:Version ...
Special Version accessible ...:✅
Mediawiki Version 1.39.8= expected  1.39.8?:✅
Maria DB Version 10.11 fitting expected 10.11?:✅
4:checking 1.40.4 ...
mediawiki webserver container mw-140-mw:✅
mediawiki database container mw-140-db:✅
port binding 9083= expected  port 9083?:✅
Checking http://fixit.bitplan.com:9083/index.php/Special:Version ...
Special Version accessible ...:✅
Mediawiki Version 1.40.4= expected  1.40.4?:✅
Maria DB Version 10.11 fitting expected 10.11?:✅
5:checking 1.41.2 ...
mediawiki webserver container mw-141-mw:✅
mediawiki database container mw-141-db:✅
port binding 9084= expected  port 9084?:✅
Checking http://fixit.bitplan.com:9084/index.php/Special:Version ...
Special Version accessible ...:✅
Mediawiki Version 1.41.2= expected  1.41.2?:✅
Maria DB Version 10.11 fitting expected 10.11?:✅
6:checking 1.42.1 ...
mediawiki webserver container mw-142-mw:✅
mediawiki database container mw-142-db:✅
port binding 9085= expected  port 9085?:✅
Checking http://fixit.bitplan.com:9085/index.php/Special:Version ...
Special Version accessible ...:✅
Mediawiki Version 1.42.1= expected  1.42.1?:✅
Maria DB Version 10.11 fitting expected 10.11?:✅

down

mwcluster --down -f
running docker compose down for mediawiki versions ['1.35.13', '1.38.6', '1.39.8', '1.40.4', '1.41.2', '1.42.1']
running docker compose down for mw-135 1.35.13 docker application ...
[+] Running 6/6
 ✔ Container mw-135-mw       Removed                                       1.1s 
 ✔ Container mw-135-db       Removed                                       0.5s 
 ✔ Volume mw-135_wiki-etc    Removed                                       0.3s 
 ✔ Volume mw-135_mysql-data  Removed                                       0.3s 
 ✔ Volume mw-135_wiki-www    Removed                                       0.3s 
 ✔ Network mw-135_default    Removed                                       0.1s 
running docker compose down for mw-138 1.38.6 docker application ...
[+] Running 6/6
 ✔ Container mw-138-mw       Removed                                       1.2s 
 ✔ Container mw-138-db       Removed                                       0.5s 
 ✔ Volume mw-138_wiki-etc    Removed                                       0.0s 
 ✔ Volume mw-138_mysql-data  Removed                                       0.0s 
 ✔ Volume mw-138_wiki-www    Removed                                       0.4s 
 ✔ Network mw-138_default    Removed                                       0.1s 
running docker compose down for mw-139 1.39.8 docker application ...
[+] Running 6/6
 ✔ Container mw-139-mw       Removed                                       1.2s 
 ✔ Container mw-139-db       Removed                                       0.5s 
 ✔ Volume mw-139_wiki-www    Removed                                       0.3s 
 ✔ Volume mw-139_wiki-etc    Removed                                       0.3s 
 ✔ Volume mw-139_mysql-data  Removed                                       0.3s 
 ✔ Network mw-139_default    Removed                                       0.1s 
running docker compose down for mw-140 1.40.4 docker application ...
[+] Running 6/6
 ✔ Container mw-140-mw       Removed                                       1.2s 
 ✔ Container mw-140-db       Removed                                       0.5s 
 ✔ Volume mw-140_wiki-etc    Removed                                       0.0s 
 ✔ Volume mw-140_mysql-data  Removed                                       0.0s 
 ✔ Volume mw-140_wiki-www    Removed                                       0.4s 
 ✔ Network mw-140_default    Removed                                       0.1s 
running docker compose down for mw-141 1.41.2 docker application ...
[+] Running 6/6
 ✔ Container mw-141-mw       Removed                                       1.2s 
 ✔ Container mw-141-db       Removed                                       0.5s 
 ✔ Volume mw-141_mysql-data  Removed                                       0.4s 
 ✔ Volume mw-141_wiki-etc    Removed                                       0.4s 
 ✔ Volume mw-141_wiki-www    Removed                                       0.4s 
 ✔ Network mw-141_default    Removed                                       0.1s 
running docker compose down for mw-142 1.42.1 docker application ...
[+] Running 6/6
 ✔ Container mw-142-mw       Removed                                       1.2s 
 ✔ Container mw-142-db       Removed                                       0.5s 
 ✔ Volume mw-142_wiki-www    Removed                                       0.4s 
 ✔ Volume mw-142_mysql-data  Removed                                       0.0s 
 ✔ Volume mw-142_wiki-etc    Removed                                       0.0s 
 ✔ Network mw-142_default    Removed                                       0.1s

Single 1.35.13 LTS

# python mwdocker/mwcluster.py  can be used instead of mwcluster if you test from your locally cloned environment
mwcluster \
  --create \
  --forceRebuild\
  --version_list 1.35.13\
  --base_port 8480\
  --sql_base_port 10306\
  --mariaDBVersion 10.11\
  --container_name mw135lts\
  --logo https://upload.wikimedia.org/wikipedia/commons/thumb/f/f9/Bigsmile_smiley_yellow_simple.svg/128px-Bigsmile_smiley_yellow_simple.svg.png
creating docker compose applications for mediawiki versions ['1.35.13']
starting mw135lts 1.35.13 docker application ..

Mediawiki application will be available at

http://localhost:8480 http://localhost:8480/index.php/Special:Version PyMediaWikiDocker-ExampleMW135WithLogo.png

Single 1.35 with Semantic Mediawiki

There might be more current versions of SMW available see https://www.semantic-mediawiki.org/wiki/Semantic_MediaWiki_Version_History and adapt the example below if you see the need to do so.

Version 4.1.2

mwcluster \
  --create \
  --container_name smw4\
  --forceRebuild\
  --version_list 1.35.13\
  --base_port 8481\
  --sql_base_port 10307\
  --mariaDBVersion 10.11\
  --smw_version 4.1.2
creating docker compose applications for mediawiki versions ['1.35.13']
starting smw4 1.35.13 docker application ...

debug docker compose results - ports and other configuration parts

cd ~/.pymediawikidocker/smw4
docker-compose up
# in another terminal session:
docker ps
CONTAINER ID   IMAGE                   COMMAND                  CREATED              STATUS         PORTS                     NAMES
87a795d444eb   smw4_mw                 "docker-php-entrypoi…"   About a minute ago   Up 7 seconds   0.0.0.0:8481->80/tcp      smw4-mw
7ca3a0434d31   mariadb:10.11           "docker-entrypoint.s…"   About a minute ago   Up 7 seconds   0.0.0.0:10307->3306/tcp   smw4-db

open bash in docker container

docker exec -it smw4-mw /bin/bash
root@87a795d444eb:/var/www/html#

check installExtensions.sh script 

cat /root/installExtensions.sh 
#!/bin/bash
# install the required extensions
# WF 2021-06-23
cd /var/www/html/extensions

# Admin Links
# https://www.mediawiki.org/wiki/Extension:Admin_Links
git clone https://github.com/wikimedia/mediawiki-extensions-AdminLinks --single-branch --branch REL1_35 AdminLinks

# Header Tabs
# https://www.mediawiki.org/wiki/Extension:Header_Tabs
git clone https://github.com/wikimedia/mediawiki-extensions-HeaderTabs --single-branch --branch REL1_35 HeaderTabs

# SyntaxHighlight
# https://www.mediawiki.org/wiki/Extension:SyntaxHighlight
# no installation script command specified

# Variables
# https://www.mediawiki.org/wiki/Extension:Variables
git clone https://github.com/wikimedia/mediawiki-extensions-Variables --single-branch --branch REL1_35 Variables

check Semantic MediaWiki settings in LocalSettings.php 

grep -i semantic LocalSettings.php 
# enable Support for Semantic MediaWiki
# see https://www.semantic-mediawiki.org/wiki/Help:EnableSemantics
# Version of SemanticMediaWiki at install time: 4.1.1
wfLoadExtension( 'SemanticMediaWiki' );
enableSemantics();
# https://www.semantic-mediawiki.org/wiki/Help:$smwgQMaxInlineLimit

SMW Version 3.2.3

Installation

mwcluster \
  --create \
  --container_name smw3\
  --forceRebuild\
  --version_list 1.35.13\
  --base_port 8482\
  --sql_base_port 10308\
  --mariaDBVersion 10.11\
  --logo https://www.semantic-mediawiki.org/w/images/4/4e/SMW_logo_180.png\
  --smw_version 3.2.3

Check

mwcluster --check --container_name smw3 --version_list 1.35.13

Down

mwcluster --down --container_name smw3 --version_list 1.35.13 -f
running docker compose down for mediawiki versions ['1.35.13']
running docker compose down for smw3 1.35.10 docker application ...
[+] Running 3/3
 ⠿ Volume smw3_wiki-etc    Removed                                0.0s
 ⠿ Volume smw3_mysql-data  Removed                                0.1s
 ⠿ Volume smw3_wiki-www    Removed                                1.2s

Extensions

see Pymediawikidocker/Extensions

Debugging

Docker files

You'll find the files created by pymediawikidocker in ".pymediawikidocker" in your homedirectory. 

pwd
/Users/***/.pymediawikidocker/mw1_35_2
ls -l
total 176
-rw-r--r--  1 wf  staff   1057 22 Jun 07:03 Dockerfile
-rw-r--r--  1 wf  staff   4580 22 Jun 07:03 LocalSettings.php
-rw-r--r--  1 wf  staff   1161 22 Jun 07:03 docker-compose.yml
-rw-r--r--  1 wf  staff    317 22 Jun 07:03 initdb.sh
-rw-r--r--  1 wf  staff    125 22 Jun 07:03 phpinfo.php
-rw-r--r--  1 wf  staff  64515 22 Jun 07:03 wiki.sql

Check installed docker images

docker images | grep mw
mw-142-mw     latest    13e469c7d859   54 minutes ago      1.17GB
mw-141-mw     latest    99578ea06019   55 minutes ago      1.2GB
mw-140-mw     latest    d84a2410dbea   56 minutes ago      1.13GB
mw-139-mw     latest    f83c36dee73d   58 minutes ago      1.06GB
mw-138-mw     latest    8bfb68343139   59 minutes ago      1.03GB
mw-135-mw     latest    bf7ae8fe499c   About an hour ago   1.03GB

Bash into docker containers

# Mediawiki container
docker exec -it mw-142-mw /bin/bash
docker exec -it mw-141-mw /bin/bash
docker exec -it mw-140-mw /bin/bash
docker exec -it mw-139-mw /bin/bash
docker exec -it mw-138-mw /bin/bash
docker exec -it mw-135-mw /bin/bash

# MariaDB container
docker exec -it mw-142-db /bin/bash
docker exec -it mw-141-db /bin/bash
docker exec -it mw-140-db /bin/bash
docker exec -it mw-139-db /bin/bash
docker exec -it mw-138-db /bin/bash
docker exec -it mw-135-db /bin/bash

Monkey patching

# Enable raw html in for the wiki
docker exec  mw1_35_8-mw sh -c "echo '\$wgRawHtml = true;\n' >> /var/www/html/LocalSettings.php"

No space left on device issue

e.g. when starting mariadb container see https://stackoverflow.com/questions/37645879/how-can-i-fix-docker-mac-no-space-left-on-device-error

Wiki Backup

The passwords are available from the docker files e.g. 

grep PASS docker-compose.yml 
      MYSQL_PASSWORD: "TnbKf2zGRmZPjUvz7B"
      MYSQL_ROOT_PASSWORD: "QWehnR9yLMY7YAb9pw"

In a docker bash a dump can be created 

mysqldump wiki -u wikiuser --password=TnbKf2zGRmZPjUvz7B > /tmp/wiki.sql

and then copied to your local machine 

docker cp mw1_36_0_db_1:/tmp/wiki.sql /tmp

Docker cleanup procedure

#!/bin/bash
# WF 2023-04-08
# 
# ⚠️ ⚠️ ⚠️ 
#
#  kids  don't do this at home ...
#
# ⚠️ These commands will stop and remove all running docker containers (not only the ones created pymediawikidocker/the mwcluster command) 
# kill/stop all running containers
docker kill $(docker ps -q)
#docker stop $(docker ps -aq)
# remove all docker containers
docker rm $(docker ps -aq)
# remove all images
# ⚠️ This command will remove all your images not only the ones created by pymediawikidocker (docker will hopefully refuse to do this without the -f option) 
docker rmi $(docker images -q)
# ⚠️ these commands will remove all your volumes - you might loose valuable data if you don't check what your volumes contain before starting this command 
# see https://stackoverflow.com/a/46091568/1497139
docker volume rm $(docker volume ls -qf dangling=true | xargs)
#⚠️ WARNING! This will remove:
#  - all stopped containers
#  - all networks not used by at least one container
#  - all dangling images
#  - all dangling build cache
docker system prune --force

Links

Retrieved from "https://wiki.bitplan.com/index.php?title=Pymediawikidocker&oldid=19141"