Multi-Module Maven with github pages
Wolfgang Fahl
Motivation
The issues
led to the creation of the example project:
- https://github.com/BITPlan/com.bitplan.multimodule which uses
- https://github.com/BITPlan/com.bitplan.pom for the parent pom definition
in which the github maven site plugin is defined with the configuration
<!-- git hub site plugin https://github.com/github/maven-plugins -->
<plugin>
<groupId>com.github.github</groupId>
<artifactId>site-maven-plugin</artifactId>
<version>${site-maven-plugin.version}</version>
<configuration>
<message>Creating site for ${github.owner} ${github.project}
${project.version}</message>
<repositoryName>${github.project}</repositoryName> <!-- github repo name -->
<repositoryOwner>${github.owner}</repositoryOwner> <!-- github username -->
</configuration>
<executions>
<execution>
<goals>
<goal>site</goal>
</goals>
<phase>site</phase>
</execution>
</executions>
</plugin>
Following the suggestion of https://stackoverflow.com/a/19336536/1497139 led to adding
<distributionManagement>
<site>
<id>${project.artifactId}-site</id>
<url>${project.baseUri}</url>
</site>
</distributionManagement>
to the parent pom.xml
mvn clean site
... wait a long while ...
[INFO] Multi-Module ....................................... SUCCESS [04:22 min]
[INFO] Multi-Module 1 ..................................... SUCCESS [04:09 min]
[INFO] Multi-Module 2 ..................................... SUCCESS [04:09 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12:42 min
and you end up with Module 2 overriding the results of the Multi-Module and Module 1 results
Motivation[edit]
The issues
led to the creation of the example project:
- https://github.com/BITPlan/com.bitplan.multimodule which uses
- https://github.com/BITPlan/com.bitplan.pom for the parent pom definition
in which the github maven site plugin is defined with the configuration
<!-- git hub site plugin https://github.com/github/maven-plugins -->
<plugin>
<groupId>com.github.github</groupId>
<artifactId>site-maven-plugin</artifactId>
<version>${site-maven-plugin.version}</version>
<configuration>
<message>Creating site for ${github.owner} ${github.project}
${project.version}</message>
<repositoryName>${github.project}</repositoryName> <!-- github repo name -->
<repositoryOwner>${github.owner}</repositoryOwner> <!-- github username -->
</configuration>
<executions>
<execution>
<goals>
<goal>site</goal>
</goals>
<phase>site</phase>
</execution>
</executions>
</plugin>
Following the suggestion of https://stackoverflow.com/a/19336536/1497139 led to adding
<distributionManagement>
<site>
<id>${project.artifactId}-site</id>
<url>${project.baseUri}</url>
</site>
</distributionManagement>
to the parent pom.xml
mvn clean site
... wait a long while ...
[INFO] Multi-Module ....................................... SUCCESS [04:22 min]
[INFO] Multi-Module 1 ..................................... SUCCESS [04:09 min]
[INFO] Multi-Module 2 ..................................... SUCCESS [04:09 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12:42 min
and you end up with Module 2 overriding the results of the Multi-Module and Module 1 results
Is it a bug or a feature?[edit]
Whats the work-around?[edit]
- Do not use the plugin!
- git clone or pull the gh-pages to a local folder
- mvn site:stage to a temporary staging directory
- rsync the results into the git local folder
- check in the local folder
- have a cup of coffee or drink a beer / be happy ✓
Is it a bug or a feature?[edit]
Whats the work-around?[edit]
- Do not use the plugin!
- git clone or pull the gh-pages to a local folder
- mvn site:stage to a temporary staging directory
- rsync the results into the git local folder
- check in the local folder
- have a cup of coffee or drink a beer / be happy ✓
Do not use the plugin![edit]
- It's slow (as in really slow ...)
- It's not reliable (as in 500 HTML Codes)
- It's not maintained well (as in 57 open issues as of 2018-08
- It's superflous (see below)
- Even the committer's don't use it any more (as in a personal mail I got today ..)
Steps to work around[edit]
git clone or pull the gh-pages to a local folder[edit]
git clone https://github.com/$owner/$project --branch gh-pages --single-branch
mvn site:stage to a temporary staging directory[edit]
mkdir -p /tmp/stage/4site
mvn -U clean install site site:stage -DstagingDirectory=/tmp/stage/4site
[INFO] --- maven-site-plugin:3.7.1:stage (default-cli) @ multimodule2 ---
[INFO] Using this base directory for staging: /tmp/stage/4site
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.multimodule/multimodule-module2/target/site
[INFO] >>> to file:///tmp/stage/4site/../multimodule2/com.bitplan.multimodule/multimodule2
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Multi-Module ....................................... SUCCESS [ 5.423 s]
[INFO] Multi-Module 1 ..................................... SUCCESS [ 4.043 s]
[INFO] Multi-Module 2 ..................................... SUCCESS [ 3.393 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 13.665 s
Yes - thats right seconds not minutes!
rsync the results into the git local folder[edit]
rsync -avz /tmp/stage/com.bitplan.multimodule/* $ghpages/com.bitplan.multimodule
check in the local folder[edit]
in the $gphages/com.bitplan.multimodule folder:
git add *
git commit -m "checking new site in ..."
git push
have a cup of coffee or drink a beer / be happy ✓[edit]
gist of a script[edit]
The following bash createSite function does the core work. To call it set the globals ws variable to your workspace and supply the three arguments e.g.
createSite com.bitplan.multimodule $HOME/Documents/gh-pages "multimodule1 multimodule2"
see the comments of the function below for the description of the three arguments.
createSite bash function[edit]
#
# createSite
# ws: global variable pointing to workspace
# param 1: l_project - project name/directory
# param 2: l_ghpages - directory where gh-pages branch has been git cloned/pulled
# param 3: l_modules - non-empty for a multi-module project (e.g. containing the list of modules)
#
createSite() {
local l_project="$1"
local l_ghpages="$2"
local l_modules="$3"
color_msg $green "creating site for $l_project $l_modules"
cd $ws/$l_project
stage=/tmp/stage$$
sitelog=/tmp/sitelog$$.txt
rm -rf $stage
# the stagingDirectory needs to be subdirectory
mkdir -p $stage/$l_project
# run the staging of the site against this directory and log the results
mvn -U clean install site site:stage -DstagingDirectory=$stage/$l_project | tee $sitelog
# rsync the result into the github-pages folder
rsync -avz --del $stage/* $l_ghpages/$l_project/
# is this a multi module project?
if [ "$l_modules" != "" ]
then
cd $l_ghpages/$l_project/
if [ ! -f index.html ]
then
cat << EOF > index.html
<!DOCTYPE html>
<html>
<head>
<!-- HTML meta refresh URL redirection -->
<meta http-equiv="refresh"
content="0; url=./$l_project/$l_project/index.html">
</head>
<body>
<p>This is a multimodule mvn site click below to get to the index.html of
<a href="./$l_project/$l_project/index.html">$l_project</a></p>
</body>
</html>
EOF
fi
# add potentially new files
git add *
# commit results
git commit -m "checked in by checksite script"
# push results
git push
fi
if [ "$debug" = "false" ]
then
rm -rf $stage
rm $sitelog
fi
}
Speed comparison[edit]
On my computer the plugin takes 18 minutes for the simple com.bitplan.multimodule example
The script needs 25 seconds!
Aggregating Reports[edit]
The most highly rated answer of
suggests
<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-report-plugin</artifactId>
<version>2.4.2</version>
<configuration>
<aggregate>true</aggregate>
<!--also set this to link to generated source reports-->
<linkXRef>true</linkXRef>
</configuration>
</plugin>
</plugins>
</reporting>
while https://maven.apache.org/plugins/maven-site-plugin/examples/configuring-reports.html suggest that you need different reportsets depending on inherited/non inherited pom configurations
describes details of aggregating. Especially that you have to run the mvn site command twice:
mvn verify site
mvn site:site site:stageto get an aggregated result
Experimenting with aggregate settings[edit]
mvn surefire-report:report-only -Daggregate=true
creates a file surefire-report.html in the directory target/site
Zusammenfassung
[Zusammenfassung] [Pakete] [Testfälle]
| Tests | Fehler | Fehlschläge | Ausgelassen | Erfolgsrate | Zeit |
|---|---|---|---|---|---|
| 2 | 0 | 0 | 0 | 100% | 0,002 |
Hinweis: Fehlschläge werden erwartet und durch Behauptungen überprüft während Fehler unerwartet sind.
Pakete
[Zusammenfassung] [Pakete] [Testfälle]
| Paket | Tests | Fehler | Fehlschläge | Ausgelassen | Erfolgsrate | Zeit |
|---|---|---|---|---|---|---|
| com.bitplan.multimodule | 2 | 0 | 0 | 0 | 100% | 0,002 |
Hinweis: Die Paketstatistiken werden nicht rekursiv berechnet, es werden lediglich die Ergebnisse aller enthaltenen Tests aufsummiert.
com.bitplan.multimodule
| Klasse | Tests | Fehler | Fehlschläge | Ausgelassen | Erfolgsrate | Zeit | |
|---|---|---|---|---|---|---|---|
 |
TestMain | 1 | 0 | 0 | 0 | 100% | 0,001 |
|
TestMain | 1 | 0 | 0 | 0 | 100% | 0,001 |
Experimenting with distributionManagement.url settings[edit]
For this we wrote a script to log the results of the mvn site command and analyze where the results go
#!/bin/bash
# WF 2018-08-24
# prepare to log the mvn site command
sitelog=/tmp/sitelog$$.txt
# create an empty staging area
stage=/tmp/stage
rm -rf $stage
mkdir -p $stage/4site
# run the staging of the site against this directory and log the results
mvn -U clean install site site:stage -DstagingDirectory=$stage/4site | tee $sitelog
# what url did we set for the site in distributionManagement?
cat ../com.bitplan.pom/pom.xml | xml2 | grep /project/distributionManagement/site/url
# and where did the results go?
egrep "(Pushing|to file)" $sitelog
${project.baseUri}[edit]
/project/distributionManagement/site/url=${project.baseUri}
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.multimodule/target/site
[INFO] >>> to file:///tmp/stage/4site/../com.bitplan.multimodule/com.bitplan.multimodule
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.multimodule/multimodule-module1/target/site
[INFO] >>> to file:///tmp/stage/4site/../com.bitplan.multimodule/multimodule-module1/com.bitplan.multimodule/multimodule1
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.multimodule/multimodule-module2/target/site
[INFO] >>> to file:///tmp/stage/4site/../com.bitplan.multimodule/multimodule-module2/com.bitplan.multimodule/multimodule2
- the multimodule index file is at /tmp/stage/com.bitplan.multimodule/com.bitplan.multimodule/index.html ✓
- inter-module links work and the main parent module is at /tmp/stage/com.bitplan.pom/index.html ✓
${project.artifactId}[edit]
/project/distributionManagement/site/url=${project.artifactId}
- failed: Base URI is not absolute: com.bitplan.pom -> [Help 1]
https://${github.owner}.github.io/${project.artifactId}[edit]
/project/distributionManagement/site/url=https://${github.owner}.github.io/${project.artifactId}
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.multimodule/target/site
[INFO] >>> to file:///tmp/stage/4site/../com.bitplan.multimodule/com.bitplan.multimodule
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.multimodule/multimodule-module1/target/site
[INFO] >>> to file:///tmp/stage/4site/../multimodule1/com.bitplan.multimodule/multimodule1
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.multimodule/multimodule-module2/target/site
[INFO] >>> to file:///tmp/stage/4site/../multimodule2/com.bitplan.multimodule/multimodule2
Same result as the working result above ..
- the multimodule index file is at /tmp/stage/com.bitplan.multimodule/com.bitplan.multimodule/index.html ✓
- inter-module links work and the main parent module is at /tmp/stage/com.bitplan.pom/index.html ✓
Test with non -module pom based project[edit]
/project/distributionManagement/site/url=https://${github.owner}.github.io/${project.artifactId}
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.fritzbox/target/site
[INFO] >>> to file:///tmp/stage/4site/../com.bitplan.fritzbox/com.bitplan.fritzbox
${project.baseUri}[edit]
/project/distributionManagement/site/url=${project.baseUri}
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.multimodule/target/site
[INFO] >>> to file:///tmp/stage/4site/../com.bitplan.multimodule/com.bitplan.multimodule
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.multimodule/multimodule-module1/target/site
[INFO] >>> to file:///tmp/stage/4site/../com.bitplan.multimodule/multimodule-module1/com.bitplan.multimodule/multimodule1
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.multimodule/multimodule-module2/target/site
[INFO] >>> to file:///tmp/stage/4site/../com.bitplan.multimodule/multimodule-module2/com.bitplan.multimodule/multimodule2
- the multimodule index file is at /tmp/stage/com.bitplan.multimodule/com.bitplan.multimodule/index.html ✓
- inter-module links work and the main parent module is at /tmp/stage/com.bitplan.pom/index.html ✓
${project.artifactId}[edit]
/project/distributionManagement/site/url=${project.artifactId}
- failed: Base URI is not absolute: com.bitplan.pom -> [Help 1]
https://${github.owner}.github.io/${project.artifactId}[edit]
/project/distributionManagement/site/url=https://${github.owner}.github.io/${project.artifactId}
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.multimodule/target/site
[INFO] >>> to file:///tmp/stage/4site/../com.bitplan.multimodule/com.bitplan.multimodule
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.multimodule/multimodule-module1/target/site
[INFO] >>> to file:///tmp/stage/4site/../multimodule1/com.bitplan.multimodule/multimodule1
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.multimodule/multimodule-module2/target/site
[INFO] >>> to file:///tmp/stage/4site/../multimodule2/com.bitplan.multimodule/multimodule2
Same result as the working result above ..
- the multimodule index file is at /tmp/stage/com.bitplan.multimodule/com.bitplan.multimodule/index.html ✓
- inter-module links work and the main parent module is at /tmp/stage/com.bitplan.pom/index.html ✓
Test with non -module pom based project[edit]
/project/distributionManagement/site/url=https://${github.owner}.github.io/${project.artifactId}
[INFO] Pushing /Users/wf/Documents/workspace/com.bitplan.fritzbox/target/site
[INFO] >>> to file:///tmp/stage/4site/../com.bitplan.fritzbox/com.bitplan.fritzbox
Example pom.xml files[edit]
Stackoverflow questions[edit]
site/github pages[edit]
- https://stackoverflow.com/questions/10848715/multi-module-pom-creating-a-site-that-works
- https://stackoverflow.com/questions/21520904/multi-module-example-of-using-mvn-site-deploy-with-github-pages
aggregating reports[edit]
- https://stackoverflow.com/questions/1274523/maven-surefire-reporting-plugin-configuration
- https://stackoverflow.com/questions/41766854/maven-multimodule-project-using-surefire-report-plugin-should-generate-aggregate