Skip to content

Commit 9a1b497

Browse files
borsboomborsboom
committed
Script and Dockerfiles for building fpco/stack-build images (fixes commercialhaskell#624).
This replaces our previous proprietary image building system, which was used for LTS <= 7.x.
1 parent b25c69d commit 9a1b497

4 files changed

Lines changed: 254 additions & 17 deletions

File tree

‎doc/docker_integration.md‎

Lines changed: 3 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -268,19 +268,6 @@ FP Complete provides the following public image repositories on Docker Hub:
268268
- [fpco/stack-build](https://registry.hub.docker.com/u/fpco/stack-build/) (the
269269
default) - GHC (patched), tools (stack, cabal-install, happy, alex, etc.), and
270270
system developer libraries required to build all Stackage packages.
271-
- [fpco/stack-ghcjs-build](https://registry.hub.docker.com/u/fpco/stack-ghcjs-build/) -
272-
Like `stack-build`, but adds GHCJS.
273-
- [fpco/stack-full](https://registry.hub.docker.com/u/fpco/stack-full/) -
274-
Includes all Stackage packages pre-installed in GHC's global package database.
275-
These images are over 10 GB!
276-
- [fpco/stack-ghcjs-full](https://registry.hub.docker.com/u/fpco/stack-ghcjs-full/) -
277-
Like `stack-full`, but adds GHCJS.
278-
- [fpco/stack-run](https://registry.hub.docker.com/u/fpco/stack-run/) -
279-
Runtime environment for binaries built with Stackage. Includes system shared
280-
libraries required by all Stackage packages. Does not necessarily include all
281-
data required for every use (e.g. has texlive-binaries for HaTeX, but does not
282-
include LaTeX fonts), as that would be prohibitively large. Based on
283-
[phusion/baseimage](https://registry.hub.docker.com/u/phusion/baseimage/).
284271

285272
FP Complete also builds custom variants of these images for their clients.
286273

@@ -426,10 +413,9 @@ We recommend using either the `overlay` or `aufs` storage driver for stack, as
426413
they are least likely to give you trouble. On Ubuntu, `aufs` is the default for
427414
new installations, but older installations sometimes used `devicemapper`.
428415

429-
The `devicemapper` storage driver's default configuration limits it to a 10 GB
430-
file system, which the "full" images exceed. We have experienced other
431-
instabilities with it as well on Ubuntu, and recommend against its use for this
432-
purpose.
416+
The `devicemapper` storage driver's doesn't work well with large filesystems,
417+
and we have experienced other instabilities with it as well. We recommend
418+
against its use.
433419

434420
The `btrfs` storage driver has problems running out of metadata space long
435421
before running out of actual disk space, which requires rebalancing or adding

‎etc/dockerfiles/stack-build/README.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
dockerfiles/stack-build
2+
=======================
3+
4+
Build [fpco/stack-build](https://hub.docker.com/r/fpco/stack-build/) Docker
5+
images. This script and its Dockerfiles are used for building images for LTS >=
6+
8.0.
7+
8+
Usage
9+
-----
10+
11+
./build.sh [--push] [--dry-run] lts-X.Y|lts-X|lts]"
12+
13+
Options
14+
-------
15+
16+
`--help`: show this help
17+
18+
`--push`: push the image after building/tagging it
19+
20+
`--dry-run`: print the important commands that will be run, but don't actually
21+
run them
22+
23+
Argument
24+
---------
25+
26+
If you specify and exact LTS version, that image will be built. If you only
27+
specify the major version, then an image for the latest minor version for that
28+
major version will be built. If you specify 'lts', an image for the latest LTS
29+
snapshot is built.
30+
31+
This searches for a Dockerfile for the selected snapshot in
32+
`lts-X.Y/Dockerfile`, and if one isn't found reuses the same image as built the
33+
most recent `lts-X.Y/Dockerfile` found for earlier minor versions of the same
34+
major version.

‎etc/dockerfiles/stack-build/build.sh‎

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
#!/usr/bin/env bash
2+
3+
set -eu
4+
cd "$(dirname "$0")"
5+
6+
#
7+
# Constants and defaults
8+
#
9+
10+
DOCKER_REPO=fpco/stack-build
11+
PUSH=false
12+
DRY=false
13+
14+
#
15+
# Functions
16+
#
17+
18+
# Print usage information and exit with failure status. First argument is an
19+
# error message.
20+
usage() {
21+
echo "$0: $1" >&2
22+
echo
23+
echo "Usage: $0 [--push] [--dry-run] lts-X.Y|lts-X|lts]"
24+
echo "See README.md for more information."
25+
echo
26+
exit 1
27+
}
28+
29+
# Print a command, and if --dry-run disabled also run it
30+
dry() {
31+
echo ">>> $*"
32+
[[ $DRY = true ]] || "$@"
33+
}
34+
35+
# Push an image if --push is enabled (otherwise do nothing)
36+
push() {
37+
[[ $PUSH = false ]] || dry docker push "$1"
38+
}
39+
40+
# Tag an image, and then push it if --push is enabled
41+
tagpush() {
42+
dry docker tag "$1" "$2"
43+
push "$2"
44+
}
45+
46+
#
47+
# Parse command-line
48+
#
49+
50+
LTS_SLUG_ARG=
51+
while [[ $# -gt 0 ]]; do
52+
case "$1" in
53+
--push)
54+
PUSH=true
55+
shift
56+
;;
57+
--dry-run)
58+
DRY=true
59+
shift
60+
;;
61+
-*)
62+
usage "Unknown option: $1"
63+
;;
64+
*)
65+
if [[ -n "$LTS_SLUG_ARG" ]]; then
66+
usage "Cannot specify multiple snaphots: $1"
67+
fi
68+
LTS_SLUG_ARG="$1"
69+
shift
70+
;;
71+
esac
72+
done
73+
74+
#
75+
# Determine actual snapshot version from aliases
76+
#
77+
78+
SNAPSHOTS="$(mktemp "lts-snapshots.json.XXXXXX")"
79+
trap "rm -f \"$SNAPSHOTS\"" EXIT
80+
wget -qO- https://www.stackage.org/download/lts-snapshots.json >"$SNAPSHOTS"
81+
82+
case "$LTS_SLUG_ARG" in
83+
"")
84+
usage "Missing argument: snapshot or alias"
85+
;;
86+
lts-*.*)
87+
LTS_SLUG="$LTS_SLUG_ARG"
88+
;;
89+
*)
90+
LTS_SLUG=$(jq -r ".[\"$LTS_SLUG_ARG\"]" "$SNAPSHOTS")
91+
if [[ -z "$LTS_SLUG" || "$LTS_SLUG" = "null" ]]; then
92+
echo "$0: Cannot find LTS version for slug: $LTS_SLUG_ARG" >&2
93+
exit 1
94+
fi
95+
;;
96+
esac
97+
98+
LTS_VERSION="${LTS_SLUG#lts-}"
99+
LTS_MAJOR="${LTS_VERSION%.*}"
100+
LTS_MINOR="${LTS_VERSION#*.}"
101+
102+
#
103+
# Determine latest LTS version
104+
#
105+
106+
LATEST_LTS_SLUG=$(jq -r ".[\"lts\"]" $SNAPSHOTS)
107+
LATEST_LTS_VERSION="${LATEST_LTS_SLUG#lts-}"
108+
LATEST_LTS_MAJOR="${LATEST_LTS_VERSION%.*}"
109+
LATEST_LTS_MINOR="${LATEST_LTS_VERSION#*.}"
110+
111+
#
112+
# Determine latest minor version of the selected major version
113+
#
114+
115+
MAJOR_LATEST_LTS_SLUG=$(jq -r ".[\"lts-$LTS_MAJOR\"]" $SNAPSHOTS)
116+
MAJOR_LATEST_LTS_VERSION="${MAJOR_LATEST_LTS_SLUG#lts-}"
117+
MAJOR_LATEST_LTS_MAJOR="${MAJOR_LATEST_LTS_VERSION%.*}"
118+
MAJOR_LATEST_LTS_MINOR="${MAJOR_LATEST_LTS_VERSION#*.}"
119+
120+
#
121+
# Find the Dockerfile for the selected snapshot
122+
#
123+
124+
if [[ -s "$LTS_SLUG/Dockerfile" ]]; then
125+
126+
# If there is an exact match, build and push that image
127+
128+
dry docker build -t "$DOCKER_REPO:$LTS_SLUG" --build-arg "LTS_SLUG=$LTS_SLUG" $LTS_SLUG
129+
push "$DOCKER_REPO:$LTS_SLUG"
130+
else
131+
132+
# If no exact match, find a dockerfile for any earlier minor version of the
133+
# selected major version, and just create a new tag rom version's image with the selected
134+
# minor version (assuming that nothing needs to change), and push it.
135+
136+
minor=$(( LTS_MINOR - 1 ))
137+
while [[ ! -s "lts-$LTS_MAJOR.$minor/Dockerfile" && $minor -ge 0 ]]; do
138+
minor=$(( minor - 1 ))
139+
done
140+
if [[ $minor -lt 0 ]]; then
141+
echo "$0: Cannot find any Dockerfile for LTS major version" >&2
142+
exit 1
143+
fi
144+
dry docker pull "$DOCKER_REPO:lts-$LTS_MAJOR.$minor" || true
145+
tagpush "$DOCKER_REPO:lts-$LTS_MAJOR.$minor" "$DOCKER_REPO:$LTS_SLUG"
146+
fi
147+
148+
#
149+
# Create and push additional tags
150+
#
151+
152+
# If we select the latest minor version for the selected major version, then
153+
# also create and push an 'lts-X' tag.
154+
if [[ "$MAJOR_LATEST_LTS_VERSION" = "$LTS_VERSION" ]]; then
155+
tagpush "$DOCKER_REPO:$LTS_SLUG" "$DOCKER_REPO:lts-$LTS_MAJOR"
156+
fi
157+
158+
# If we selected the latest LTS snapshot, also create and push the 'lts' and 'latest' tags.
159+
if [[ "$LATEST_LTS_VERSION" = "$LTS_VERSION" ]]; then
160+
tagpush "$DOCKER_REPO:$LTS_SLUG" "$DOCKER_REPO:lts"
161+
tagpush "$DOCKER_REPO:$LTS_SLUG" "$DOCKER_REPO:latest"
162+
fi

‎etc/dockerfiles/stack-build/lts-8.0/Dockerfile‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
FROM ubuntu:16.04
2+
3+
MAINTAINER Emanuel Borsboom <manny@fpcomplete.com>
4+
5+
ARG GHC_VERSION=8.0.2
6+
ARG LTS_SLUG=lts-8.0
7+
ARG PID1_VERSION=0.1.0.1
8+
ARG DEBIAN_FRONTEND=noninteractive
9+
10+
#
11+
# Set encoding to UTF-8 and PATH to find GHC and cabal/stack-installed binaries.
12+
#
13+
14+
ENV LANG=C.UTF-8 \
15+
LC_ALL=C.UTF-8 \
16+
PATH=/root/.cabal/bin:/root/.local/bin:/opt/ghc/$GHC_VERSION/bin:$PATH
17+
18+
#
19+
# Use Stackage's debian-bootstrap.sh script to install system libraries and
20+
# tools required to build any Stackage package.
21+
#
22+
23+
#@@@ TEST documentation location
24+
RUN apt-get update && \
25+
apt-get install -y wget && \
26+
wget -qO- https://raw.githubusercontent.com/fpco/stackage/93738c5865fdd0627508617215d01b78d4e2bf74/debian-bootstrap.sh | bash && \
27+
ln -s ghc /opt/ghc/$GHC_VERSION/share/doc/ghc-$GHC_VERSION && \
28+
rm -rf /var/lib/apt/lists/*
29+
30+
#
31+
# Use 'stack' to install basic Haskell tools like alex, happy, and cpphs. We
32+
# remove most of the STACK_ROOT afterward to save space, but keep the 'share'
33+
# files that some of these tools require.
34+
#
35+
36+
RUN stack --system-ghc --resolver=$LTS_SLUG --local-bin-path=/usr/local/bin install \
37+
cabal-install happy alex cpphs gtk2hs-buildtools hscolour && \
38+
cd $HOME/.stack && \
39+
find . -type f -not -path './snapshots/*/share/*' -exec rm '{}' \; && \
40+
find . -type d -print0 |sort -rz |xargs -0 rmdir 2>/dev/null || true
41+
42+
#
43+
# Install proper 'pid1' init daemon
44+
#
45+
46+
RUN wget -O- "https://github.com/fpco/pid1/releases/download/pid1%2F$PID1_VERSION/pid1-$PID1_VERSION-linux-x86_64.tar.gz" |tar xzf - -C /usr/local
47+
48+
#
49+
# Set up pid1 entrypoint and default command
50+
#
51+
52+
ENTRYPOINT ["/usr/local/sbin/pid1"]
53+
CMD ["bash"]
54+
55+
#@@@: update stack docks to omit run and full images

0 commit comments

Comments
 (0)