Skip to content

Commit 690d05e

Browse files
TristanSpeakEasyTristanSpeakEasy
authored
chore: improve test coverage and reporting (#61)
1 parent 0bd0aca commit 690d05e

12 files changed

Lines changed: 1846 additions & 50 deletions

File tree

‎.github/scripts/README.md‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# CI Scripts
2+
3+
This directory contains scripts used by GitHub Actions workflows.
4+
5+
## format-coverage.sh
6+
7+
Formats Go coverage output as a markdown table with color-coded indicators.
8+
9+
### Usage
10+
11+
```bash
12+
./format-coverage.sh <coverage-file> [current-coverage] [main-coverage]
13+
```
14+
15+
**Arguments:**
16+
- `coverage-file`: Path to the Go coverage file (typically `coverage.out`)
17+
- `current-coverage`: (optional) Overall coverage percentage for display (e.g., "74.3%")
18+
- `main-coverage`: (optional) Main branch coverage percentage for comparison (e.g., "74.0%")
19+
20+
### Examples
21+
22+
**Basic usage:**
23+
```bash
24+
# Generate coverage file first
25+
mise test-coverage
26+
27+
# Format the coverage report
28+
./.github/scripts/format-coverage.sh coverage.out
29+
```
30+
31+
**With coverage percentages:**
32+
```bash
33+
# Calculate coverage
34+
COVERAGE=$(go tool cover -func=coverage.out | grep total | awk '{print $3}')
35+
36+
# Format with coverage display
37+
./.github/scripts/format-coverage.sh coverage.out "$COVERAGE"
38+
```
39+
40+
**With comparison to main:**
41+
```bash
42+
./.github/scripts/format-coverage.sh coverage.out "75.2%" "74.3%"
43+
```
44+
45+
### Output Format
46+
47+
The script generates a markdown report with:
48+
- Overall coverage statistics
49+
- Coverage comparison (if main branch coverage provided)
50+
- Table of coverage by package with color-coded indicators:
51+
- 🟢 Green: ≥90% coverage
52+
- 🟡 Yellow: ≥75% coverage
53+
- 🟠 Orange: ≥50% coverage
54+
- 🔴 Red: <50% coverage
55+
- Collapsible detailed coverage by function
56+
57+
### Local Testing
58+
59+
To test the output locally:
60+
61+
```bash
62+
# Run tests with coverage
63+
mise test-coverage
64+
65+
# Format and preview the output
66+
./.github/scripts/format-coverage.sh coverage.out "74.3%" "74.3%" | less
67+
```
68+
69+
Or save to a file for inspection:
70+
71+
```bash
72+
./.github/scripts/format-coverage.sh coverage.out "74.3%" > coverage-report.md

‎.github/scripts/format-coverage.sh‎

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
#!/usr/bin/env bash
2+
set -euo pipefail
3+
4+
# Script to format Go coverage output as a markdown table
5+
# Usage: ./format-coverage.sh coverage.out [current-coverage] [main-coverage]
6+
7+
COVERAGE_FILE="${1:-coverage.out}"
8+
CURRENT_COVERAGE="${2:-}"
9+
MAIN_COVERAGE="${3:-}"
10+
11+
if [ ! -f "$COVERAGE_FILE" ]; then
12+
echo "Error: Coverage file '$COVERAGE_FILE' not found"
13+
exit 1
14+
fi
15+
16+
# Start markdown output
17+
echo "## 📊 Test Coverage Report"
18+
echo ""
19+
20+
# Show current and main coverage if provided
21+
if [ -n "$CURRENT_COVERAGE" ]; then
22+
echo "**Current Coverage:** \`$CURRENT_COVERAGE\`"
23+
24+
if [ -n "$MAIN_COVERAGE" ]; then
25+
echo "**Main Branch Coverage:** \`$MAIN_COVERAGE\`"
26+
echo ""
27+
28+
# Calculate difference
29+
CURRENT_NUM=$(echo "$CURRENT_COVERAGE" | sed 's/%//')
30+
MAIN_NUM=$(echo "$MAIN_COVERAGE" | sed 's/%//')
31+
32+
if [ -n "$CURRENT_NUM" ] && [ -n "$MAIN_NUM" ]; then
33+
DIFF=$(echo "$CURRENT_NUM - $MAIN_NUM" | bc -l 2>/dev/null || echo "0")
34+
35+
if [ "$(echo "$DIFF > 0" | bc -l 2>/dev/null || echo "0")" = "1" ]; then
36+
echo "**Coverage Change:** 📈 +${DIFF}% (improved)"
37+
elif [ "$(echo "$DIFF < 0" | bc -l 2>/dev/null || echo "0")" = "1" ]; then
38+
DIFF_ABS=$(echo "$DIFF * -1" | bc -l 2>/dev/null || echo "${DIFF#-}")
39+
echo "**Coverage Change:** 📉 -${DIFF_ABS}% (decreased)"
40+
else
41+
echo "**Coverage Change:** ✅ No change"
42+
fi
43+
fi
44+
fi
45+
fi
46+
47+
echo ""
48+
echo "### Coverage by Package"
49+
echo ""
50+
51+
# Create table header
52+
echo "| Package | Coverage |"
53+
echo "|---------|----------|"
54+
55+
# Parse coverage and group by package
56+
go tool cover -func="$COVERAGE_FILE" | grep -E '\.go:[0-9]+:' | \
57+
awk -F: '{
58+
# Extract package path from filename
59+
split($1, parts, "/");
60+
pkg = "";
61+
for(i=1; i<length(parts); i++) {
62+
if(pkg != "") pkg = pkg "/";
63+
pkg = pkg parts[i];
64+
}
65+
66+
# Extract coverage percentage from the last field
67+
split($0, line, /[[:space:]]+/);
68+
coverage = line[length(line)];
69+
70+
# Store coverage by package
71+
if(pkg in packages) {
72+
packages[pkg] = packages[pkg] "," coverage;
73+
} else {
74+
packages[pkg] = coverage;
75+
}
76+
}
77+
END {
78+
for(pkg in packages) {
79+
# Calculate average coverage for package
80+
split(packages[pkg], covs, ",");
81+
sum = 0;
82+
count = 0;
83+
for(i in covs) {
84+
gsub(/%/, "", covs[i]);
85+
sum += covs[i];
86+
count++;
87+
}
88+
avg = (count > 0) ? sum/count : 0;
89+
90+
# Format package name (remove common prefix)
91+
display_pkg = pkg;
92+
gsub(/github\.com\/speakeasy-api\/openapi\//, "", display_pkg);
93+
94+
# Add emoji indicators based on coverage level
95+
emoji = "🔴";
96+
if(avg >= 90) emoji = "🟢";
97+
else if(avg >= 75) emoji = "🟡";
98+
else if(avg >= 50) emoji = "🟠";
99+
100+
# Output with coverage value for sorting
101+
printf "%.1f|`%s`|%s\n", avg, display_pkg, emoji;
102+
}
103+
}' | sort -n | awk -F'|' '{
104+
# Re-format after sorting by coverage
105+
printf "| %s | %s %.1f%% |\n", $2, $3, $1;
106+
}'
107+
108+
echo ""
109+
echo "<details>"
110+
echo "<summary>📋 Detailed Coverage by Function (click to expand)</summary>"
111+
echo ""
112+
echo "\`\`\`"
113+
go tool cover -func="$COVERAGE_FILE"
114+
echo "\`\`\`"
115+
echo "</details>"
116+
echo ""
117+
echo "- 🧪 All tests passed"
118+
echo "- 📈 Full coverage report available in workflow artifacts"
119+
echo ""
120+
echo "_Generated by GitHub Actions_"

‎.github/workflows/ci.yaml‎

Lines changed: 3 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -190,47 +190,11 @@ jobs:
190190
- name: Generate coverage summary
191191
id: coverage-summary
192192
run: |
193-
echo "## 📊 Test Coverage Report" > coverage-summary.md
194-
echo "" >> coverage-summary.md
195-
196-
# Current coverage
193+
# Use the formatting script
197194
CURRENT_COV="${{ steps.coverage.outputs.coverage }}"
198-
echo "**Current Coverage:** \`$CURRENT_COV\`" >> coverage-summary.md
199-
200-
# Compare with main if this is a PR
201-
if [ "${{ github.event_name }}" = "pull_request" ] && [ "${{ steps.main-coverage.outputs.main-coverage }}" != "" ]; then
202-
MAIN_COV="${{ steps.main-coverage.outputs.main-coverage }}"
203-
echo "**Main Branch Coverage:** \`$MAIN_COV\`" >> coverage-summary.md
204-
echo "" >> coverage-summary.md
205-
206-
# Calculate difference
207-
CURRENT_NUM=$(echo $CURRENT_COV | sed 's/%//')
208-
MAIN_NUM=$(echo $MAIN_COV | sed 's/%//')
209-
210-
if [ "$CURRENT_NUM" != "" ] && [ "$MAIN_NUM" != "" ]; then
211-
DIFF=$(echo "$CURRENT_NUM - $MAIN_NUM" | bc -l 2>/dev/null || echo "0")
212-
213-
if [ "$(echo "$DIFF > 0" | bc -l 2>/dev/null)" = "1" ]; then
214-
echo "**Coverage Change:** 📈 +${DIFF}% (improved)" >> coverage-summary.md
215-
elif [ "$(echo "$DIFF < 0" | bc -l 2>/dev/null)" = "1" ]; then
216-
DIFF_ABS=$(echo "$DIFF * -1" | bc -l 2>/dev/null || echo "${DIFF#-}")
217-
echo "**Coverage Change:** 📉 -${DIFF_ABS}% (decreased)" >> coverage-summary.md
218-
else
219-
echo "**Coverage Change:** ✅ No change" >> coverage-summary.md
220-
fi
221-
fi
222-
fi
195+
MAIN_COV="${{ steps.main-coverage.outputs.main-coverage }}"
223196
224-
echo "" >> coverage-summary.md
225-
echo "### Coverage by Package" >> coverage-summary.md
226-
echo "\`\`\`" >> coverage-summary.md
227-
go tool cover -func=coverage.out >> coverage-summary.md
228-
echo "\`\`\`" >> coverage-summary.md
229-
echo "" >> coverage-summary.md
230-
echo "- 🧪 All tests passed" >> coverage-summary.md
231-
echo "- 📈 Full coverage report available in workflow artifacts" >> coverage-summary.md
232-
echo "" >> coverage-summary.md
233-
echo "_Generated by GitHub Actions_" >> coverage-summary.md
197+
./.github/scripts/format-coverage.sh coverage.out "$CURRENT_COV" "$MAIN_COV" > coverage-summary.md
234198
235199
- name: Upload coverage artifact
236200
uses: actions/upload-artifact@v4

‎AGENTS.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -218,6 +218,7 @@ import (
218218
- Use `assert.Equal()` for value comparisons with descriptive messages
219219
- Use `assert.Nil()` and `assert.NotNil()` for pointer checks
220220
- Use `require.*` when the test should stop on failure (e.g., setup operations)
221+
- **Use `require.Error()` for error assertions** - The linter enforces this via testifylint
221222
- **Always include descriptive error messages**
222223

223224
```go

0 commit comments

Comments
 (0)