This directory contains scripts to help manage redirects for LocalStack's docs migration from:
docs.localstack.cloud→docs.localstack.cloud/aws/snowflake.localstack.cloud→docs.localstack.cloud/snowflake/
| File | Purpose |
|---|---|
scrap_sitemap.py |
Extract URLs from XML sitemaps |
generate_config_template.py |
Generate JSON template from scraped URLs |
redirects_config.json |
Manual redirect mappings (you edit this) |
generate_redirects.py |
Convert JSON config to CloudFlare _redirects format |
test_redirects.py |
Test redirects against staging environment |
First, get all URLs from your current sites:
# Get AWS docs URLs
python scrap_sitemap.py > aws_urls.txt
# Or for Snowflake (modify the script URL)
# Edit scrap_sitemap.py to change the sitemap URL to snowflake.localstack.cloudCreate a template JSON config with all URLs:
# From sitemaps directly
python generate_config_template.py \
--aws-sitemap "https://docs.localstack.cloud/sitemap.xml" \
--snowflake-sitemap "https://snowflake.localstack.cloud/sitemap.xml" \
--output redirects_config_template.json
# Or from saved URL files
python generate_config_template.py \
--aws-file aws_urls.txt \
--snowflake-file snowflake_urls.txt \
--output redirects_config_template.jsonEdit the generated template to map old URLs to new URLs:
{
"aws": [
{
"old_link": "https://docs.localstack.cloud/getting-started/",
"new_link": "https://docs.localstack.cloud/aws/getting-started/",
"status_code": 301
}
],
"snowflake": [
{
"old_link": "https://snowflake.localstack.cloud/features/",
"new_link": "https://docs.localstack.cloud/snowflake/features/",
"status_code": 301
}
]
}Important:
- Remove the
_notefields after manual review - Update
new_linkvalues to match your actual new URL structure - Save as
redirects_config.json
Convert your JSON config to CloudFlare's _redirects format:
python3 generate_redirects.py \
--config redirects_config.json \
--output _redirectsThis creates a _redirects file like:
# LocalStack Docs Redirects
# Generated automatically from redirects_config.json
# Static redirects
/getting-started/ /aws/getting-started/ 301
/user-guide/ /aws/capabilities/ 301
Test your redirects against the staging environment:
python3 test_redirects.py \
--config redirects_config.json \
--staging-url "https://localstack-docs.pages.dev/" \
--report redirect_test_report.mdThis will:
- Test each redirect against your staging URL
- Show pass/fail results in real-time
- Generate a detailed markdown report
Generate a JSON template from scraped URLs.
python generate_config_template.py --helpOptions:
--aws-sitemap URL- Load AWS URLs from sitemap--snowflake-sitemap URL- Load Snowflake URLs from sitemap--aws-file FILE- Load AWS URLs from text file--snowflake-file FILE- Load Snowflake URLs from text file--output FILE- Output template file
Convert JSON config to CloudFlare _redirects format.
python generate_redirects.py --helpOptions:
--config FILE- JSON config file (default:redirects_config.json)--output FILE- Output redirects file (default:_redirects)
Test redirects against staging environment.
python test_redirects.py --helpOptions:
--config FILE- JSON config file--staging-url URL- Staging base URL--timeout SECONDS- Request timeout--report FILE- Save detailed report to file
- Upload the generated
_redirectsfile to your CloudFlare Pages project root - CloudFlare will automatically apply the redirects
- Test with your staging URL first before going live
CloudFlare Limits:
- Max 2,000 static redirects
- Max 100 dynamic redirects
- Max 2,100 total redirects
- 1,000 character limit per redirect
- Staging First: Always test with staging URL
- Sample Testing: Test a subset of critical URLs manually
- Automated Testing: Run the test script regularly
- Monitor: Check for 404s after going live
# 1. Generate template from sitemaps
python generate_config_template.py \
--aws-sitemap "https://docs.localstack.cloud/sitemap.xml" \
--snowflake-sitemap "https://snowflake.localstack.cloud/sitemap.xml"
# 2. Manually edit redirects_config_template.json → redirects_config.json
# 3. Generate CloudFlare redirects file
python generate_redirects.py
# 4. Test redirects
python test_redirects.py --report test_results.md
# 5. Upload _redirects file to CloudFlare Pages- Manual Review Required: Always manually review and update the generated mappings
- URL Structure: Ensure new URLs match your actual site structure
- Status Codes: Use 301 for permanent redirects, 302 for temporary
- Testing: Test thoroughly on staging before production
- Staging URL: Update staging URL as needed in commands
Common Issues:
-
Import Error: Make sure
requestsandbeautifulsoup4are installed:pip install requests beautifulsoup4
-
Sitemap Not Found: Check sitemap URLs are correct and accessible
-
Redirect Not Working:
- Verify the
_redirectsfile is in the correct location - Check CloudFlare Pages build logs
- Ensure paths match exactly
- Verify the
-
Too Many Redirects: Check for redirect loops in your mappings