The Fetchers Module is a Go package designed to fetch IP ranges for various cloud providers and services. It is part of the Caddy Defender project, which provides middleware for blocking or manipulating traffic based on IP ranges. This module supports fetching IP ranges from AWS, Google Cloud Platform (GCP), OpenAI, GitHub Copilot, and more.

• Multiple Cloud Providers: Fetch IP ranges for AWS, GCP, OpenAI, GitHub Copilot, and other services.
• Region-Specific IP Ranges: Fetch IP ranges for specific AWS regions (e.g., us-east-1, eu-west-1).
• Dynamic Fetching: IP ranges are fetched dynamically from official sources (e.g., AWS, GCP).
• Pregenerated Results: Use pregenerated IP ranges from the ranges/data directory for faster setup.
• Extensible: Easily add new fetchers for additional services or providers.
• Concurrency: Fetch IP ranges concurrently for improved performance.

Please see the readme

For convenience, pregenerated IP ranges are available in the ranges/data directory. These ranges are generated by running the fetchers and can be used directly in your project without needing to fetch them dynamically.

To use the pregenerated IP ranges, import the data package and access the IPRanges map:

package main

import (
	"fmt"
	"pkg.jsn.cam/caddy-defender/ranges/data"
)

func main() {
	// Access pregenerated IP ranges for AWS
	awsRanges := data.IPRanges["aws"]
	fmt.Println("AWS IP ranges:", awsRanges)

	// Access pregenerated IP ranges for GCP
	gcloudRanges := data.IPRanges["gcloud"]
	fmt.Println("GCP IP ranges:", gcloudRanges)

	// Access pregenerated IP ranges for OpenAI
	openaiRanges := data.IPRanges["openai"]
	fmt.Println("OpenAI IP ranges:", openaiRanges)

	// Access pregenerated IP ranges for Mistral
	mistralRanges := data.IPRanges["mistral"]
	fmt.Println("Mistral IP ranges:", mistralRanges)
    // ...
}

The IPRanges map in the data package contains the following keys:

To regenerate the pregenerated results, run the main.go file in the ranges directory:

cd ranges
go run main.go

This will fetch the latest IP ranges from all supported services and update the generated.go file in the data directory.

To use the Fetchers Module in your Go project, install it using go get:

go get github.com/jasonlovesdoggo/caddy-defender/ranges/fetchers

To fetch IP ranges for a specific service, create an instance of the corresponding fetcher and call the FetchIPRanges method:

package main

import (
	"fmt"
	"github.com/jasonlovesdoggo/caddy-defender/ranges/fetchers"
)

func main() {
	// Fetch global AWS IP ranges
	awsFetcher := fetchers.AWSFetcher{}
	ranges, err := awsFetcher.FetchIPRanges()
	if err != nil {
		fmt.Println("Error fetching AWS IP ranges:", err)
	} else {
		fmt.Println("AWS IP ranges:", ranges)
	}

	// Fetch GCP IP ranges
	gcloudFetcher := fetchers.GCloudFetcher{}
	ranges, err = gcloudFetcher.FetchIPRanges()
	if err != nil {
		fmt.Println("Error fetching GCP IP ranges:", err)
	} else {
		fmt.Println("GCP IP ranges:", ranges)
	}
}

The Fetchers Module is integrated into the Caddy Defender middleware. To use it, configure your Caddyfile with the defender directive:

localhost:8080 {
    defender block {
        ranges aws gcloud openai mistral
    }
    respond "Hello, world!"
}

This configuration blocks requests from IP ranges associated with AWS, GCP, OpenAI, and Mistral.

To add a new fetcher for a service or provider:

1. Create a New Fetcher:

   • Create a new file in the fetchers directory (e.g., my_service.go).

   • Implement the IPRangeFetcher interface:

     package fetchers
     
     import (
         "fmt"
     )
     
     // MyServiceFetcher implements the IPRangeFetcher interface for MyService.
     type MyServiceFetcher struct{}
     
     func (f MyServiceFetcher) Name() string {
         return "MyService"
     }
     
     func (f MyServiceFetcher) Description() string {
         return "Fetches IP ranges for MyService."
     }
     
     func (f MyServiceFetcher) FetchIPRanges() ([]string, error) {
         // Fetch IP ranges for MyService
         return []string{"203.0.113.0/24", "198.51.100.0/24"}, nil
     }

2. Add the Fetcher to main.go:

   • Update the fetchersList in main.go to include your new fetcher:

     fetchersList := []fetchers.IPRangeFetcher{
         fetchers.AWSFetcher{},
         fetchers.GCloudFetcher{},
         fetchers.MyServiceFetcher{}, // Add your new fetcher here
     }

3. Rebuild and Test:

   • Rebuild the project and test the new fetcher to ensure it works as expected.

Fetches global IP ranges for AWS services:

awsFetcher := fetchers.AWSFetcher{}
ranges, err := awsFetcher.FetchIPRanges()

Fetches IP ranges for a specific AWS region (e.g., us-east-1):

awsRegionFetcher := fetchers.AWSRegionFetcher{Region: "us-east-1"}
ranges, err := awsRegionFetcher.FetchIPRanges()

Fetches IP ranges for Google Cloud Platform (GCP):

gcloudFetcher := fetchers.GCloudFetcher{}
ranges, err := gcloudFetcher.FetchIPRanges()

We welcome contributions! If you want to add new fetchers or improve existing ones, follow these steps:

1. Fork the Repository:

   • Fork the Caddy Defender repository.

2. Create a New Branch:

   • Create a branch for your changes:

     git checkout -b my-new-fetcher

3. Make Your Changes:

   • Add your new fetcher or make improvements to existing code.

4. Test Your Changes:

   • Run the tests and ensure everything works as expected.

5. Submit a Pull Request:

   • Submit a pull request with a description of your changes.

This project is licensed under the MIT License. See the LICENSE file for details.

• Caddy Server: Built with ❤️ using Caddy.
• AWS and GCP: For providing publicly accessible IP range data.
• OpenAI and GitHub: For their IP range documentation.