A production-ready REST API built with FastAPI that provides dynamic CRUD operations for any MySQL or PostgreSQL database table. The API features:
- β Multi-Database Support: MySQL (via aiomysql) or PostgreSQL (via asyncpg)
- β Dynamic CRUD Operations: CREATE, READ, UPDATE, DELETE on any table
- β Resiliency Engine: Automatic exponential backoff retry logic (200ms β 1600ms)
- β CSV Operations: Batch import/export for data management
- β SQL Safety: Parameterized queries prevent SQL injection
- β Comprehensive Error Handling: Standardized JSON error responses
- β Async/Await: High-performance asynchronous operations
- β CORS Enabled: Cross-origin resource sharing support
- β Docker Ready: Containerized deployment support
| Method | Endpoint | Description |
|---|---|---|
| POST | /api/{tableName} |
CREATE a new record |
| GET | /api/{tableName}/query |
READ all records |
| POST | /api/{tableName}/query |
READ filtered records |
| PUT | /api/{tableName} |
UPDATE existing records |
| DELETE | /api/{tableName} |
DELETE records |
| POST | /api/batch/import |
CSV bulk import |
| GET | /api/batch/{tableName}/export |
CSV export |
/app/
βββ core/ # Business logic layer
β βββ resiliency.py # Retry engine with exponential backoff
β βββ database.py # Query execution & parameterized statements
β βββ config.py # Database configuration (MySQL/PostgreSQL)
βββ routes/ # HTTP endpoint handlers
β βββ crud.py # CREATE, READ, UPDATE, DELETE endpoints
β βββ csv_routes.py # CSV import/export endpoints
βββ services/ # Business logic utilities
β βββ csv_service.py # CSV processing & validation
βββ schemas/ # Pydantic data models
β βββ requests.py # Request body validation
βββ utils/ # Helper functions
βββ error_handler.py # Standardized response formatting
main.py # FastAPI application entry point
requirements.txt # Python dependencies
Dockerfile # Docker container configuration
deploy.sh # Deployment automation script
test_main.py # Unit tests
test_comprehensive.py # Comprehensive test suite
# Clone the repository
git clone https://github.com/burkaslarry/python_mysql_http.git
cd python_mysql_http
# Create virtual environment
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Install dependencies
pip install -r requirements.txtFor MySQL:
cat > .env << EOF
# Database Type: mysql or postgresql
DB_TYPE=mysql
# MySQL Configuration
DB_HOST=localhost
DB_PORT=3306
DB_USER=root
DB_PASSWORD=your_password
DB_NAME=test_db
EOFFor PostgreSQL:
cat > .env << EOF
# Database Type: mysql or postgresql
DB_TYPE=postgresql
# PostgreSQL Configuration
DB_POSTGRES_HOST=localhost
DB_POSTGRES_PORT=5432
DB_POSTGRES_USER=postgres
DB_POSTGRES_PASSWORD=your_password
DB_POSTGRES_NAME=test_db
EOF# For MySQL: Create test database
mysql -u root -p -e "CREATE DATABASE test_db;"
# For PostgreSQL: Create test database
psql -U postgres -c "CREATE DATABASE test_db;"
# Start the API server
python main.py
# or with auto-reload:
uvicorn main:app --reload --host 0.0.0.0 --port 8000# Test the API
curl http://localhost:8000/
# Expected response: {"message": "Hello, World"}# .env file configuration
cat > .env << EOF
DB_TYPE=mysql
DB_HOST=mydb.xxxxxxxxxxxx.us-east-1.rds.amazonaws.com
DB_PORT=3306
DB_USER=admin
DB_PASSWORD=YourSecurePassword123
DB_NAME=production_db
EOFAWS RDS Setup Steps:
- Go to AWS RDS Console β Create Database β MySQL
- Choose Multi-AZ for high availability
- Set Master username:
admin - Set Master password:
YourSecurePassword123 - Create security group allowing port 3306
- Copy endpoint to
DB_HOSTin.env
| Provider | Setup Link | Cost |
|---|---|---|
| DigitalOcean | https://www.digitalocean.com/products/managed-databases | $15/month |
| Google Cloud SQL | https://cloud.google.com/sql/docs/mysql | $3.50/month |
| Azure Database | https://azure.microsoft.com/en-us/services/mysql/ | $20/month |
| Aiven | https://aiven.io/mysql | Free tier + $10/month |
# .env file configuration
cat > .env << EOF
DB_TYPE=postgresql
DB_POSTGRES_HOST=mydb.xxxxxxxxxxxx.us-east-1.rds.amazonaws.com
DB_POSTGRES_PORT=5432
DB_POSTGRES_USER=postgres
DB_POSTGRES_PASSWORD=YourSecurePassword123
DB_POSTGRES_NAME=production_db
EOFAWS RDS Setup Steps:
- Go to AWS RDS Console β Create Database β PostgreSQL
- Choose Multi-AZ for high availability
- Set Master username:
postgres - Set Master password:
YourSecurePassword123 - Create security group allowing port 5432
- Copy endpoint to
DB_POSTGRES_HOSTin.env
| Provider | Setup Link | Cost |
|---|---|---|
| DigitalOcean | https://www.digitalocean.com/products/managed-databases | $15/month |
| Heroku PostgreSQL | https://www.heroku.com/postgres | Free tier + $9/month |
| Google Cloud SQL | https://cloud.google.com/sql/docs/postgres | $3.50/month |
| Aiven | https://aiven.io/postgresql | Free tier + $10/month |
| Supabase | https://supabase.com | Free tier included |
Complete .env Example:
# Database Selection
DB_TYPE=mysql # mysql or postgresql
# MySQL Settings (if DB_TYPE=mysql)
DB_HOST=localhost
DB_PORT=3306
DB_USER=root
DB_PASSWORD=password
DB_NAME=test_db
# PostgreSQL Settings (if DB_TYPE=postgresql)
DB_POSTGRES_HOST=localhost
DB_POSTGRES_PORT=5432
DB_POSTGRES_USER=postgres
DB_POSTGRES_PASSWORD=password
DB_POSTGRES_NAME=test_db
# API Server Settings
PORT=8000
ENVIRONMENT=development # development or production
LOG_LEVEL=INFOCreate deploy.sh in your project root:
#!/bin/bash
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m'
echo -e "${YELLOW}=== Python MySQL/PostgreSQL FastAPI Deployment ===${NC}\n"
show_menu() {
echo "Choose an option:"
echo "1. Run locally (development)"
echo "2. Run tests"
echo "3. Build Docker image"
echo "4. Deploy to remote server"
echo "5. Deploy to Heroku"
echo "6. Exit"
read -p "Enter choice [1-6]: " choice
}
run_local() {
echo -e "${YELLOW}Starting API server locally...${NC}"
python3 main.py
}
run_tests() {
echo -e "${YELLOW}Running test suite...${NC}"
python3 -m pytest test_main.py test_comprehensive.py -v --tb=short
if [ $? -eq 0 ]; then
echo -e "${GREEN}β All tests passed!${NC}\n"
else
echo -e "${RED}β Tests failed!${NC}\n"
fi
}
build_docker() {
echo -e "${YELLOW}Building Docker image...${NC}"
read -p "Enter Docker image name (default: my-api): " image_name
image_name=${image_name:-my-api}
docker build -t $image_name:latest .
if [ $? -eq 0 ]; then
echo -e "${GREEN}β Docker image built: $image_name:latest${NC}"
echo "To run: docker run -p 8000:8000 -e DB_HOST=... $image_name:latest"
else
echo -e "${RED}β Docker build failed!${NC}"
fi
}
deploy_aws_ec2() {
read -p "Enter EC2 instance IP/hostname: " ec2_host
read -p "Enter SSH key path (default: ~/.ssh/id_rsa): " ssh_key
ssh_key=${ssh_key:-~/.ssh/id_rsa}
echo -e "${YELLOW}Deploying to AWS EC2: $ec2_host${NC}"
ssh -i $ssh_key ec2-user@$ec2_host << 'DEPLOY'
# Install Docker on Amazon Linux
sudo yum update -y
sudo amazon-linux-extras install docker -y
sudo systemctl start docker
sudo systemctl enable docker
# Create app directory
mkdir -p ~/app && cd ~/app
# Create .env file (customize these values)
cat > .env << 'ENV'
DB_TYPE=mysql
DB_HOST=your_remote_db_host
DB_PORT=3306
DB_USER=your_user
DB_PASSWORD=your_password
DB_NAME=your_database
PORT=8000
ENV
# Pull and run Docker image
docker pull your_docker_registry/my-api:latest
docker run -d --restart always \
--name my-api \
-p 8000:8000 \
--env-file .env \
your_docker_registry/my-api:latest
DEPLOY
echo -e "${GREEN}β Deployed to AWS EC2${NC}"
echo "API available at: http://$ec2_host:8000"
}
deploy_digitalocean() {
read -p "Enter Droplet IP address: " do_host
read -p "Enter SSH key path (default: ~/.ssh/id_rsa): " ssh_key
ssh_key=${ssh_key:-~/.ssh/id_rsa}
echo -e "${YELLOW}Deploying to DigitalOcean: $do_host${NC}"
ssh -i $ssh_key root@$do_host << 'DEPLOY'
# Update system
apt-get update && apt-get upgrade -y
# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh
systemctl start docker
systemctl enable docker
# Create app directory
mkdir -p /opt/app && cd /opt/app
# Create .env file
cat > .env << 'ENV'
DB_TYPE=postgresql
DB_POSTGRES_HOST=your_remote_db_host
DB_POSTGRES_PORT=5432
DB_POSTGRES_USER=postgres
DB_POSTGRES_PASSWORD=your_password
DB_POSTGRES_NAME=your_database
PORT=8000
ENV
# Pull and run Docker image
docker pull your_docker_registry/my-api:latest
docker run -d --restart always \
--name my-api \
-p 8000:8000 \
--env-file .env \
your_docker_registry/my-api:latest
DEPLOY
echo -e "${GREEN}β Deployed to DigitalOcean${NC}"
echo "API available at: http://$do_host:8000"
}
deploy_heroku() {
echo -e "${YELLOW}=== Heroku Deployment ===${NC}\n"
if ! command -v heroku &> /dev/null; then
echo "Installing Heroku CLI..."
curl https://cli-assets.heroku.com/install.sh | sh
fi
read -p "Enter Heroku app name: " heroku_app
echo "Logging into Heroku..."
heroku login
echo "Creating Heroku app..."
heroku create $heroku_app 2>/dev/null || true
echo "Setting environment variables..."
heroku config:set --app $heroku_app \
DB_TYPE=postgresql \
DB_POSTGRES_HOST=your_db_host \
DB_POSTGRES_USER=your_user \
DB_POSTGRES_PASSWORD=your_password \
DB_POSTGRES_NAME=your_database
echo "Deploying to Heroku..."
git push heroku main
echo -e "${GREEN}β Deployed to Heroku${NC}"
echo "API available at: https://$heroku_app.herokuapp.com"
}
# Main loop
while true; do
show_menu
case $choice in
1)
run_local
;;
2)
run_tests
;;
3)
build_docker
;;
4)
echo ""
echo "Choose deployment platform:"
echo "1. AWS EC2"
echo "2. DigitalOcean"
echo "3. Linode (manual SSH)"
echo "4. Google Cloud (manual SSH)"
echo "5. Azure (manual SSH)"
read -p "Enter choice [1-5]: " deploy_choice
case $deploy_choice in
1) deploy_aws_ec2 ;;
2) deploy_digitalocean ;;
*)
echo -e "${YELLOW}For other platforms, use manual SSH deployment${NC}"
echo "Commands: git clone, install docker, pull image, run container"
;;
esac
;;
5)
deploy_heroku
;;
6)
echo -e "${GREEN}Goodbye!${NC}"
exit 0
;;
*)
echo -e "${RED}Invalid choice. Please try again.${NC}"
;;
esac
doneMake executable and run:
chmod +x deploy.sh
./deploy.sh# Start the server in background
python main.py &
# Test root endpoint
curl http://localhost:8000/
# Test with database
curl -X POST http://localhost:8000/api/users \
-H "Content-Type: application/json" \
-d '{"input": {"name": "John", "email": "john@example.com"}}'# Run all tests
python -m pytest test_main.py test_comprehensive.py -v
# Run specific test file
python -m pytest test_comprehensive.py -v
# Run with coverage report
python -m pytest --cov=app test_main.pyTest Results: β 25/25 tests passing
docker build -t my-api:latest .docker run -d \
-p 8000:8000 \
-e DB_TYPE=mysql \
-e DB_HOST=mysql-db \
-e DB_PORT=3306 \
-e DB_USER=root \
-e DB_PASSWORD=password \
-e DB_NAME=test_db \
--name my-api \
my-api:latest
# Check logs
docker logs my-api
# Stop container
docker stop my-apiCreate docker-compose.yml:
version: '3.8'
services:
mysql:
image: mysql:8.0
environment:
MYSQL_ROOT_PASSWORD: password
MYSQL_DATABASE: test_db
ports:
- "3306:3306"
volumes:
- mysql_data:/var/lib/mysql
postgres:
image: postgres:15
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
POSTGRES_DB: test_db
ports:
- "5432:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
api:
build: .
ports:
- "8000:8000"
environment:
DB_TYPE: mysql
DB_HOST: mysql
DB_PORT: 3306
DB_USER: root
DB_PASSWORD: password
DB_NAME: test_db
depends_on:
- mysql
- postgres
volumes:
mysql_data:
postgres_data:Run with Docker Compose:
docker-compose up -d
# API available at: http://localhost:8000
# MySQL: localhost:3306
# PostgreSQL: localhost:5432| Platform | Type | Cost | Setup Time | Best For |
|---|---|---|---|---|
| AWS EC2 + RDS | VM + Managed DB | $10-50/month | 20 min | Enterprise, auto-scaling |
| DigitalOcean | Droplet + Managed DB | $5-20/month | 10 min | Startups, simplicity |
| Heroku | PaaS | $7-50/month | 5 min | Quick deployment |
| Railway.app | Modern PaaS | Free-$20/month | 3 min | Fastest setup |
| Render | Modern PaaS | Free-$19/month | 5 min | Simple & fast |
| Google Cloud Run | Serverless | Pay-per-use | 15 min | Sporadic traffic |
| Azure App Service | PaaS | $10-50/month | 20 min | Microsoft ecosystem |
Recommended for Quick Start: Railway.app or Render
Recommended for Production: AWS or DigitalOcean
curl -X POST http://localhost:8000/api/users \
-H "Content-Type: application/json" \
-d '{
"input": {
"name": "John Doe",
"email": "john@example.com",
"age": 30
}
}'curl http://localhost:8000/api/users/querycurl -X POST http://localhost:8000/api/users/query \
-H "Content-Type: application/json" \
-d '{
"input": {
"age": 30
}
}'curl -X PUT http://localhost:8000/api/users \
-H "Content-Type: application/json" \
-d '{
"input": {"age": 31},
"where": {"id": 1}
}'curl -X DELETE http://localhost:8000/api/users \
-H "Content-Type: application/json" \
-d '{
"input": {"id": 1}
}'curl -X POST http://localhost:8000/api/batch/import \
-F "file=@users.csv"curl http://localhost:8000/api/batch/users/export -o users.csv- Database configured (MySQL or PostgreSQL)
- All tests passing (
python -m pytest test_main.py -v) - Environment variables set in
.env(or secrets management) - Docker image built and tested
- API endpoints tested with curl or Postman
- Error handling verified
- Database backups configured
- Logging enabled and monitored
- Security group/firewall rules configured
- SSL/TLS certificate installed (if using domain)
- Monitoring and alerts set up
- Rate limiting configured (if needed)
- QUICKSTART.md - 15-minute setup guide
- API_DOCUMENTATION.md - Complete API specification
- API_REFERENCE.md - Visual examples and copy-paste commands
- ARCHITECTURE.md - System design and internals
- TESTING_GUIDE.sh - 16 test examples
- INDEX.md - Complete documentation index
Common Issues:
-
"Database connection failed"
- Verify database is running and accessible
- Check
.envfile configuration matches database - Confirm database credentials are correct
- For remote databases, check security group/firewall allows connection
-
"Port 8000 already in use"
- Change port:
uvicorn main:app --port 8001 - Or kill process:
lsof -i :8000 | grep LISTEN | awk '{print $2}' | xargs kill -9
- Change port:
-
"Module not found"
- Activate virtual environment:
source venv/bin/activate - Install dependencies:
pip install -r requirements.txt
- Activate virtual environment:
-
"Cannot connect to remote database"
- Check network connectivity:
telnet db_host db_port - Verify security group allows your IP
- Confirm
.envhas correct host/port
- Check network connectivity:
For more details, see API_DOCUMENTATION.md section.
-
Clone repository
git clone https://github.com/burkaslarry/python_mysql_http.git cd python_mysql_http -
Install dependencies
pip install -r requirements.txt
-
Configure database
cat > .env << EOF DB_TYPE=mysql DB_HOST=localhost DB_USER=root DB_PASSWORD=your_password DB_NAME=test_db EOF
-
Create database
mysql -u root -p -e "CREATE DATABASE test_db;" -
Run tests
python -m pytest test_main.py -v
-
Start API
python main.py
-
Deploy
./deploy.sh # Interactive deployment menu
- Tests: β 25/25 passing
- Databases: β MySQL & PostgreSQL
- Documentation: β Complete
- Docker: β Production ready
- Deployment: β Automated scripts
Version: 1.0.0
Last Updated: 2025-12-30
Status: Production Ready β