5 MinIO Quota Enforcement Script - nginx_request_checker.py

Lutchy Horace edited this page 2025-04-02 11:27:41 -04:00

Table of Contents

• MinIO Quota Enforcement Proxy
• Overview
• Features
• Configuration
• Configurable Parameters
• Example Configuration File (`/etc/minio_quota.conf`)
• Installation & Usage
• Prerequisites
• Running the Script
• Running as a Systemd Service
• API Behavior
• Example Error Response
• Notes
• License

MinIO Quota Enforcement Proxy

Overview

This script acts as a middleware between Nginx and MinIO to enforce storage quotas per user. It checks Redis for quota status before proxying requests to MinIO. If a user has exceeded their quota, the script returns an S3-style XML error message.

Features

• Quota Enforcement: Blocks users who exceed their storage quota.
• Configurable Settings: Reads configurations from environment variables and /etc/minio_quota.conf.
• Logging: Always logs to stdout and optionally logs to a file if LOG_FILE is set.
• Proxying: Forwards valid requests to MinIO.

Configuration

The script prioritizes configuration values in this order:

1. Environment Variables
2. Configuration File (/etc/minio_quota.conf)
3. Default Values

Configurable Parameters

Variable	Default Value	Description
REDIS_HOST	127.0.0.1	Redis server hostname/IP
REDIS_PORT	6379	Redis server port
REDIS_DB	2	Redis database number
MINIO_ENDPOINT	http://127.0.0.1:9000	MinIO server URL
LOG_FILE	None	Path to log file (logs only if set)

Example Configuration File (/etc/minio_quota.conf)

REDIS_HOST=192.168.1.100
REDIS_PORT=6379
REDIS_DB=2
MINIO_ENDPOINT=http://minio.example.com
LOG_FILE=/var/log/minio_quota.log

Installation & Usage

Prerequisites

• Python 3.8+
• fastapi, redis, httpx, and uvicorn

Install dependencies:

pip install fastapi redis httpx uvicorn

Running the Script

Run the script using uvicorn:

uvicorn nginx_request_checker:app --host 0.0.0.0 --port 5000

To run in the background:

nohup uvicorn nginx_request_checker:app --host 0.0.0.0 --port 5000 > nginx_request_checker.log 2>&1 &

Running as a Systemd Service

1. Create a systemd service file:

sudo nano /etc/systemd/system/minio_quota_checker.service

2. Add the following:

[Unit]
Description=MinIO Quota Checker API
After=network.target

[Service]
User=your_user
WorkingDirectory=/path/to/script
EnvironmentFile=-/etc/default/minio_quota_checker
ExecStart=/usr/bin/uvicorn nginx_request_checker:app --host 0.0.0.0 --port 5000
Restart=always

[Install]
WantedBy=multi-user.target

3. Enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable minio_quota_checker
sudo systemctl start minio_quota_checker

API Behavior

The script intercepts all requests and performs the following:

1. Extracts the username from the request path.
2. Checks Redis for the key quota_exceeded:<username>.
3. If quota is exceeded, it returns a 403 Forbidden response with an XML error message.
4. Otherwise, it forwards the request to MinIO.

Example Error Response

<?xml version="1.0" encoding="UTF-8"?>
<Error>
  <Code>QuotaExceeded</Code>
  <Message>User has exceeded storage quota.</Message>
  <Resource>/test-bucket/object</Resource>
  <RequestId>request-id-12345</RequestId>
</Error>

Notes

• Ensure Redis is running and accessible by the script.
• Update Nginx to forward requests to this script instead of directly to MinIO.

License

This script is provided under the Creative Commons Attribution-NonCommercial 4.0 International license.