commit 4d44f96e0d2eeecc30266ab0e1a4b5f980219e02 Author: Lutchy Horace Date: Wed Apr 2 08:43:41 2025 -0400 Add MinIO Quota Enforcement Script - minio_quota_checker.py diff --git a/MinIO Quota Enforcement Script - minio_quota_checker.py.-.md b/MinIO Quota Enforcement Script - minio_quota_checker.py.-.md new file mode 100644 index 0000000..da4688a --- /dev/null +++ b/MinIO Quota Enforcement Script - minio_quota_checker.py.-.md @@ -0,0 +1,65 @@ +# MinIO Quota Enforcement Script + +This script monitors and enforces user-based storage quotas on a MinIO object storage instance. It periodically aggregates disk usage per user, logs warnings for overages, and sets a Redis key when a user exceeds their quota. + +## Features +- Fetches MinIO buckets and groups them by user. +- Calculates total storage usage for each user. +- Logs warnings when a user exceeds their quota. +- Uses Redis to store quota exceedance status (`quota_exceeded:username`). +- Supports configurable quotas, whitelisted users, and customizable intervals. + +## Configuration +The script loads its configuration from environment variables or from `/etc/minio_quota.conf`. Available settings: + +| Variable | Description | Default Value | +|--------------------|------------------------------------------------|--------------| +| `MINIO_ENDPOINT` | MinIO server endpoint | `play.min.io` | +| `MINIO_ACCESS_KEY` | MinIO access key | `your-access-key` | +| `MINIO_SECRET_KEY` | MinIO secret key | `your-secret-key` | +| `TOTAL_SIZE_LIMIT` | Storage quota per user (e.g., `1G`, `500M`) | `1G` | +| `WHITELIST` | Comma-separated list of users exempt from quotas | `admin,superuser` | +| `AGGREGATE_INTERVAL` | Interval (seconds) for disk usage aggregation | `600` (10 minutes) | +| `REDIS_HOST` | Redis server hostname | `localhost` | +| `REDIS_PORT` | Redis server port | `6379` | +| `REDIS_DB` | Redis database ID | `0` | +| `LOG_FILE` | Log file location | `minio_quota.log` | + +## Installation +1. Install dependencies: + ```sh + pip install minio redis + ``` +2. (Optional) Create a configuration file at `/etc/minio_quota.conf`: + ```ini + MINIO_ENDPOINT=minio.example.com + MINIO_ACCESS_KEY=my-access-key + MINIO_SECRET_KEY=my-secret-key + TOTAL_SIZE_LIMIT=500M + WHITELIST=admin,superuser + AGGREGATE_INTERVAL=600 + REDIS_HOST=localhost + REDIS_PORT=6379 + REDIS_DB=1 + LOG_FILE=/var/log/minio_quota.log + ``` +3. Run the script: + ```sh + python minio_quota.py + ``` + +## How It Works +- The script runs in an infinite loop, executing every `AGGREGATE_INTERVAL` seconds. +- It fetches all MinIO buckets and groups them by username. +- It calculates total storage usage per user and compares it with `TOTAL_SIZE_LIMIT`. +- If a user exceeds their quota, a warning is logged and a Redis key `quota_exceeded:username` is set. +- Users in the `WHITELIST` are ignored. + +## Handling Quota Enforcement +Other services (e.g., Nginx or an API gateway) can check Redis for quota exceedance and enforce restrictions accordingly. + +## Signals & Graceful Shutdown +The script handles `SIGINT` and `SIGTERM` signals to allow a clean exit. + +## License +This script is provided as-is under an open-source license. \ No newline at end of file