Add MinIO Quota Enforcement Script - minio_quota_checker.py

MinIO Quota Enforcement Script - minio_quota_checker.py.-.md

@@ -0,0 +1,65 @@
+# MinIO Quota Enforcement Script
+
+This script monitors and enforces user-based storage quotas on a MinIO object storage instance. It periodically aggregates disk usage per user, logs warnings for overages, and sets a Redis key when a user exceeds their quota.
+
+## Features
+- Fetches MinIO buckets and groups them by user.
+- Calculates total storage usage for each user.
+- Logs warnings when a user exceeds their quota.
+- Uses Redis to store quota exceedance status (`quota_exceeded:username`).
+- Supports configurable quotas, whitelisted users, and customizable intervals.
+
+## Configuration
+The script loads its configuration from environment variables or from `/etc/minio_quota.conf`. Available settings:
+
+| Variable            | Description                                      | Default Value |
+|--------------------|------------------------------------------------|--------------|
+| `MINIO_ENDPOINT`   | MinIO server endpoint                          | `play.min.io` |
+| `MINIO_ACCESS_KEY` | MinIO access key                              | `your-access-key` |
+| `MINIO_SECRET_KEY` | MinIO secret key                              | `your-secret-key` |
+| `TOTAL_SIZE_LIMIT` | Storage quota per user (e.g., `1G`, `500M`)   | `1G` |
+| `WHITELIST`        | Comma-separated list of users exempt from quotas | `admin,superuser` |
+| `AGGREGATE_INTERVAL` | Interval (seconds) for disk usage aggregation | `600` (10 minutes) |
+| `REDIS_HOST`       | Redis server hostname                         | `localhost` |
+| `REDIS_PORT`       | Redis server port                             | `6379` |
+| `REDIS_DB`         | Redis database ID                             | `0` |
+| `LOG_FILE`         | Log file location                             | `minio_quota.log` |
+
+## Installation
+1. Install dependencies:
+   ```sh
+   pip install minio redis
+   ```
+2. (Optional) Create a configuration file at `/etc/minio_quota.conf`:
+   ```ini
+   MINIO_ENDPOINT=minio.example.com
+   MINIO_ACCESS_KEY=my-access-key
+   MINIO_SECRET_KEY=my-secret-key
+   TOTAL_SIZE_LIMIT=500M
+   WHITELIST=admin,superuser
+   AGGREGATE_INTERVAL=600
+   REDIS_HOST=localhost
+   REDIS_PORT=6379
+   REDIS_DB=1
+   LOG_FILE=/var/log/minio_quota.log
+   ```
+3. Run the script:
+   ```sh
+   python minio_quota.py
+   ```
+
+## How It Works
+- The script runs in an infinite loop, executing every `AGGREGATE_INTERVAL` seconds.
+- It fetches all MinIO buckets and groups them by username.
+- It calculates total storage usage per user and compares it with `TOTAL_SIZE_LIMIT`.
+- If a user exceeds their quota, a warning is logged and a Redis key `quota_exceeded:username` is set.
+- Users in the `WHITELIST` are ignored.
+
+## Handling Quota Enforcement
+Other services (e.g., Nginx or an API gateway) can check Redis for quota exceedance and enforce restrictions accordingly.
+
+## Signals & Graceful Shutdown
+The script handles `SIGINT` and `SIGTERM` signals to allow a clean exit.
+
+## License
+This script is provided as-is under an open-source license.