MinIO Quota Enforcement Script

This script monitors and enforces user-based storage quotas on a MinIO object storage instance. It periodically aggregates disk usage per user, logs warnings for overages, and sets a Redis key when a user exceeds their quota.

Important: This script assumes that MinIO buckets are named in the format username-bucketname, where username is the user associated with the bucket.

Features

Fetches MinIO buckets and groups them by user.
Calculates total storage usage for each user.
Logs warnings when a user exceeds their quota.
Uses Redis to store quota exceedance status (quota_exceeded:username).
Supports configurable quotas, whitelisted users, and customizable intervals.

Configuration

The script loads its configuration from environment variables or from /etc/minio_quota.conf. Available settings:

Variable	Description	Default Value
`MINIO_ENDPOINT`	MinIO server endpoint	`play.min.io`
`MINIO_ACCESS_KEY`	MinIO access key	`your-access-key`
`MINIO_SECRET_KEY`	MinIO secret key	`your-secret-key`
`TOTAL_SIZE_LIMIT`	Storage quota per user (e.g., `1G`, `500M`)	`1G`
`WHITELIST`	Comma-separated list of users exempt from quotas	`admin,superuser`
`AGGREGATE_INTERVAL`	Interval (seconds) for disk usage aggregation	`600` (10 minutes)
`REDIS_HOST`	Redis server hostname	`localhost`
`REDIS_PORT`	Redis server port	`6379`
`REDIS_DB`	Redis database ID	`0`
`LOG_FILE`	Log file location	`minio_quota.log`

Installation

Install dependencies:
```
pip install minio redis
```

(Optional) Create a configuration file at /etc/minio_quota.conf:

MINIO_ENDPOINT=minio.example.com
MINIO_ACCESS_KEY=my-access-key
MINIO_SECRET_KEY=my-secret-key
TOTAL_SIZE_LIMIT=500M
WHITELIST=admin,superuser
AGGREGATE_INTERVAL=600
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_DB=1
LOG_FILE=/var/log/minio_quota.log

Run the script:
```
python minio_quota.py
```

How It Works

The script runs in an infinite loop, executing every AGGREGATE_INTERVAL seconds.
It fetches all MinIO buckets and groups them by username.
It calculates total storage usage per user and compares it with TOTAL_SIZE_LIMIT.
If a user exceeds their quota, a warning is logged and a Redis key quota_exceeded:username is set.
Users in the WHITELIST are ignored.

Handling Quota Enforcement

Other services (e.g., Nginx or an API gateway) can check Redis for quota exceedance and enforce restrictions accordingly.

Signals & Graceful Shutdown

The script handles SIGINT and SIGTERM signals to allow a clean exit.

License

This script is provided under the Creative Commons Attribution-NonCommercial 4.0 International license.