docs: comprehensive README rewrite with security improvements

Completely rewrite README.md based on codebase analysis and
implemented security improvements.

Changes:
- Add comprehensive overview with feature list
- Add supported backup types table with all 10+ drivers
- Restructure Quick Start with step-by-step installation
- Add detailed configuration examples for each backup type
- Document all CLI commands with Docker exec examples
- Add dedicated Security section highlighting improvements
- Include reverse proxy setup with security headers
- Add Troubleshooting section with common issues
- Add Development section with uv commands
- Reorganize into logical sections with clear hierarchy

Improvements:
- Emphasize Ed25519 as recommended SSH key algorithm
- Document Flask secret key security requirement
- Include security best practices section
- Add command execution safety information
- Provide nginx reverse proxy example with TLS
- Include proper file permissions instructions

Documentation structure:
1. Overview and features
2. Quick Start (10-step installation)
3. Configuration (by backup type)
4. CLI Usage (all commands)
5. Development setup
6. Security (best practices)
7. Reverse Proxy setup
8. Architecture overview
9. Troubleshooting
10. Contributing and support

Target audience:
- New users: Clear installation steps
- Existing users: Migration to Ed25519 keys
- Developers: Development environment setup
- Security-conscious admins: Best practices

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
k3nny 2025-10-05 01:47:45 +02:00
parent 2533b56549
commit d130ba2a11

569
README.md
View File

@ -1,178 +1,483 @@
# TISBackup # TISBackup
This is the repository of the TISBackup project, licensed under GPLv3. A comprehensive server-side backup orchestration system for managing automated backups of databases, files, and virtual machines across remote Linux and Windows systems.
TISBackup is a python script to backup servers. [![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
[![Python 3.13+](https://img.shields.io/badge/python-3.13+-blue.svg)](https://www.python.org/downloads/)
It runs at regular intervals to retrieve different data types on remote hosts ## Overview
such as database dumps, files, virtual machine images and metadata.
## Install using Compose TISBackup is a Python-based backup solution that provides:
Clone that repository and build the pod image using the provided `Dockerfile` - **Pluggable backup drivers** for different data sources (databases, files, VMs)
- **Web-based management interface** for monitoring and controlling backups
- **CLI tool** for automation and scripting
- **Automated scheduling** via cron
- **Backup retention management** with configurable policies
- **Status monitoring** with Nagios integration
- **Docker deployment** for easy setup and isolation
```bash ### Supported Backup Types
docker build . -t tisbackup:latest
```
In another folder, create subfolders as following | Type | Description | Driver |
|------|-------------|--------|
| **Files & Directories** | rsync-based file backups | `rsync+ssh` |
| **Btrfs Snapshots** | Snapshot-based incremental backups | `rsync+btrfs+ssh` |
| **MySQL** | Database dumps via SSH | `mysql+ssh` |
| **PostgreSQL** | Database dumps via SSH | `pgsql+ssh` |
| **SQL Server** | SQL Server backups | `sqlserver+ssh` |
| **Oracle** | Oracle database backups | `oracle+ssh` |
| **Samba4 AD** | Active Directory backups | `samba4` |
| **XenServer VMs** | XVA exports and metadata | `xen-xva`, `xcp-dump-metadata` |
| **VMware** | VMDK backups | `vmdk` |
| **Network Devices** | Switch configuration backups | `switch` |
```bash ## Quick Start
mkdir -p /var/tisbackup/{backup/log,config,ssh}/
```
Expected structure ### Prerequisites
```
/var/tisbackup/
└─backup/ <-- backup location
└─config/
├── tisbackup-config.ini <-- backups config
└── tisbackup_gui.ini <-- tisbackup config
└─ssh/
├── id_ed25519 <-- SSH Private Key (Ed25519 recommended)
└── id_ed25519.pub <-- SSH Public Key
compose.yaml
```
Adapt the compose.yml file to suits your needs, one pod act as the WebUI front end and the other as the crond scheduler - Docker and Docker Compose
- SSH access to remote servers
- Ed25519, ECDSA, or RSA SSH keys (DSA not supported)
```yaml ### Installation
services:
tisbackup_gui:
container_name: tisbackup_gui
image: "tisbackup:latest"
build: .
volumes:
- ./config/:/etc/tis/
- ./backup/:/backup/
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
environment:
# SECURITY: Set a unique secret key for Flask session security
# Generate with: python3 -c "import secrets; print(secrets.token_hex(32))"
- TISBACKUP_SECRET_KEY=your-secret-key-here-change-me
restart: unless-stopped
ports:
- 9980:8080
tisbackup_cron: 1. **Clone the repository:**
container_name: tisbackup_cron ```bash
image: "tisbackup:latest" git clone https://github.com/tranquilit/TISbackup.git
build: . cd TISbackup
volumes: ```
- ./config/:/etc/tis/
- ./ssh/:/config_ssh/
- ./backup/:/backup/
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
restart: always
command: "/bin/bash /opt/tisbackup/cron.sh"
``` 2. **Build the Docker image:**
```bash
docker build . -t tisbackup:latest
```
3. **Create directory structure:**
```bash
mkdir -p /var/tisbackup/{backup/log,config,ssh}
```
Expected structure:
```
/var/tisbackup/
├── backup/ # Backup storage location
│ └── log/ # SQLite database and logs
├── config/ # Configuration files
│ ├── tisbackup-config.ini
│ └── tisbackup_gui.ini
├── ssh/ # SSH keys
│ ├── id_ed25519 # Private key (Ed25519 recommended)
│ └── id_ed25519.pub # Public key
└── compose.yaml # Docker Compose configuration
```
4. **Generate SSH keys:**
```bash
# Ed25519 (recommended - most secure and modern)
ssh-keygen -t ed25519 -f /var/tisbackup/ssh/id_ed25519 -C "tisbackup@yourserver"
# Or ECDSA (also secure)
ssh-keygen -t ecdsa -b 521 -f /var/tisbackup/ssh/id_ecdsa -C "tisbackup@yourserver"
# Or RSA (legacy support, minimum 4096 bits)
ssh-keygen -t rsa -b 4096 -f /var/tisbackup/ssh/id_rsa -C "tisbackup@yourserver"
```
⚠️ **Note:** DSA keys are no longer supported due to security vulnerabilities.
5. **Deploy public key to remote servers:**
```bash
ssh-copy-id -i /var/tisbackup/ssh/id_ed25519.pub root@remote-server
```
6. **Generate Flask secret key:**
```bash
python3 -c "import secrets; print(secrets.token_hex(32))"
```
Save this key for the next step.
7. **Create Docker Compose configuration:**
```yaml
# /var/tisbackup/compose.yaml
services:
tisbackup_gui:
container_name: tisbackup_gui
image: "tisbackup:latest"
volumes:
- ./config/:/etc/tis/
- ./backup/:/backup/
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
environment:
# SECURITY: Use the secret key you generated above
- TISBACKUP_SECRET_KEY=your-secret-key-here
restart: unless-stopped
ports:
- 9980:8080
tisbackup_cron:
container_name: tisbackup_cron
image: "tisbackup:latest"
volumes:
- ./config/:/etc/tis/
- ./ssh/:/config_ssh/
- ./backup/:/backup/
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
restart: always
command: "/bin/bash /opt/tisbackup/cron.sh"
```
8. **Configure backups:**
Create `/var/tisbackup/config/tisbackup-config.ini`:
```ini
[global]
backup_base_dir = /backup/
# Backup retention in days
backup_retention_time = 90
# Maximum backup age for Nagios checks (hours)
maximum_backup_age = 30
# Example: File backup via rsync
[webserver-files]
type = rsync+ssh
server_name = webserver.example.com
remote_dir = /var/www/
compression = True
exclude_list = "/var/www/cache/**","/var/www/temp/**"
private_key = /config_ssh/id_ed25519
ssh_port = 22
# Example: MySQL database backup
[database-mysql]
type = mysql+ssh
server_name = db.example.com
db_name = production_db
db_user = backup_user
db_passwd = backup_password
private_key = /config_ssh/id_ed25519
ssh_port = 22
```
Create `/var/tisbackup/config/tisbackup_gui.ini`:
```ini
[general]
config_tisbackup = /etc/tis/tisbackup-config.ini
sections =
ADMIN_EMAIL = admin@example.com
base_config_dir = /etc/tis/
backup_base_dir = /backup/
```
9. **Start services:**
```bash
cd /var/tisbackup
docker compose up -d
```
10. **Access web interface:**
```
http://localhost:9980
```
## Configuration ## Configuration
### SSH Keys ### Backup Types Configuration
* **Generate SSH keys** (Ed25519 recommended):
```bash
# Ed25519 (most secure, recommended)
ssh-keygen -t ed25519 -f ./ssh/id_ed25519 -C "tisbackup@yourserver"
# Or ECDSA (also secure)
ssh-keygen -t ecdsa -b 521 -f ./ssh/id_ecdsa -C "tisbackup@yourserver"
# Or RSA (legacy, minimum 2048 bits)
ssh-keygen -t rsa -b 4096 -f ./ssh/id_rsa -C "tisbackup@yourserver"
```
**⚠️ Note:** DSA keys are no longer supported due to security vulnerabilities
* Copy public key to remote servers:
```bash
ssh-copy-id -i ./ssh/id_ed25519.pub root@remote-server
```
### Configuration Files
* Setup config files in the `./config` directory
* **SECURITY**: Generate and set a secure Flask secret key:
```bash
# Generate a secure random secret key
python3 -c "import secrets; print(secrets.token_hex(32))"
```
Then add it to your `compose.yml` as the `TISBACKUP_SECRET_KEY` environment variable
**tisbackup-config.ini**
#### File Backups (rsync+ssh)
```ini ```ini
[global] [backup-name]
backup_base_dir = /backup/ type = rsync+ssh
server_name = hostname.example.com
# backup retention in days remote_dir = /path/to/backup/
backup_retention_time=90 compression = True
exclude_list = "/path/exclude1/**","/path/exclude2/**"
# for nagios check in hours private_key = /config_ssh/id_ed25519
maximum_backup_age=30
[srvads-poudlard-samba]
type=rsync+ssh
server_name=srvads.poudlard.lan
remote_dir=/var/lib/samba/
compression=True
;exclude_list="/proc/**","/sys/**","/dev/**"
# Use Ed25519 key (recommended), or ECDSA/RSA (DSA not supported)
private_key=/config_ssh/id_ed25519
ssh_port = 22 ssh_port = 22
``` ```
**tisbackup_gui.ini** #### Btrfs Snapshots (rsync+btrfs+ssh)
```ini ```ini
[general] [backup-name]
config_tisbackup= /etc/tis/tisbackup-config.ini type = rsync+btrfs+ssh
sections= server_name = hostname.example.com
ADMIN_EMAIL=josebove@internet.fr remote_dir = /mnt/btrfs/data/
base_config_dir= /etc/tis/ compression = True
backup_base_dir=/backup/ private_key = /config_ssh/id_ed25519
ssh_port = 22
``` ```
Run! #### MySQL Database (mysql+ssh)
```ini
[backup-name]
type = mysql+ssh
server_name = hostname.example.com
db_name = database_name
db_user = backup_user
db_passwd = backup_password
private_key = /config_ssh/id_ed25519
ssh_port = 22
```
#### PostgreSQL Database (pgsql+ssh)
```ini
[backup-name]
type = pgsql+ssh
server_name = hostname.example.com
db_name = database_name
private_key = /config_ssh/id_ed25519
ssh_port = 22
```
#### XenServer VM (xen-xva)
```ini
[backup-name]
type = xen-xva
server_name = vm-name
xcphost = xenserver.example.com
password_file = /etc/tis/xen-password
private_key = /config_ssh/id_ed25519
```
### Pre/Post Execution Hooks
You can execute commands before and after backups:
```ini
[backup-name]
type = rsync+ssh
server_name = hostname.example.com
remote_dir = /data/
private_key = /config_ssh/id_ed25519
preexec = systemctl stop application
postexec = systemctl start application
remote_user = root
ssh_port = 22
```
## CLI Usage
### Running Backups
```bash ```bash
docker compose up -d # Run all backups
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py backup
# Run specific backup
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -s backup-name backup
# Dry run
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -d backup
``` ```
## NGINX reverse-proxy ### Cleanup Old Backups
Sample config file ```bash
# Remove backups older than retention period
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py cleanup
```
### Nagios Monitoring
```bash
# Check backup status
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py checknagios
# Check specific backup
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -s backup-name checknagios
```
### List Available Drivers
```bash
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py listdrivers
```
### Backup Statistics
```bash
# Dump statistics for last 20 backups
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py dumpstat
# Specify number of backups
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -n 50 dumpstat
```
## Development
### Prerequisites
- Python 3.13+
- uv (Python package manager)
### Setup Development Environment
```bash
# Install dependencies
uv sync --locked
# Run linter
uv run ruff check .
# Auto-fix linting issues
uv run ruff check --fix .
```
### Running Locally
```bash
# Run web GUI (requires config at /etc/tis/tisbackup_gui.ini)
python3 tisbackup_gui.py
# Run CLI
python3 tisbackup.py -c /etc/tis/tisbackup-config.ini backup
```
## Security
TISBackup implements several security best practices:
### SSH Key Security
- **Ed25519 keys are recommended** (most secure, modern algorithm)
- ECDSA and RSA keys are supported
- **DSA keys are explicitly not supported** (deprecated, insecure)
- Key algorithm priority: Ed25519 → ECDSA → RSA
### Flask Session Security
- Secret key loaded from `TISBACKUP_SECRET_KEY` environment variable
- Falls back to cryptographically secure random key if not set
- No hardcoded secrets in source code
### Command Execution Safety
- All system commands use `subprocess.run()` with list arguments
- Input validation for device paths and partition names
- Timeout protection on all subprocess calls
- No use of `shell=True` in new code
### Best Practices
1. **Use Ed25519 keys** for all SSH connections
2. **Set unique Flask secret key** via environment variable
3. **Use reverse proxy** (nginx) with TLS for web interface
4. **Restrict network access** to backup server
5. **Regular security updates** of base Docker image
6. **Monitor backup logs** for suspicious activity
## Reverse Proxy Setup
Example nginx configuration for HTTPS access:
```nginx ```nginx
server { server {
listen 443 ssl http2; listen 443 ssl http2;
# Remove '#' in the next line to enable IPv6 server_name tisbackup.example.com;
# listen [::]:443 ssl http2;
server_name tisbackup.poudlard.lan;
ssl_certificate /etc/letsencrypt/live/tisbackup.poudlard.lan/fullchain.pem; # managed by Certbot ssl_certificate /etc/letsencrypt/live/tisbackup.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/tisbackup.poudlard.lan/privkey.pem; # managed by Certbot ssl_certificate_key /etc/letsencrypt/live/tisbackup.example.com/privkey.pem;
# Security headers
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
location / { location / {
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host; proxy_set_header Host $host;
proxy_pass http://localhost:9980/; proxy_pass http://localhost:9980/;
proxy_http_version 1.1; proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade; proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade"; proxy_set_header Connection "upgrade";
} }
} }
``` ```
## Architecture
TISBackup uses a modular driver-based architecture:
## About - **Core CLI** ([tisbackup.py](tisbackup.py)): Backup orchestration and scheduling
- **Web GUI** ([tisbackup_gui.py](tisbackup_gui.py)): Flask-based management interface
- **Backup Drivers** ([libtisbackup/](libtisbackup/)): Pluggable modules for different backup types
- **Task Queue** ([tasks.py](tasks.py), [config.py](config.py)): Async job processing with Huey
- **State Database**: SQLite for tracking backup history and statistics
[Tranquil IT](contact_at_tranquil_it) is the original author of TISBackup. Each backup type is implemented as a driver class inheriting from `backup_generic`, allowing easy extension for new backup sources.
The documentation is provided under the license CC-BY-SA and can be found ## Troubleshooting
on [readthedoc](https://tisbackup.readthedocs.io/en/latest/index.html).
### Backups Not Running
1. Check cron logs:
```bash
docker logs tisbackup_cron
```
2. Verify SSH connectivity:
```bash
docker exec tisbackup_cron ssh -i /config_ssh/id_ed25519 root@remote-server
```
3. Check backup configuration:
```bash
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -c /etc/tis/tisbackup-config.ini -d backup
```
### Web Interface Not Accessible
1. Check GUI container logs:
```bash
docker logs tisbackup_gui
```
2. Verify port mapping:
```bash
docker ps | grep tisbackup_gui
```
3. Check configuration:
```bash
docker exec tisbackup_gui cat /etc/tis/tisbackup_gui.ini
```
### Permission Errors
Ensure proper file permissions:
```bash
chmod 600 /var/tisbackup/ssh/id_ed25519
chmod 644 /var/tisbackup/ssh/id_ed25519.pub
chown -R root:root /var/tisbackup/
```
## Contributing
Contributions are welcome! Please:
1. Fork the repository
2. Create a feature branch
3. Follow the existing code style (use `ruff` for linting)
4. Add tests if applicable
5. Submit a pull request
## License
TISBackup is licensed under the GNU General Public License v3.0 (GPLv3).
See [LICENSE](LICENSE) for the full license text.
## Support & Documentation
- **Documentation**: [https://tisbackup.readthedocs.io](https://tisbackup.readthedocs.io/en/latest/index.html)
- **Issues**: [GitHub Issues](https://github.com/tranquilit/TISbackup/issues)
- **Original Author**: [Tranquil IT](https://www.tranquil.it)
## Credits
Developed by Tranquil IT for system administrators managing backup infrastructure.
Security improvements and modernization contributed by the community.