backup server side executed python scripts for managing linux and windows system and application data backups, developed by adminsys for adminsys
Go to file
k3nny 1cb731cbdb
Some checks failed
lint / docker (push) Has been cancelled
refactor(drivers): organize backup modules into drivers subfolder
- Move all backup_*.py files to libtisbackup/drivers/ subdirectory
- Move XenAPI.py and copy_vm_xcp.py to drivers/ (driver-specific)
- Create drivers/__init__.py with automatic driver imports
- Update tisbackup.py imports to use new structure
- Add pyvmomi>=8.0.0 as mandatory dependency
- Sync requirements.txt with pyproject.toml dependencies
- Add pylint>=3.0.0 and pytest-cov>=6.0.0 to dev dependencies
- Configure pylint and coverage tools in pyproject.toml
- Add conventional commits guidelines to CLAUDE.md
- Enhance .gitignore with comprehensive patterns for Python, IDEs, testing, and secrets
- Update CLAUDE.md documentation with new structure and tooling

Breaking Changes:
- Drivers must now be imported from libtisbackup.drivers instead of libtisbackup
- All backup driver files relocated to drivers/ subdirectory

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-05 23:54:26 +02:00
.gitea/workflows Python 3.13 + add nginx reverse-proxy 2025-03-07 22:24:27 +01:00
.vscode fix(tisbackup): 🐛 Dockerfile fix venv uv 2025-04-18 23:48:25 +02:00
deb EOF & whitespace 2024-11-29 00:54:09 +01:00
docs EOF & whitespace 2024-11-29 00:54:09 +01:00
docs-sphinx-rst docs: add comprehensive security and authentication documentation 2025-10-05 20:58:19 +02:00
libtisbackup refactor(drivers): organize backup modules into drivers subfolder 2025-10-05 23:54:26 +02:00
nginx Python 3.13 + add nginx reverse-proxy 2025-03-07 22:24:27 +01:00
rpm EOF & whitespace 2024-11-29 00:54:09 +01:00
samples feat(auth): add pluggable authentication system for Flask routes 2025-10-05 02:02:46 +02:00
scripts EOF & whitespace 2024-11-29 00:54:09 +01:00
static EOF & whitespace 2024-11-29 00:54:09 +01:00
templates EOF & whitespace 2024-11-29 00:54:09 +01:00
tests refactor(drivers): organize backup modules into drivers subfolder 2025-10-05 23:54:26 +02:00
.dockerignore fix(tisbackup): 🐛 fix dockerignore pyproject.toml absent 2025-04-18 23:36:26 +02:00
.gitignore refactor(drivers): organize backup modules into drivers subfolder 2025-10-05 23:54:26 +02:00
.hadolint.yml EOF & whitespace 2024-11-29 00:54:31 +01:00
.pre-commit-config.yaml fix iniparse 2024-11-29 23:45:40 +01:00
.python-version docs: add comprehensive security and authentication documentation 2025-10-05 20:58:19 +02:00
AUTHENTICATION.md feat(auth): install all authentication providers by default 2025-10-05 21:17:30 +02:00
backup.sh fix(tisbackup): fix iniparse wrong check 2025-04-14 23:37:16 +02:00
CLAUDE.md refactor(drivers): organize backup modules into drivers subfolder 2025-10-05 23:54:26 +02:00
compose.yml EOF & whitespace 2024-11-29 00:54:09 +01:00
config.py fix iniparse 2024-11-29 23:45:40 +01:00
cron.sh fix first pass 2024-11-28 23:20:19 +01:00
Dockerfile docs: add comprehensive security and authentication documentation 2025-10-05 20:58:19 +02:00
entrypoint.sh EOF & whitespace 2024-11-29 00:54:09 +01:00
pyproject.toml refactor(drivers): organize backup modules into drivers subfolder 2025-10-05 23:54:26 +02:00
README.md docs: comprehensive README rewrite with security improvements 2025-10-05 01:47:45 +02:00
REFACTORING.md refactor(drivers): organize backup modules into drivers subfolder 2025-10-05 23:54:26 +02:00
requirements.txt refactor(drivers): organize backup modules into drivers subfolder 2025-10-05 23:54:26 +02:00
SECURITY_IMPROVEMENTS.md feat(security): modernize SSH key algorithm support with Ed25519 2025-10-05 01:39:17 +02:00
tasks.py fix(security): replace os.popen/os.system with subprocess for command injection prevention 2025-10-05 01:23:53 +02:00
tisbackup_gui.py refactor(drivers): organize backup modules into drivers subfolder 2025-10-05 23:54:26 +02:00
tisbackup.py refactor(drivers): organize backup modules into drivers subfolder 2025-10-05 23:54:26 +02:00
uv.lock refactor(drivers): organize backup modules into drivers subfolder 2025-10-05 23:54:26 +02:00

TISBackup

A comprehensive server-side backup orchestration system for managing automated backups of databases, files, and virtual machines across remote Linux and Windows systems.

License: GPL v3 Python 3.13+

Overview

TISBackup is a Python-based backup solution that provides:

  • Pluggable backup drivers for different data sources (databases, files, VMs)
  • Web-based management interface for monitoring and controlling backups
  • CLI tool for automation and scripting
  • Automated scheduling via cron
  • Backup retention management with configurable policies
  • Status monitoring with Nagios integration
  • Docker deployment for easy setup and isolation

Supported Backup Types

Type Description Driver
Files & Directories rsync-based file backups rsync+ssh
Btrfs Snapshots Snapshot-based incremental backups rsync+btrfs+ssh
MySQL Database dumps via SSH mysql+ssh
PostgreSQL Database dumps via SSH pgsql+ssh
SQL Server SQL Server backups sqlserver+ssh
Oracle Oracle database backups oracle+ssh
Samba4 AD Active Directory backups samba4
XenServer VMs XVA exports and metadata xen-xva, xcp-dump-metadata
VMware VMDK backups vmdk
Network Devices Switch configuration backups switch

Quick Start

Prerequisites

  • Docker and Docker Compose
  • SSH access to remote servers
  • Ed25519, ECDSA, or RSA SSH keys (DSA not supported)

Installation

  1. Clone the repository:

    git clone https://github.com/tranquilit/TISbackup.git
    cd TISbackup
    
  2. Build the Docker image:

    docker build . -t tisbackup:latest
    
  3. Create directory structure:

    mkdir -p /var/tisbackup/{backup/log,config,ssh}
    

    Expected structure:

    /var/tisbackup/
    ├── backup/              # Backup storage location
    │   └── log/             # SQLite database and logs
    ├── config/              # Configuration files
    │   ├── tisbackup-config.ini
    │   └── tisbackup_gui.ini
    ├── ssh/                 # SSH keys
    │   ├── id_ed25519       # Private key (Ed25519 recommended)
    │   └── id_ed25519.pub   # Public key
    └── compose.yaml         # Docker Compose configuration
    
  4. Generate SSH keys:

    # Ed25519 (recommended - most secure and modern)
    ssh-keygen -t ed25519 -f /var/tisbackup/ssh/id_ed25519 -C "tisbackup@yourserver"
    
    # Or ECDSA (also secure)
    ssh-keygen -t ecdsa -b 521 -f /var/tisbackup/ssh/id_ecdsa -C "tisbackup@yourserver"
    
    # Or RSA (legacy support, minimum 4096 bits)
    ssh-keygen -t rsa -b 4096 -f /var/tisbackup/ssh/id_rsa -C "tisbackup@yourserver"
    

    ⚠️ Note: DSA keys are no longer supported due to security vulnerabilities.

  5. Deploy public key to remote servers:

    ssh-copy-id -i /var/tisbackup/ssh/id_ed25519.pub root@remote-server
    
  6. Generate Flask secret key:

    python3 -c "import secrets; print(secrets.token_hex(32))"
    

    Save this key for the next step.

  7. Create Docker Compose configuration:

    # /var/tisbackup/compose.yaml
    services:
      tisbackup_gui:
        container_name: tisbackup_gui
        image: "tisbackup:latest"
        volumes:
          - ./config/:/etc/tis/
          - ./backup/:/backup/
          - /etc/timezone:/etc/timezone:ro
          - /etc/localtime:/etc/localtime:ro
        environment:
          # SECURITY: Use the secret key you generated above
          - TISBACKUP_SECRET_KEY=your-secret-key-here
        restart: unless-stopped
        ports:
          - 9980:8080
    
      tisbackup_cron:
        container_name: tisbackup_cron
        image: "tisbackup:latest"
        volumes:
          - ./config/:/etc/tis/
          - ./ssh/:/config_ssh/
          - ./backup/:/backup/
          - /etc/timezone:/etc/timezone:ro
          - /etc/localtime:/etc/localtime:ro
        restart: always
        command: "/bin/bash /opt/tisbackup/cron.sh"
    
  8. Configure backups:

    Create /var/tisbackup/config/tisbackup-config.ini:

    [global]
    backup_base_dir = /backup/
    # Backup retention in days
    backup_retention_time = 90
    # Maximum backup age for Nagios checks (hours)
    maximum_backup_age = 30
    
    # Example: File backup via rsync
    [webserver-files]
    type = rsync+ssh
    server_name = webserver.example.com
    remote_dir = /var/www/
    compression = True
    exclude_list = "/var/www/cache/**","/var/www/temp/**"
    private_key = /config_ssh/id_ed25519
    ssh_port = 22
    
    # Example: MySQL database backup
    [database-mysql]
    type = mysql+ssh
    server_name = db.example.com
    db_name = production_db
    db_user = backup_user
    db_passwd = backup_password
    private_key = /config_ssh/id_ed25519
    ssh_port = 22
    

    Create /var/tisbackup/config/tisbackup_gui.ini:

    [general]
    config_tisbackup = /etc/tis/tisbackup-config.ini
    sections =
    ADMIN_EMAIL = admin@example.com
    base_config_dir = /etc/tis/
    backup_base_dir = /backup/
    
  9. Start services:

    cd /var/tisbackup
    docker compose up -d
    
  10. Access web interface:

    http://localhost:9980
    

Configuration

Backup Types Configuration

File Backups (rsync+ssh)

[backup-name]
type = rsync+ssh
server_name = hostname.example.com
remote_dir = /path/to/backup/
compression = True
exclude_list = "/path/exclude1/**","/path/exclude2/**"
private_key = /config_ssh/id_ed25519
ssh_port = 22

Btrfs Snapshots (rsync+btrfs+ssh)

[backup-name]
type = rsync+btrfs+ssh
server_name = hostname.example.com
remote_dir = /mnt/btrfs/data/
compression = True
private_key = /config_ssh/id_ed25519
ssh_port = 22

MySQL Database (mysql+ssh)

[backup-name]
type = mysql+ssh
server_name = hostname.example.com
db_name = database_name
db_user = backup_user
db_passwd = backup_password
private_key = /config_ssh/id_ed25519
ssh_port = 22

PostgreSQL Database (pgsql+ssh)

[backup-name]
type = pgsql+ssh
server_name = hostname.example.com
db_name = database_name
private_key = /config_ssh/id_ed25519
ssh_port = 22

XenServer VM (xen-xva)

[backup-name]
type = xen-xva
server_name = vm-name
xcphost = xenserver.example.com
password_file = /etc/tis/xen-password
private_key = /config_ssh/id_ed25519

Pre/Post Execution Hooks

You can execute commands before and after backups:

[backup-name]
type = rsync+ssh
server_name = hostname.example.com
remote_dir = /data/
private_key = /config_ssh/id_ed25519
preexec = systemctl stop application
postexec = systemctl start application
remote_user = root
ssh_port = 22

CLI Usage

Running Backups

# Run all backups
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py backup

# Run specific backup
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -s backup-name backup

# Dry run
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -d backup

Cleanup Old Backups

# Remove backups older than retention period
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py cleanup

Nagios Monitoring

# Check backup status
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py checknagios

# Check specific backup
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -s backup-name checknagios

List Available Drivers

docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py listdrivers

Backup Statistics

# Dump statistics for last 20 backups
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py dumpstat

# Specify number of backups
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -n 50 dumpstat

Development

Prerequisites

  • Python 3.13+
  • uv (Python package manager)

Setup Development Environment

# Install dependencies
uv sync --locked

# Run linter
uv run ruff check .

# Auto-fix linting issues
uv run ruff check --fix .

Running Locally

# Run web GUI (requires config at /etc/tis/tisbackup_gui.ini)
python3 tisbackup_gui.py

# Run CLI
python3 tisbackup.py -c /etc/tis/tisbackup-config.ini backup

Security

TISBackup implements several security best practices:

SSH Key Security

  • Ed25519 keys are recommended (most secure, modern algorithm)
  • ECDSA and RSA keys are supported
  • DSA keys are explicitly not supported (deprecated, insecure)
  • Key algorithm priority: Ed25519 → ECDSA → RSA

Flask Session Security

  • Secret key loaded from TISBACKUP_SECRET_KEY environment variable
  • Falls back to cryptographically secure random key if not set
  • No hardcoded secrets in source code

Command Execution Safety

  • All system commands use subprocess.run() with list arguments
  • Input validation for device paths and partition names
  • Timeout protection on all subprocess calls
  • No use of shell=True in new code

Best Practices

  1. Use Ed25519 keys for all SSH connections
  2. Set unique Flask secret key via environment variable
  3. Use reverse proxy (nginx) with TLS for web interface
  4. Restrict network access to backup server
  5. Regular security updates of base Docker image
  6. Monitor backup logs for suspicious activity

Reverse Proxy Setup

Example nginx configuration for HTTPS access:

server {
  listen 443 ssl http2;
  server_name tisbackup.example.com;

  ssl_certificate /etc/letsencrypt/live/tisbackup.example.com/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/tisbackup.example.com/privkey.pem;

  # Security headers
  add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
  add_header X-Frame-Options "SAMEORIGIN" always;
  add_header X-Content-Type-Options "nosniff" always;

  location / {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $host;
    proxy_pass http://localhost:9980/;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
  }
}

Architecture

TISBackup uses a modular driver-based architecture:

  • Core CLI (tisbackup.py): Backup orchestration and scheduling
  • Web GUI (tisbackup_gui.py): Flask-based management interface
  • Backup Drivers (libtisbackup/): Pluggable modules for different backup types
  • Task Queue (tasks.py, config.py): Async job processing with Huey
  • State Database: SQLite for tracking backup history and statistics

Each backup type is implemented as a driver class inheriting from backup_generic, allowing easy extension for new backup sources.

Troubleshooting

Backups Not Running

  1. Check cron logs:

    docker logs tisbackup_cron
    
  2. Verify SSH connectivity:

    docker exec tisbackup_cron ssh -i /config_ssh/id_ed25519 root@remote-server
    
  3. Check backup configuration:

    docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -c /etc/tis/tisbackup-config.ini -d backup
    

Web Interface Not Accessible

  1. Check GUI container logs:

    docker logs tisbackup_gui
    
  2. Verify port mapping:

    docker ps | grep tisbackup_gui
    
  3. Check configuration:

    docker exec tisbackup_gui cat /etc/tis/tisbackup_gui.ini
    

Permission Errors

Ensure proper file permissions:

chmod 600 /var/tisbackup/ssh/id_ed25519
chmod 644 /var/tisbackup/ssh/id_ed25519.pub
chown -R root:root /var/tisbackup/

Contributing

Contributions are welcome! Please:

  1. Fork the repository
  2. Create a feature branch
  3. Follow the existing code style (use ruff for linting)
  4. Add tests if applicable
  5. Submit a pull request

License

TISBackup is licensed under the GNU General Public License v3.0 (GPLv3).

See LICENSE for the full license text.

Support & Documentation

Credits

Developed by Tranquil IT for system administrators managing backup infrastructure.

Security improvements and modernization contributed by the community.