k3nny/TISbackup
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: k3nny:master
Branches Tags
k3nny:master
k3nny:feat/refacto
k3nny:py3
..
compare: k3nny:feat/refacto
Branches Tags
k3nny:master
k3nny:feat/refacto
k3nny:py3

No commits in common. "master" and "feat/refacto" have entirely different histories.
master ... feat/refac

53 changed files with 950 additions and 6250 deletions
Show Stats Expand all files Collapse all files

137
.gitignore vendored
View File

@ -1,137 +1,20 @@
# ===============================================
# TISBackup .gitignore
# ===============================================
# Python compiled files
# ===============================================
*.pyc
*.pyo
*.pyd
__pycache__/
*.so
*.egg
*.egg-info/
dist/
build/
*.whl
# Python virtual environments
# ===============================================
.venv/
venv/
env/
ENV/
.Python
# IDE and editor files
# ===============================================
.idea/
.vscode/
*.bak
*.swp
*.swo
*~
.DS_Store
Thumbs.db
*.sublime-project
*.sublime-workspace
# Testing and coverage
# ===============================================
.pytest_cache/
.coverage
.coverage.*
htmlcov/
.tox/
.nox/
coverage.xml
*.cover
.hypothesis/
# Linting and type checking
# ===============================================
*.pyc
__pycache__/
.venv/
.ruff_cache/
.mypy_cache/
.dmypy.json
dmypy.json
.pylint.d/
# Backup and temporary files
# ===============================================
*.bak
*.backup
*.tmp
*.temp
*.old
*.orig
*.log
*.log.*
# TISBackup runtime files
# ===============================================
# Task queue database
/tasks.sqlite
/tasks.sqlite-wal
/srvinstallation
/tasks.sqlite-shm
# Local configuration (samples are tracked, local overrides are not)
/tisbackup-config.ini
/tisbackup_gui.ini
# Backup data and logs (should never be in git)
/backups/
/log/
*.sqlite-journal
# Build artifacts
# ===============================================
/deb/builddir/
.idea
/deb/builddir
/deb/*.deb
/lib
/rpm/*.rpm
/rpm/RPMS/
/rpm/BUILD/
/rpm/RPMS
/rpm/BUILD
/rpm/__VERSION__
/srvinstallation/
# Documentation builds
# ===============================================
docs-sphinx-rst/build/
docs/_build/
site/
# Package manager files
# ===============================================
pip-log.txt
pip-delete-this-directory.txt
# OS generated files
# ===============================================
.DS_Store
.DS_Store?
._*
.Spotlight-V100
.Trashes
ehthumbs.db
Thumbs.db
Desktop.ini
# Secret and sensitive files
# ===============================================
*.pem
*.key
*.cert
*.p12
*.pfx
.env
.env.*
!.env.example
secrets/
private/
# Claude Code files
# ===============================================
.claude/
# Project specific
# ===============================================
# Legacy library (should use libtisbackup instead)
/lib/

1
.python-version
View File

@ -1 +0,0 @@
3.13

410
AUTHENTICATION.md
View File

@ -1,410 +0,0 @@
# TISBackup Authentication System
TISBackup provides a pluggable authentication system for securing the Flask web interface. You can choose between multiple authentication methods based on your security requirements.
## Supported Authentication Methods
1. **None** - No authentication (default, NOT recommended for production)
2. **Basic Auth** - HTTP Basic Authentication with username/password
3. **Flask-Login** - Session-based authentication with username/password
4. **OAuth2** - OAuth authentication (Google, GitHub, GitLab, or generic provider)
## Quick Start
### 1. Choose Authentication Method
Add an `[authentication]` section to `/etc/tis/tisbackup_gui.ini`:
```ini
[authentication]
type = basic
username = admin
password = $2b$12$LQv3c1yqBWVHxkd0LHAkCOYz6TtxMQJqhN8/LewY5GyYWv.5qVQK6
use_bcrypt = True
```
### 2. Restart TISBackup
```bash
docker compose restart tisbackup_gui
```
## Configuration Guide
### Basic Authentication
Simple HTTP Basic Auth with username and password.
**Pros:**
- Easy to set up
- Works with all HTTP clients
- No session management needed
**Cons:**
- Credentials sent with every request
- No logout functionality
- Browser password prompt can be confusing
**Configuration:**
```ini
[authentication]
type = basic
username = admin
# Use bcrypt hash (recommended)
password = $2b$12$LQv3c1yqBWVHxkd0LHAkCOYz6TtxMQJqhN8/LewY5GyYWv.5qVQK6
use_bcrypt = True
realm = TISBackup Admin
```
**Generate bcrypt hash:**
```bash
python3 -c "import bcrypt; print(bcrypt.hashpw(b'yourpassword', bcrypt.gensalt()).decode())"
```
**Docker environment:**
```yaml
services:
tisbackup_gui:
environment:
- TISBACKUP_SECRET_KEY=your-secret-key
# Optional: Pass credentials via env vars
# Then reference in config with ${AUTH_PASSWORD}
```
---
### Flask-Login Authentication
Session-based authentication with login page and user management.
**Pros:**
- Clean login/logout workflow
- Session-based (no credentials in each request)
- Multiple users supported
- Password hashing with bcrypt
**Cons:**
- Requires custom login page
- Session management overhead
- Cookies must be enabled
**Configuration:**
```ini
[authentication]
type = flask-login
users_file = /etc/tis/users.txt
use_bcrypt = True
login_view = login
```
**Create users file** (`/etc/tis/users.txt`):
```
admin:$2b$12$LQv3c1yqBWVHxkd0LHAkCOYz6TtxMQJqhN8/LewY5GyYWv.5qVQK6
operator:$2b$12$abcdefghijklmnopqrstuvwxyz1234567890ABCDEFGHIJKLMNO
viewer:$2b$12$ANOTHERBCRYPTHASHHERE1234567890ABCDEFGHIJKLMNOPQRS
```
**Generate user entry:**
```bash
USERNAME="admin"
PASSWORD="yourpassword"
HASH=$(python3 -c "import bcrypt; print(bcrypt.hashpw(b'$PASSWORD', bcrypt.gensalt()).decode())")
echo "$USERNAME:$HASH" >> /etc/tis/users.txt
```
**Permissions:**
```bash
chmod 600 /etc/tis/users.txt
chown root:root /etc/tis/users.txt
```
---
### OAuth2 Authentication
Delegate authentication to external OAuth providers (Google, GitHub, GitLab, etc.)
**Pros:**
- No password management
- Leverage existing identity providers
- Support for SSO
- Can restrict by domain or specific users
**Cons:**
- Requires OAuth app registration
- Internet connectivity required
- More complex setup
- External dependency
#### Google OAuth
**Setup:**
1. Go to [Google Cloud Console](https://console.cloud.google.com/apis/credentials)
2. Create OAuth 2.0 Client ID
3. Add authorized redirect URI: `http://your-server:8080/oauth/callback`
4. Note the Client ID and Client Secret
**Configuration:**
```ini
[authentication]
type = oauth
provider = google
client_id = 123456789-abcdefghijklmnop.apps.googleusercontent.com
client_secret = GOCSPX-your-client-secret-here
redirect_uri = http://your-server:8080/oauth/callback
# Restrict to specific domain(s)
authorized_domains = example.com,mycompany.com
# Or restrict to specific users
authorized_users = admin@example.com,backup-admin@example.com
```
#### GitHub OAuth
**Setup:**
1. Go to GitHub Settings > Developer settings > [OAuth Apps](https://github.com/settings/developers)
2. Register a new application
3. Set Authorization callback URL: `http://your-server:8080/oauth/callback`
4. Note the Client ID and Client Secret
**Configuration:**
```ini
[authentication]
type = oauth
provider = github
client_id = your-github-client-id
client_secret = your-github-client-secret
redirect_uri = http://your-server:8080/oauth/callback
authorized_users = admin@example.com,devops@example.com
```
#### GitLab OAuth
**Setup:**
1. Go to GitLab User Settings > [Applications](https://gitlab.com/-/profile/applications)
2. Create application with scopes: `read_user`, `email`
3. Set Redirect URI: `http://your-server:8080/oauth/callback`
4. Note the Application ID and Secret
**Configuration:**
```ini
[authentication]
type = oauth
provider = gitlab
client_id = your-gitlab-application-id
client_secret = your-gitlab-secret
redirect_uri = http://your-server:8080/oauth/callback
authorized_domains = example.com
```
#### Generic OAuth Provider
For custom OAuth providers:
```ini
[authentication]
type = oauth
provider = generic
client_id = your-client-id
client_secret = your-client-secret
redirect_uri = http://your-server:8080/oauth/callback
# Custom endpoints
authorization_endpoint = https://auth.example.com/oauth/authorize
token_endpoint = https://auth.example.com/oauth/token
userinfo_endpoint = https://auth.example.com/oauth/userinfo
scopes = openid,email,profile
# Authorization rules
authorized_domains = example.com
```
---
## Security Best Practices
### 1. Use HTTPS in Production
Always use a reverse proxy with TLS:
```nginx
server {
listen 443 ssl http2;
server_name tisbackup.example.com;
ssl_certificate /etc/letsencrypt/live/tisbackup.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/tisbackup.example.com/privkey.pem;
location / {
proxy_pass http://localhost:8080/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
}
}
```
### 2. Set Strong Flask Secret Key
```bash
# Generate secret
python3 -c "import secrets; print(secrets.token_hex(32))"
# Set in environment
export TISBACKUP_SECRET_KEY=your-generated-secret-key
```
### 3. Protect Configuration Files
```bash
chmod 600 /etc/tis/tisbackup_gui.ini
chmod 600 /etc/tis/users.txt # if using Flask-Login
chown root:root /etc/tis/*.ini
```
### 4. Use Environment Variables for Secrets
Instead of hardcoding secrets in config files:
```ini
[authentication]
type = oauth
client_id = ${OAUTH_CLIENT_ID}
client_secret = ${OAUTH_CLIENT_SECRET}
```
Then set in Docker Compose:
```yaml
services:
tisbackup_gui:
environment:
- OAUTH_CLIENT_ID=your-client-id
- OAUTH_CLIENT_SECRET=your-client-secret
- TISBACKUP_SECRET_KEY=your-secret-key
```
### 5. Regularly Rotate Credentials
- Change passwords/secrets every 90 days
- Rotate OAuth client secrets annually
- Review authorized users/domains regularly
### 6. Monitor Authentication Logs
Check logs for failed authentication attempts:
```bash
docker logs tisbackup_gui | grep -i "auth"
```
## Troubleshooting
### Basic Auth Not Working
1. **Verify password hash:**
```bash
python3 -c "import bcrypt; print(bcrypt.checkpw(b'yourpassword', b'$2b$12$your-hash'))"
```
3. **Check browser credentials:**
- Clear browser cache
- Try incognito/private mode
### Flask-Login Issues
1. **Users file not found:**
```bash
ls -la /etc/tis/users.txt
chmod 600 /etc/tis/users.txt
```
2. **Session problems:**
- Check `TISBACKUP_SECRET_KEY` is set
- Ensure cookies are enabled
### OAuth Problems
1. **Redirect URI mismatch:**
- Ensure redirect URI in config matches OAuth app settings exactly
- Check for http vs https mismatch
2. **Unauthorized domain/user:**
- Verify email matches `authorized_users` or domain matches `authorized_domains`
- Check OAuth provider returns email in user info
3. **Token errors:**
- Verify client ID and secret are correct
- Check OAuth app is enabled
- Ensure scopes are correct
## API Access with Authentication
### Basic Auth
```bash
curl -u admin:password http://localhost:8080/api/backups
```
### OAuth (with access token)
```bash
# Not recommended for API access - use service account or API keys instead
```
### Recommendation for API Access
For programmatic API access, use Basic Auth with a dedicated API user:
```ini
[authentication]
type = basic
username = api-user
password = $2b$12$...
```
Or implement API key authentication separately for API endpoints.
## Migration Guide
### From No Auth to Basic Auth
1. Add authentication section to config
2. Restart service
3. Update client scripts with credentials
### From Basic Auth to OAuth
1. Register OAuth application
2. Update configuration
3. Test OAuth login flow
4. Update redirect URI if needed
### From Flask-Login to OAuth
1. Register OAuth application
2. Map user emails to OAuth provider
3. Update configuration
4. Test migration with test users first
## Support
For issues or questions:
- Check logs: `docker logs tisbackup_gui`
- Review configuration syntax
- Verify dependencies are installed
- See [SECURITY_IMPROVEMENTS.md](../SECURITY_IMPROVEMENTS.md) for security context

253
CLAUDE.md
View File

@ -1,253 +0,0 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
TISBackup is a server-side backup orchestration system written in Python. It executes scheduled backups of various data sources (databases, files, VMs) from remote Linux and Windows systems. The project consists of:
- A CLI tool ([tisbackup.py](tisbackup.py)) for executing backups, cleanup, and monitoring
- A Flask web GUI ([tisbackup_gui.py](tisbackup_gui.py)) for managing backups
- A pluggable backup driver architecture in [libtisbackup/](libtisbackup/)
- Task queue system using Huey with Redis ([tasks.py](tasks.py), [config.py](config.py))
- Docker-based deployment with cron scheduling
## Development Commands
**IMPORTANT: Always use `uv run` to execute Python commands in this project.**
### Dependency Management
```bash
# Install dependencies (uses uv)
uv sync --locked
# Update dependencies
uv lock
```
### Linting
```bash
# Run ruff linter (fast, primary linter)
uv run ruff check .
# Auto-fix linting issues
uv run ruff check --fix .
# Run pylint (comprehensive static analysis)
uv run pylint libtisbackup/
# Run pylint on specific file
uv run pylint libtisbackup/ssh.py
```
### Testing
```bash
# Run all tests
uv run pytest
# Run tests for specific module
uv run pytest tests/test_ssh.py
# Run with verbose output
uv run pytest -v
# Run tests matching a pattern
uv run pytest -k "ssh"
# Run with coverage report
uv run pytest --cov=libtisbackup --cov-report=html --cov-report=term-missing
# Run tests with coverage and show only missing lines
uv run pytest --cov=libtisbackup --cov-report=term-missing
# Generate HTML coverage report (opens in browser)
uv run pytest --cov=libtisbackup --cov-report=html
# Then open htmlcov/index.html
```
**Coverage reports:**
- Terminal report: Shows coverage percentage with missing line numbers
- HTML report: Detailed interactive report in `htmlcov/` directory
See [tests/README.md](tests/README.md) for detailed testing documentation.
### Running the Application
**Web GUI (development):**
```bash
uv run python tisbackup_gui.py
# Runs on port 8080, requires config at /etc/tis/tisbackup_gui.ini
```
**CLI Commands:**
```bash
# Run backups
uv run python tisbackup.py -c /etc/tis/tisbackup-config.ini backup
# Run specific backup section
uv run python tisbackup.py -c /etc/tis/tisbackup-config.ini -s section_name backup
# Cleanup old backups
uv run python tisbackup.py -c /etc/tis/tisbackup-config.ini cleanup
# Check backup status (for Nagios)
uv run python tisbackup.py -c /etc/tis/tisbackup-config.ini checknagios
# List available backup drivers
uv run python tisbackup.py listdrivers
```
### Docker
```bash
# Build image
docker build . -t tisbackup:latest
# Run via docker compose (see README.md for full setup)
docker compose up -d
```
## Architecture
### Core Components
**Main Entry Points:**
- [tisbackup.py](tisbackup.py) - CLI application with argument parsing and action routing (backup, cleanup, checknagios, etc.)
- [tisbackup_gui.py](tisbackup_gui.py) - Flask web application providing UI for backup management and status monitoring
- [tasks.py](tasks.py) - Huey task definitions for async operations (export_backup)
**Backup Driver System:**
All backup logic is implemented via driver classes in [libtisbackup/drivers/](libtisbackup/drivers/):
- Base class: `backup_generic` in [base_driver.py](libtisbackup/base_driver.py) (abstract)
- Each driver inherits from `backup_generic` and implements specific backup logic
- Drivers are registered via the `register_driver()` decorator function
- Configuration is read from INI files using the `read_config()` method
- All driver implementations are in [libtisbackup/drivers/](libtisbackup/drivers/) subdirectory
**Library Modules:**
- [base_driver.py](libtisbackup/base_driver.py) - Core `backup_generic` class, driver registry, Nagios states
- [database.py](libtisbackup/database.py) - `BackupStat` class for SQLite operations
- [ssh.py](libtisbackup/ssh.py) - SSH utilities with modern key support (Ed25519, ECDSA, RSA)
- [process.py](libtisbackup/process.py) - Process execution and monitoring utilities
- [utils.py](libtisbackup/utils.py) - Date/time formatting, number formatting, validation helpers
- [__init__.py](libtisbackup/__init__.py) - Package exports for backward compatibility
- [drivers/](libtisbackup/drivers/) - All backup driver implementations
**Available Drivers:**
- `backup_rsync` / `backup_rsync_ssh` - File-based backups via rsync
- `backup_rsync_btrfs` / `backup_rsync__btrfs_ssh` - Btrfs snapshot-based backups
- `backup_mysql` - MySQL database dumps
- `backup_pgsql` - PostgreSQL database dumps
- `backup_oracle` - Oracle database backups
- `backup_sqlserver` - SQL Server backups
- `backup_samba4` - Samba4 AD backups
- `backup_xva` / `backup_xcp_metadata` / `copy_vm_xcp` - XenServer VM backups
- `backup_vmdk` - VMware VMDK backups (requires pyVmomi)
- `backup_switch` - Network switch configuration backups
- `backup_null` - No-op driver for testing
**State Management:**
- SQLite database tracks backup history, status, and statistics
- `BackupStat` class in [common.py](libtisbackup/common.py) handles DB operations
- Database location: `{backup_base_dir}/log/tisbackup.sqlite`
### Configuration
Two separate INI configuration files:
1. **tisbackup-config.ini** - Backup definitions
- `[global]` section with defaults (backup_base_dir, backup_retention_time, maximum_backup_age)
- One section per backup job with driver type and parameters
2. **tisbackup_gui.ini** - GUI settings
- Points to tisbackup-config.ini location(s)
- Defines admin email, base directories
### Task Queue
- Uses Huey (Redis-backed) for async job processing
- Currently implements `run_export_backup` for exporting backups to external storage
- Task state tracked in tasks.sqlite
### Docker Deployment
Two-container architecture:
- **tisbackup_gui**: Runs Flask web interface
- **tisbackup_cron**: Runs scheduled backups via cron (executes [backup.sh](backup.sh) at 03:59 daily)
## Code Style
- Line length: 140 characters (configured in pyproject.toml)
- Ruff ignores: F401, F403, F405, E402, E701, E722, E741
- Python 3.13+ required
## Commit Message Guidelines
**IMPORTANT: This project uses [Conventional Commits](https://www.conventionalcommits.org/) format.**
All commit messages must follow this format:
```
<type>(<scope>): <description>
[optional body]
[optional footer(s)]
```
**Types:**
- `feat`: A new feature
- `fix`: A bug fix
- `docs`: Documentation only changes
- `refactor`: Code change that neither fixes a bug nor adds a feature
- `test`: Adding missing tests or correcting existing tests
- `chore`: Changes to build process or auxiliary tools
- `perf`: Performance improvements
- `style`: Code style changes (formatting, missing semicolons, etc.)
**Scopes (commonly used):**
- `auth`: Authentication/authorization changes
- `security`: Security-related changes
- `drivers`: Backup driver changes
- `gui`: Web GUI changes
- `api`: API changes
- `readme`: README.md changes
- `claude`: CLAUDE.md changes
- `core`: Core library changes
**Examples:**
- `feat(auth): add pluggable authentication system for Flask routes`
- `fix(security): replace os.popen/os.system with subprocess`
- `docs(readme): add comprehensive security and authentication documentation`
- `refactor(drivers): organize backup modules into drivers subfolder`
- `chore(deps): add pyvmomi as mandatory dependency`
**Breaking Changes:**
Add `!` after type/scope for breaking changes:
- `feat(api)!: remove deprecated endpoint`
**Note:** Always include a scope in parentheses, even for documentation changes.
When Claude Code creates commits, it will automatically follow this format.
## Important Patterns
**Adding a new backup driver:**
1. Create `backup_<type>.py` in [libtisbackup/drivers/](libtisbackup/drivers/)
2. Inherit from `backup_generic`
3. Set class attributes: `type`, `required_params`, `optional_params`
4. Implement abstract methods: `do_backup()`, `cleanup()`, `checknagios()`
5. Register with `register_driver(backup_<type>)`
6. Import in [libtisbackup/drivers/__init__.py](libtisbackup/drivers/__init__.py)
**SSH Operations:**
- Uses paramiko for SSH connections
- Supports both RSA and DSA keys
- Private key path specified per backup section via `private_key` parameter
- Pre/post-exec hooks run remote commands via SSH
**Path Handling:**
- Module imports use sys.path manipulation to include lib/ and libtisbackup/
- All backup drivers expect absolute paths for backup_dir
- Backup directory structure: `{backup_base_dir}/{section_name}/{timestamp}/`

1
Dockerfile
View File

@ -13,6 +13,7 @@ ENV UV_PYTHON_DOWNLOADS=never
RUN apt-get update && apt-get upgrade -y \
&& apt-get install --no-install-recommends -y rsync ssh cron \
&& rm -rf /var/lib/apt/lists/* \
#&& /usr/local/bin/python3.13 -m pip install --no-cache-dir -r requirements.txt \
&& uv sync --locked --no-dev --no-install-project \
&& rm -f /bin/uv /bin/uvx \
&& mkdir -p /var/spool/cron/crontabs \

536
README.md
View File

@ -1,483 +1,145 @@
# TISBackup
A comprehensive server-side backup orchestration system for managing automated backups of databases, files, and virtual machines across remote Linux and Windows systems.
This is the repository of the TISBackup project, licensed under GPLv3.
[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
[![Python 3.13+](https://img.shields.io/badge/python-3.13+-blue.svg)](https://www.python.org/downloads/)
TISBackup is a python script to backup servers.
## Overview
It runs at regular intervals to retrieve different data types on remote hosts
such as database dumps, files, virtual machine images and metadata.
TISBackup is a Python-based backup solution that provides:
## Install using Compose
- **Pluggable backup drivers** for different data sources (databases, files, VMs)
- **Web-based management interface** for monitoring and controlling backups
- **CLI tool** for automation and scripting
- **Automated scheduling** via cron
- **Backup retention management** with configurable policies
- **Status monitoring** with Nagios integration
- **Docker deployment** for easy setup and isolation
Clone that repository and build the pod image using the provided `Dockerfile`
### Supported Backup Types
```bash
docker build . -t tisbackup:latest
```
| Type | Description | Driver |
|------|-------------|--------|
| **Files & Directories** | rsync-based file backups | `rsync+ssh` |
| **Btrfs Snapshots** | Snapshot-based incremental backups | `rsync+btrfs+ssh` |
| **MySQL** | Database dumps via SSH | `mysql+ssh` |
| **PostgreSQL** | Database dumps via SSH | `pgsql+ssh` |
| **SQL Server** | SQL Server backups | `sqlserver+ssh` |
| **Oracle** | Oracle database backups | `oracle+ssh` |
| **Samba4 AD** | Active Directory backups | `samba4` |
| **XenServer VMs** | XVA exports and metadata | `xen-xva`, `xcp-dump-metadata` |
| **VMware** | VMDK backups | `vmdk` |
| **Network Devices** | Switch configuration backups | `switch` |
In another folder, create subfolders as following
## Quick Start
```bash
mkdir -p /var/tisbackup/{backup/log,config,ssh}/
```
### Prerequisites
Expected structure
```
/var/tisbackup/
└─backup/ <-- backup location
└─config/
├── tisbackup-config.ini <-- backups config
└── tisbackup_gui.ini <-- tisbackup config
└─ssh/
├── id_rsa <-- SSH Key
└── id_rsa.pub <-- SSH PubKey
compose.yaml
```
- Docker and Docker Compose
- SSH access to remote servers
- Ed25519, ECDSA, or RSA SSH keys (DSA not supported)
Adapt the compose.yml file to suits your needs, one pod act as the WebUI front end and the other as the crond scheduler
### Installation
```yaml
services:
tisbackup_gui:
container_name: tisbackup_gui
image: "tisbackup:latest"
build: .
volumes:
- ./config/:/etc/tis/
- ./backup/:/backup/
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
restart: unless-stopped
ports:
- 9980:8080
1. **Clone the repository:**
```bash
git clone https://github.com/tranquilit/TISbackup.git
cd TISbackup
```
tisbackup_cron:
container_name: tisbackup_cron
image: "tisbackup:latest"
build: .
volumes:
- ./config/:/etc/tis/
- ./ssh/:/config_ssh/
- ./backup/:/backup/
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
restart: always
command: "/bin/bash /opt/tisbackup/cron.sh"
2. **Build the Docker image:**
```bash
docker build . -t tisbackup:latest
```
3. **Create directory structure:**
```bash
mkdir -p /var/tisbackup/{backup/log,config,ssh}
```
Expected structure:
```
/var/tisbackup/
├── backup/ # Backup storage location
│ └── log/ # SQLite database and logs
├── config/ # Configuration files
│ ├── tisbackup-config.ini
│ └── tisbackup_gui.ini
├── ssh/ # SSH keys
│ ├── id_ed25519 # Private key (Ed25519 recommended)
│ └── id_ed25519.pub # Public key
└── compose.yaml # Docker Compose configuration
```
4. **Generate SSH keys:**
```bash
# Ed25519 (recommended - most secure and modern)
ssh-keygen -t ed25519 -f /var/tisbackup/ssh/id_ed25519 -C "tisbackup@yourserver"
# Or ECDSA (also secure)
ssh-keygen -t ecdsa -b 521 -f /var/tisbackup/ssh/id_ecdsa -C "tisbackup@yourserver"
# Or RSA (legacy support, minimum 4096 bits)
ssh-keygen -t rsa -b 4096 -f /var/tisbackup/ssh/id_rsa -C "tisbackup@yourserver"
```
⚠️ **Note:** DSA keys are no longer supported due to security vulnerabilities.
5. **Deploy public key to remote servers:**
```bash
ssh-copy-id -i /var/tisbackup/ssh/id_ed25519.pub root@remote-server
```
6. **Generate Flask secret key:**
```bash
python3 -c "import secrets; print(secrets.token_hex(32))"
```
Save this key for the next step.
7. **Create Docker Compose configuration:**
```yaml
# /var/tisbackup/compose.yaml
services:
tisbackup_gui:
container_name: tisbackup_gui
image: "tisbackup:latest"
volumes:
- ./config/:/etc/tis/
- ./backup/:/backup/
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
environment:
# SECURITY: Use the secret key you generated above
- TISBACKUP_SECRET_KEY=your-secret-key-here
restart: unless-stopped
ports:
- 9980:8080
tisbackup_cron:
container_name: tisbackup_cron
image: "tisbackup:latest"
volumes:
- ./config/:/etc/tis/
- ./ssh/:/config_ssh/
- ./backup/:/backup/
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
restart: always
command: "/bin/bash /opt/tisbackup/cron.sh"
```
8. **Configure backups:**
Create `/var/tisbackup/config/tisbackup-config.ini`:
```ini
[global]
backup_base_dir = /backup/
# Backup retention in days
backup_retention_time = 90
# Maximum backup age for Nagios checks (hours)
maximum_backup_age = 30
# Example: File backup via rsync
[webserver-files]
type = rsync+ssh
server_name = webserver.example.com
remote_dir = /var/www/
compression = True
exclude_list = "/var/www/cache/**","/var/www/temp/**"
private_key = /config_ssh/id_ed25519
ssh_port = 22
# Example: MySQL database backup
[database-mysql]
type = mysql+ssh
server_name = db.example.com
db_name = production_db
db_user = backup_user
db_passwd = backup_password
private_key = /config_ssh/id_ed25519
ssh_port = 22
```
Create `/var/tisbackup/config/tisbackup_gui.ini`:
```ini
[general]
config_tisbackup = /etc/tis/tisbackup-config.ini
sections =
ADMIN_EMAIL = admin@example.com
base_config_dir = /etc/tis/
backup_base_dir = /backup/
```
9. **Start services:**
```bash
cd /var/tisbackup
docker compose up -d
```
10. **Access web interface:**
```
http://localhost:9980
```
```
## Configuration
### Backup Types Configuration
* Provide an SSH key and store it in `./ssh`
* Setup config files in the `./config` directory
**tisbackup-config.ini**
#### File Backups (rsync+ssh)
```ini
[backup-name]
type = rsync+ssh
server_name = hostname.example.com
remote_dir = /path/to/backup/
compression = True
exclude_list = "/path/exclude1/**","/path/exclude2/**"
private_key = /config_ssh/id_ed25519
[global]
backup_base_dir = /backup/
# backup retention in days
backup_retention_time=90
# for nagios check in hours
maximum_backup_age=30
[srvads-poudlard-samba]
type=rsync+ssh
server_name=srvads.poudlard.lan
remote_dir=/var/lib/samba/
compression=True
;exclude_list="/proc/**","/sys/**","/dev/**"
private_key=/config_ssh/id_rsa
ssh_port = 22
```
#### Btrfs Snapshots (rsync+btrfs+ssh)
**tisbackup_gui.ini**
```ini
[backup-name]
type = rsync+btrfs+ssh
server_name = hostname.example.com
remote_dir = /mnt/btrfs/data/
compression = True
private_key = /config_ssh/id_ed25519
ssh_port = 22
[general]
config_tisbackup= /etc/tis/tisbackup-config.ini
sections=
ADMIN_EMAIL=josebove@internet.fr
base_config_dir= /etc/tis/
backup_base_dir=/backup/
```
#### MySQL Database (mysql+ssh)
```ini
[backup-name]
type = mysql+ssh
server_name = hostname.example.com
db_name = database_name
db_user = backup_user
db_passwd = backup_password
private_key = /config_ssh/id_ed25519
ssh_port = 22
```
#### PostgreSQL Database (pgsql+ssh)
```ini
[backup-name]
type = pgsql+ssh
server_name = hostname.example.com
db_name = database_name
private_key = /config_ssh/id_ed25519
ssh_port = 22
```
#### XenServer VM (xen-xva)
```ini
[backup-name]
type = xen-xva
server_name = vm-name
xcphost = xenserver.example.com
password_file = /etc/tis/xen-password
private_key = /config_ssh/id_ed25519
```
### Pre/Post Execution Hooks
You can execute commands before and after backups:
```ini
[backup-name]
type = rsync+ssh
server_name = hostname.example.com
remote_dir = /data/
private_key = /config_ssh/id_ed25519
preexec = systemctl stop application
postexec = systemctl start application
remote_user = root
ssh_port = 22
```
## CLI Usage
### Running Backups
Run!
```bash
# Run all backups
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py backup
# Run specific backup
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -s backup-name backup
# Dry run
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -d backup
docker compose up -d
```
### Cleanup Old Backups
## NGINX reverse-proxy
```bash
# Remove backups older than retention period
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py cleanup
```
### Nagios Monitoring
```bash
# Check backup status
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py checknagios
# Check specific backup
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -s backup-name checknagios
```
### List Available Drivers
```bash
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py listdrivers
```
### Backup Statistics
```bash
# Dump statistics for last 20 backups
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py dumpstat
# Specify number of backups
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -n 50 dumpstat
```
## Development
### Prerequisites
- Python 3.13+
- uv (Python package manager)
### Setup Development Environment
```bash
# Install dependencies
uv sync --locked
# Run linter
uv run ruff check .
# Auto-fix linting issues
uv run ruff check --fix .
```
### Running Locally
```bash
# Run web GUI (requires config at /etc/tis/tisbackup_gui.ini)
python3 tisbackup_gui.py
# Run CLI
python3 tisbackup.py -c /etc/tis/tisbackup-config.ini backup
```
## Security
TISBackup implements several security best practices:
### SSH Key Security
- **Ed25519 keys are recommended** (most secure, modern algorithm)
- ECDSA and RSA keys are supported
- **DSA keys are explicitly not supported** (deprecated, insecure)
- Key algorithm priority: Ed25519 → ECDSA → RSA
### Flask Session Security
- Secret key loaded from `TISBACKUP_SECRET_KEY` environment variable
- Falls back to cryptographically secure random key if not set
- No hardcoded secrets in source code
### Command Execution Safety
- All system commands use `subprocess.run()` with list arguments
- Input validation for device paths and partition names
- Timeout protection on all subprocess calls
- No use of `shell=True` in new code
### Best Practices
1. **Use Ed25519 keys** for all SSH connections
2. **Set unique Flask secret key** via environment variable
3. **Use reverse proxy** (nginx) with TLS for web interface
4. **Restrict network access** to backup server
5. **Regular security updates** of base Docker image
6. **Monitor backup logs** for suspicious activity
## Reverse Proxy Setup
Example nginx configuration for HTTPS access:
Sample config file
```nginx
server {
listen 443 ssl http2;
server_name tisbackup.example.com;
# Remove '#' in the next line to enable IPv6
# listen [::]:443 ssl http2;
server_name tisbackup.poudlard.lan;
ssl_certificate /etc/letsencrypt/live/tisbackup.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/tisbackup.example.com/privkey.pem;
ssl_certificate /etc/letsencrypt/live/tisbackup.poudlard.lan/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/tisbackup.poudlard.lan/privkey.pem; # managed by Certbot
# Security headers
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_pass http://localhost:9980/;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_pass http://localhost:9980/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
```
## Architecture
TISBackup uses a modular driver-based architecture:
- **Core CLI** ([tisbackup.py](tisbackup.py)): Backup orchestration and scheduling
- **Web GUI** ([tisbackup_gui.py](tisbackup_gui.py)): Flask-based management interface
- **Backup Drivers** ([libtisbackup/](libtisbackup/)): Pluggable modules for different backup types
- **Task Queue** ([tasks.py](tasks.py), [config.py](config.py)): Async job processing with Huey
- **State Database**: SQLite for tracking backup history and statistics
## About
Each backup type is implemented as a driver class inheriting from `backup_generic`, allowing easy extension for new backup sources.
[Tranquil IT](contact_at_tranquil_it) is the original author of TISBackup.
## Troubleshooting
### Backups Not Running
1. Check cron logs:
```bash
docker logs tisbackup_cron
```
2. Verify SSH connectivity:
```bash
docker exec tisbackup_cron ssh -i /config_ssh/id_ed25519 root@remote-server
```
3. Check backup configuration:
```bash
docker exec tisbackup_cron python3 /opt/tisbackup/tisbackup.py -c /etc/tis/tisbackup-config.ini -d backup
```
### Web Interface Not Accessible
1. Check GUI container logs:
```bash
docker logs tisbackup_gui
```
2. Verify port mapping:
```bash
docker ps | grep tisbackup_gui
```
3. Check configuration:
```bash
docker exec tisbackup_gui cat /etc/tis/tisbackup_gui.ini
```
### Permission Errors
Ensure proper file permissions:
```bash
chmod 600 /var/tisbackup/ssh/id_ed25519
chmod 644 /var/tisbackup/ssh/id_ed25519.pub
chown -R root:root /var/tisbackup/
```
## Contributing
Contributions are welcome! Please:
1. Fork the repository
2. Create a feature branch
3. Follow the existing code style (use `ruff` for linting)
4. Add tests if applicable
5. Submit a pull request
## License
TISBackup is licensed under the GNU General Public License v3.0 (GPLv3).
See [LICENSE](LICENSE) for the full license text.
## Support & Documentation
- **Documentation**: [https://tisbackup.readthedocs.io](https://tisbackup.readthedocs.io/en/latest/index.html)
- **Issues**: [GitHub Issues](https://github.com/tranquilit/TISbackup/issues)
- **Original Author**: [Tranquil IT](https://www.tranquil.it)
## Credits
Developed by Tranquil IT for system administrators managing backup infrastructure.
Security improvements and modernization contributed by the community.
The documentation is provided under the license CC-BY-SA and can be found
on [readthedoc](https://tisbackup.readthedocs.io/en/latest/index.html).

149
REFACTORING.md
View File

@ -1,149 +0,0 @@
# TISBackup Refactoring Summary
## Overview
Successfully refactored the monolithic `libtisbackup/common.py` (1079 lines, 42KB) into focused, maintainable modules with clear separation of concerns.
## New Module Structure
### 1. **[utils.py](libtisbackup/utils.py)** - 6.7KB
Utility functions for formatting and data manipulation:
- **Date/Time helpers**: `datetime2isodate`, `isodate2datetime`, `time2display`, `hours_minutes`, `fileisodate`, `dateof`
- **Number formatting**: `splitThousands`, `convert_bytes`
- **Display helpers**: `pp` (pretty-print tables), `html_table`
- **Validation**: `check_string`, `str2bool`
### 2. **[ssh.py](libtisbackup/ssh.py)** - 3.4KB
SSH operations and key management:
- **`load_ssh_private_key()`**: Modern SSH key loading with Ed25519, ECDSA, and RSA support
- **`ssh_exec()`**: Execute commands on remote servers via SSH
### 3. **[process.py](libtisbackup/process.py)** - 3.4KB
Process execution utilities:
- **`call_external_process()`**: Execute shell commands with error handling
- **`monitor_stdout()`**: Real-time process output monitoring with callbacks
### 4. **[database.py](libtisbackup/database.py)** - 8.3KB
SQLite database management for backup statistics:
- **`BackupStat` class**: Complete state management for backup history
- Database initialization and schema updates
- Backup tracking (start, finish, query)
- Formatted output (HTML, text tables)
### 5. **[base_driver.py](libtisbackup/base_driver.py)** - 25KB
Core backup driver architecture:
- **`backup_generic`**: Abstract base class for all backup drivers
- **`register_driver()`**: Driver registration system
- **`backup_drivers`**: Global driver registry
- **Nagios constants**: `nagiosStateOk`, `nagiosStateWarning`, `nagiosStateCritical`, `nagiosStateUnknown`
- Core backup logic: process_backup, cleanup_backup, checknagios, export_latestbackup
### 6. **[__init__.py](libtisbackup/__init__.py)** - 2.5KB
Package initialization with backward compatibility:
- Re-exports all public APIs from new modules
- Maintains 100% backward compatibility with existing code
- Clear `__all__` declaration for IDE support
## Migration Details
### Changed Imports
All imports have been automatically updated:
```python
# Old (common.py)
from libtisbackup.common import *
from .common import *
# New (modular structure)
from libtisbackup import *
```
### Backward Compatibility
✅ **100% backward compatible** - All existing code continues to work without changes
✅ The `__init__.py` re-exports everything that was previously in `common.py`
✅ All 12 backup drivers verified and working
✅ Main CLI (`tisbackup.py`) tested successfully
✅ GUI (`tisbackup_gui.py`) imports verified
## Benefits
### Maintainability
- **Single Responsibility**: Each module has one clear purpose
- **Easier Navigation**: Find functionality quickly by module name
- **Reduced Complexity**: Smaller files are easier to understand
### Testability
- Can test SSH, database, process, and backup logic independently
- Mock individual modules for unit testing
- Clearer boundaries for integration tests
### Developer Experience
- Better IDE autocomplete and navigation
- Explicit imports reduce cognitive load
- Clear module boundaries aid code review
### Performance
- Import only what you need (reduces memory footprint)
- Faster module loading for targeted imports
## Files Modified
### Created (6 new files)
- `libtisbackup/utils.py`
- `libtisbackup/ssh.py`
- `libtisbackup/process.py`
- `libtisbackup/database.py`
- `libtisbackup/base_driver.py`
- `libtisbackup/__init__.py` (updated)
### Backed Up
- `libtisbackup/common.py` → `libtisbackup/common.py.bak` (preserved for reference)
### Updated (15 files)
All backup drivers and main scripts updated to use new imports:
- `libtisbackup/backup_mysql.py`
- `libtisbackup/backup_null.py`
- `libtisbackup/backup_oracle.py`
- `libtisbackup/backup_pgsql.py`
- `libtisbackup/backup_rsync.py`
- `libtisbackup/backup_rsync_btrfs.py`
- `libtisbackup/backup_samba4.py`
- `libtisbackup/backup_sqlserver.py`
- `libtisbackup/backup_switch.py`
- `libtisbackup/backup_vmdk.py`
- `libtisbackup/backup_xcp_metadata.py`
- `libtisbackup/backup_xva.py`
- `libtisbackup/copy_vm_xcp.py`
- `tisbackup.py`
- `tisbackup_gui.py`
## Verification
✅ **All checks passed**
- Ruff linting: `uv run ruff check .` - ✓ All checks passed
- CLI test: `uv run python tisbackup.py listdrivers` - ✓ 10 drivers loaded successfully
- Import test: `from libtisbackup import *` - ✓ All imports successful
## Metrics
| Metric | Before | After | Improvement |
|--------|--------|-------|-------------|
| Largest file | 1079 lines (common.py) | 579 lines (base_driver.py) | 46% reduction |
| Total lines | 1079 | 1079 (distributed) | Same functionality |
| Number of modules | 1 monolith | 6 focused modules | 6x organization |
| Average file size | 42KB | 8.2KB | 81% smaller |
## Future Enhancements
Now that the codebase is modular, future improvements are easier:
1. **Add type hints** to individual modules
2. **Write unit tests** for each module independently
3. **Add documentation** with module-level docstrings
4. **Create specialized utilities** without bloating a single file
5. **Optimize imports** by using specific imports instead of `import *`
## Notes
- The original `common.py` is preserved as `common.py.bak` for reference
- No functionality was removed or changed - purely structural refactoring
- All existing configuration files, backup scripts, and workflows continue to work unchanged

272
SECURITY_IMPROVEMENTS.md
View File

@ -1,272 +0,0 @@
# Security and Code Quality Improvements
This document summarizes the security and code quality improvements made to TISBackup.
## Completed Improvements (High Priority)
### 1. Replaced `os.popen()` with `subprocess.run()`
**Files Modified:** [tisbackup_gui.py](tisbackup_gui.py)
**Changes:**
- Replaced deprecated `os.popen()` calls with modern `subprocess.run()`
- All subprocess calls now use list arguments instead of shell strings
- Added timeout protection (5-30 seconds depending on operation)
- Proper error handling with try/except blocks
**Before:**
```python
for line in os.popen("udevadm info -q env -n %s" % name):
# Process output
```
**After:**
```python
result = subprocess.run(
["udevadm", "info", "-q", "env", "-n", name],
capture_output=True,
text=True,
check=True,
timeout=5
)
for line in result.stdout.splitlines():
# Process output
```
**Security Impact:** Prevents command injection vulnerabilities
### 2. Replaced `os.system()` with `subprocess.run()`
**Files Modified:** [tasks.py](tasks.py), [libtisbackup/backup_xva.py](libtisbackup/backup_xva.py)
**Changes:**
- [tasks.py:37](tasks.py#L37): Changed `os.system("/bin/umount %s")` to `subprocess.run(["/bin/umount", mount_point])`
- [backup_xva.py:199](libtisbackup/backup_xva.py#L199): Changed `os.system('tar tf "%s"')` to `subprocess.run(["tar", "tf", filename_temp])`
- Added proper error handling and logging
**Security Impact:** Eliminates command injection risk from potentially user-controlled mount points and filenames
### 3. Added Input Validation
**Files Modified:** [tisbackup_gui.py](tisbackup_gui.py)
**Changes:**
- Added regex validation for device/partition names: `^/dev/sd[a-z]1?$`
- Validates partition names before using in mount/unmount operations
- Prevents path traversal and command injection attacks
**Example:**
```python
# Validate partition name to prevent command injection
if not re.match(r"^/dev/sd[a-z]1$", partition):
continue
```
**Security Impact:** Prevents malicious input from reaching system commands
### 4. Fixed File Operations with Context Managers
**Files Modified:** [tisbackup_gui.py](tisbackup_gui.py)
**Before:**
```python
line = open(elem).readline()
```
**After:**
```python
with open(elem) as f:
line = f.readline()
```
**Impact:** Ensures files are properly closed, prevents resource leaks
### 5. Improved `run_command()` Function
**Files Modified:** [tisbackup_gui.py:415-453](tisbackup_gui.py#L415)
**Changes:**
- Now accepts list arguments for safe command execution
- Backward compatible with string commands (marked as legacy)
- Added timeout protection (30 seconds)
- Better error handling and reporting
**Security Impact:** Provides safe command execution interface while maintaining backward compatibility
### 6. Removed Wildcard Import
**Files Modified:** [tisbackup_gui.py](tisbackup_gui.py)
**Before:**
```python
from shutil import *
```
**After:**
```python
import shutil
import subprocess
```
**Impact:** Cleaner namespace, easier to track dependencies
### 7. Fixed Hardcoded Secret Key
**Files Modified:** [tisbackup_gui.py:67-79](tisbackup_gui.py#L67), [README.md](README.md)
**Before:**
```python
app.secret_key = "fsiqefiuqsefARZ4Zfesfe34234dfzefzfe"
```
**After:**
```python
SECRET_KEY = os.environ.get("TISBACKUP_SECRET_KEY")
if not SECRET_KEY:
import secrets
SECRET_KEY = secrets.token_hex(32)
logging.warning(
"TISBACKUP_SECRET_KEY environment variable not set. "
"Using a randomly generated secret key. "
"Sessions will not persist across application restarts. "
"Set TISBACKUP_SECRET_KEY environment variable for production use."
)
app.secret_key = SECRET_KEY
```
**Changes:**
- Reads secret key from `TISBACKUP_SECRET_KEY` environment variable
- Falls back to cryptographically secure random key if not set
- Logs warning when using random key (sessions won't persist across restarts)
- Uses Python's `secrets` module for cryptographically strong random generation
- Updated README.md with setup instructions
**Setup Instructions:**
```bash
# Generate a secure secret key
python3 -c "import secrets; print(secrets.token_hex(32))"
# Set in Docker Compose (compose.yml)
environment:
- TISBACKUP_SECRET_KEY=your-generated-key-here
# Or export in shell
export TISBACKUP_SECRET_KEY=your-generated-key-here
```
**Security Impact:** Eliminates hardcoded secret in source code, prevents session hijacking and CSRF attacks
### 8. Modernized SSH Key Algorithm Support
**Files Modified:** [libtisbackup/common.py](libtisbackup/common.py#L140), all backup drivers, [README.md](README.md)
**Before:**
```python
try:
mykey = paramiko.RSAKey.from_private_key_file(self.private_key)
except paramiko.SSHException:
mykey = paramiko.DSSKey.from_private_key_file(self.private_key)
```
**After:**
```python
def load_ssh_private_key(private_key_path):
"""Load SSH private key with modern algorithm support.
Tries to load the key in order of preference:
1. Ed25519 (most secure, modern)
2. ECDSA (secure, widely supported)
3. RSA (legacy, still secure with sufficient key size)
DSA is not supported as it's deprecated and insecure.
"""
key_types = [
("Ed25519", paramiko.Ed25519Key),
("ECDSA", paramiko.ECDSAKey),
("RSA", paramiko.RSAKey),
]
for key_name, key_class in key_types:
try:
return key_class.from_private_key_file(private_key_path)
except paramiko.SSHException:
continue
raise paramiko.SSHException(
f"Unable to load private key. "
f"Supported formats: Ed25519 (recommended), ECDSA, RSA. "
f"DSA keys are no longer supported."
)
```
**Changes:**
- Created centralized `load_ssh_private_key()` helper function
- Updated all SSH key loading locations across codebase:
- [common.py](libtisbackup/common.py): `do_preexec`, `do_postexec`, `run_remote_command`
- [backup_mysql.py](libtisbackup/backup_mysql.py)
- [backup_pgsql.py](libtisbackup/backup_pgsql.py)
- [backup_sqlserver.py](libtisbackup/backup_sqlserver.py)
- [backup_oracle.py](libtisbackup/backup_oracle.py)
- [backup_samba4.py](libtisbackup/backup_samba4.py)
- Removed deprecated DSA key support
- Added Ed25519 as preferred algorithm
- Added ECDSA as second choice
- RSA remains supported for compatibility
- Clear error message indicating DSA is no longer supported
- Updated README.md with key generation instructions
**SSH Key Generation:**
```bash
# Ed25519 (recommended)
ssh-keygen -t ed25519 -f ./ssh/id_ed25519 -C "tisbackup"
# ECDSA (also secure)
ssh-keygen -t ecdsa -b 521 -f ./ssh/id_ecdsa
# RSA (legacy, minimum 4096 bits)
ssh-keygen -t rsa -b 4096 -f ./ssh/id_rsa
```
**Security Impact:**
- Eliminates support for vulnerable DSA algorithm (1024-bit limit, FIPS deprecated)
- Prioritizes Ed25519 (fast, secure, resistant to timing attacks)
- Supports ECDSA as secure alternative
- Maintains RSA compatibility for legacy systems
- Clear migration path for users with old keys
## Remaining Security Issues (Critical - Not Fixed)
### 1. **No Authentication on Flask Routes**
All routes are publicly accessible without authentication.
**Recommendation:** Implement Flask-Login or similar authentication
### 2. **Insecure SSH Host Key Policy** ([libtisbackup/common.py:649](libtisbackup/common.py#L649))
```python
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
```
**Recommendation:** Use proper host key verification with known_hosts
### 3. **Command Injection in Legacy Code**
Multiple files still use `subprocess.call(shell_string, shell=True)` and `subprocess.Popen(..., shell=True)`:
- [libtisbackup/common.py:128](libtisbackup/common.py#L128)
- [libtisbackup/common.py:883](libtisbackup/common.py#L883)
- [libtisbackup/common.py:986](libtisbackup/common.py#L986)
- [libtisbackup/backup_rsync.py:176](libtisbackup/backup_rsync.py#L176)
- [libtisbackup/backup_rsync_btrfs.py](libtisbackup/backup_rsync_btrfs.py) (multiple locations)
**Recommendation:** Refactor to use list arguments without shell=True
## Code Quality Issues Remaining
1. **Global State Management** - Use Flask application context instead
2. **Wildcard imports from common** - `from libtisbackup.common import *`
3. **Configuration loaded at module level** - Should use application factory pattern
4. **Duplicated code** - `read_config()` and `read_all_configs()` share significant logic
## Testing Recommendations
Before deploying these changes:
1. Test USB disk detection and mounting functionality
2. Test backup export operations
3. Verify XVA backup tar validation
4. Test error handling for invalid device names
5. Verify backward compatibility with existing configurations
## Migration Notes
All changes are backward compatible. The `run_command()` function accepts both:
- New format: `run_command(["/bin/command", "arg1", "arg2"])`
- Legacy format: `run_command("/bin/command arg1 arg2")` (less secure, marked for deprecation)

372
docs-sphinx-rst/source/authentication.rst
View File

@ -1,372 +0,0 @@
.. Reminder for header structure:
Level 1: ====================
Level 2: --------------------
Level 3: ++++++++++++++++++++
Level 4: """"""""""""""""""""
Level 5: ^^^^^^^^^^^^^^^^^^^^
.. meta::
:description: Configuring authentication for TISBackup web interface
:keywords: Documentation, TISBackup, authentication, security, OAuth, Flask-Login
Authentication Configuration
============================
.. _authentication_configuration:
TISBackup provides a pluggable authentication system for the Flask web interface,
supporting multiple authentication methods to suit different deployment scenarios.
Overview
--------
The authentication system supports three authentication providers:
* **Basic Authentication** - Simple HTTP Basic Auth (default)
* **Flask-Login** - Session-based authentication with user management
* **OAuth2** - Integration with external identity providers
By default, TISBackup uses Basic Authentication. You can configure the authentication
method in the :file:`/etc/tis/tisbackup_gui.ini` configuration file.
Basic Authentication
--------------------
HTTP Basic Authentication is the simplest method and is enabled by default.
Configuration via Environment Variables
+++++++++++++++++++++++++++++++++++++++
Set the following environment variables:
.. code-block:: bash
export TISBACKUP_AUTH_USERNAME="admin"
export TISBACKUP_AUTH_PASSWORD="your-secure-password"
Configuration via INI File
++++++++++++++++++++++++++
Create or edit :file:`/etc/tis/tisbackup_gui.ini`:
.. code-block:: ini
[authentication]
type=basic
username=admin
password=your-password
use_bcrypt=False
realm=TISBackup
Using Bcrypt Password Hashes (Recommended)
+++++++++++++++++++++++++++++++++++++++++++
For improved security, use bcrypt-hashed passwords:
1. Install bcrypt support:
.. code-block:: bash
uv pip install bcrypt
2. Generate a password hash:
.. code-block:: python
import bcrypt
password = b"your-password"
hash = bcrypt.hashpw(password, bcrypt.gensalt())
print(hash.decode())
3. Update configuration:
.. code-block:: ini
[authentication]
type=basic
username=admin
password_hash=$2b$12$LQv3c1yqBWVHxkd0LHAkCOYz6TtxMQJqhN8/LewY5eSZL9fJQp.Ym
use_bcrypt=True
realm=TISBackup
Flask-Login Authentication
---------------------------
Session-based authentication with user management and login pages.
Installation
++++++++++++
Install Flask-Login support:
.. code-block:: bash
uv pip install flask-login bcrypt
Configuration
+++++++++++++
Create :file:`/etc/tis/tisbackup_gui.ini`:
.. code-block:: ini
[authentication]
type=flask-login
user_file=/etc/tis/tisbackup_users.txt
secret_key=<generate-random-secret-key>
session_timeout=3600
Generate a secret key:
.. code-block:: bash
python3 -c "import secrets; print(secrets.token_hex(32))"
User File Format
++++++++++++++++
Create a user file at :file:`/etc/tis/tisbackup_users.txt`:
.. code-block:: text
admin:$2b$12$LQv3c1yqBWVHxkd0LHAkCOYz6TtxMQJqhN8/LewY5eSZL9fJQp.Ym
user1:$2b$12$KPOvd2wqZWVIxje1MIBlDPZy7UuyNRKriQ9/MfxZ6fTaM9gKRq.Wm
Each line is: ``username:bcrypt_password_hash``
Managing Users
++++++++++++++
Add a new user:
.. code-block:: python
import bcrypt
username = "newuser"
password = b"secure-password"
hash = bcrypt.hashpw(password, bcrypt.gensalt()).decode()
with open("/etc/tis/tisbackup_users.txt", "a") as f:
f.write(f"{username}:{hash}\n")
Ensure proper permissions:
.. code-block:: bash
chmod 600 /etc/tis/tisbackup_users.txt
chown root:root /etc/tis/tisbackup_users.txt
OAuth2 Authentication
---------------------
Integrate with external OAuth2 identity providers like Google, GitHub, or GitLab.
Installation
++++++++++++
Install OAuth support:
.. code-block:: bash
uv pip install authlib requests
Google OAuth
++++++++++++
1. Create OAuth credentials in Google Cloud Console
2. Configure TISBackup:
.. code-block:: ini
[authentication]
type=oauth
provider=google
client_id=<your-client-id>.apps.googleusercontent.com
client_secret=<your-client-secret>
redirect_uri=https://backup.example.com/callback
allowed_domains=example.com
GitHub OAuth
++++++++++++
1. Create OAuth App in GitHub Settings
2. Configure TISBackup:
.. code-block:: ini
[authentication]
type=oauth
provider=github
client_id=<your-client-id>
client_secret=<your-client-secret>
redirect_uri=https://backup.example.com/callback
allowed_users=user1,user2,user3
GitLab OAuth
++++++++++++
1. Create OAuth application in GitLab
2. Configure TISBackup:
.. code-block:: ini
[authentication]
type=oauth
provider=gitlab
client_id=<your-client-id>
client_secret=<your-client-secret>
redirect_uri=https://backup.example.com/callback
gitlab_url=https://gitlab.example.com
Generic OAuth Provider
++++++++++++++++++++++
For custom OAuth providers:
.. code-block:: ini
[authentication]
type=oauth
provider=generic
client_id=<your-client-id>
client_secret=<your-client-secret>
redirect_uri=https://backup.example.com/callback
authorize_url=https://provider.example.com/oauth/authorize
token_url=https://provider.example.com/oauth/token
userinfo_url=https://provider.example.com/oauth/userinfo
Advanced Configuration
----------------------
Multiple Authentication Methods
++++++++++++++++++++++++++++++++
You can only use one authentication method at a time. To switch methods,
update the ``type`` parameter in the configuration file and restart
the TISBackup GUI service.
Disabling Authentication (Not Recommended)
++++++++++++++++++++++++++++++++++++++++++
.. warning::
Disabling authentication is **not recommended** for production environments.
Only use this for testing or when the web interface is protected by other means
(e.g., VPN, firewall rules).
To disable authentication:
.. code-block:: ini
[authentication]
type=none
Custom Realm
++++++++++++
For Basic Authentication, customize the authentication realm:
.. code-block:: ini
[authentication]
type=basic
realm=My Company Backup System
Session Timeout
+++++++++++++++
For Flask-Login and OAuth, configure session timeout (in seconds):
.. code-block:: ini
[authentication]
type=flask-login
session_timeout=7200 # 2 hours
Troubleshooting
---------------
Authentication Not Working
++++++++++++++++++++++++++
Check the logs for authentication errors:
.. code-block:: bash
journalctl -u tisbackup_gui -n 100
Verify configuration file syntax:
.. code-block:: bash
python3 -c "from configparser import ConfigParser; cp = ConfigParser(); cp.read('/etc/tis/tisbackup_gui.ini'); print('OK')"
Random Password Generated
++++++++++++++++++++++++++
If you see a warning about a generated password in the logs:
.. code-block:: text
WARNING: Generated temporary password for 'admin': abc123xyz
This means no password was configured. Set ``TISBACKUP_AUTH_PASSWORD`` environment
variable or add an ``[authentication]`` section to the configuration file.
OAuth Callback Error
++++++++++++++++++++
Ensure the redirect URI in your OAuth provider configuration **exactly matches**
the ``redirect_uri`` parameter in the TISBackup configuration.
The redirect URI should be: ``https://your-domain.com/callback``
User File Not Found
+++++++++++++++++++
For Flask-Login authentication, ensure the user file exists and has proper permissions:
.. code-block:: bash
ls -l /etc/tis/tisbackup_users.txt
# Should show: -rw------- 1 root root ...
Security Recommendations
------------------------
1. **Use HTTPS**: Always use HTTPS in production (configure via reverse proxy)
2. **Strong Passwords**: Use long, random passwords or password hashes
3. **Restrict Access**: Use firewall rules to limit access to trusted networks
4. **Regular Updates**: Keep authentication dependencies updated
5. **Monitor Logs**: Regularly check logs for failed authentication attempts
6. **Session Security**: Use short session timeouts for sensitive environments
For more security best practices, see the **Security Best Practices** section of the documentation.
Migration Guide
---------------
From No Authentication
++++++++++++++++++++++
If upgrading from a version without authentication:
1. Add authentication configuration as described above
2. Restart the TISBackup GUI service
3. Update any automated tools to include authentication credentials
From Basic to OAuth
+++++++++++++++++++
1. Set up OAuth provider configuration
2. Update ``type=oauth`` in configuration file
3. Install required dependencies: ``uv pip install authlib requests``
4. Restart the service
5. Test login with OAuth provider
Additional Resources
--------------------
For comprehensive authentication setup examples and troubleshooting,
see the :file:`AUTHENTICATION.md` file in the TISBackup repository root.

24
docs-sphinx-rst/source/conf.py
View File

@ -35,7 +35,6 @@ extensions = [
"sphinx.ext.todo",
"sphinx.ext.viewcode",
"sphinx.ext.githubpages",
"sphinx_tabs.tabs",
]
# Add any paths that contain templates here, relative to this directory.
@ -125,9 +124,22 @@ todo_include_todos = True
# -- Options for HTML output ----------------------------------------------
html_theme = "alabaster"
html_theme_path = []
html_favicon = "_static/favicon.ico"
try:
import sphinx_rtd_theme
html_theme = "sphinx_rtd_theme"
html_favicon = "_static/favicon.ico"
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
html_context = {
"css_files": [
"_static/css/custom.css", # overrides for wide tables in RTD theme
"_static/css/ribbon.css",
"_static/theme_overrides.css", # override wide tables in RTD theme
],
}
except ImportError as e: # noqa : F841
html_theme = "alabaster"
html_theme_path = []
# The theme to use for HTML and HTML Help pages. See the documentation for
@ -369,9 +381,7 @@ texinfo_documents = [
# Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = {
"python": ("https://docs.python.org/3", None),
}
intersphinx_mapping = {"https://docs.python.org/": None}
# -- Options for Epub output ----------------------------------------------

4
docs-sphinx-rst/source/configuring_tisbackup.rst
View File

@ -82,7 +82,7 @@ Backing up a MySQL database
[srvintranet_mysql_mediawiki]
type=mysql+ssh
server_name=srvintranet
private_key=/root/.ssh/id_ed25519
private_key=/root/.ssh/id_dsa
db_name=mediawiki
db_user=user
db_passwd=password
@ -141,7 +141,7 @@ Backing up a file server
type=rsync+ssh
server_name=srvfiles
remote_dir=/home
private_key=/root/.ssh/id_ed25519
private_key=/root/.ssh/id_dsa
exclude_list=".mozilla",".thunderbird",".x2go","*.avi"
bwlimit = 100

7
docs-sphinx-rst/source/index.rst
View File

@ -92,13 +92,6 @@ would have been difficult to develop as an overlay of the existing one:
configuring_tisbackup.rst
using_tisbackup.rst
.. toctree::
:maxdepth: 2
:caption: Security & Authentication
security.rst
authentication.rst
.. toctree::
:maxdepth: 1
:caption: Appendix

14
docs-sphinx-rst/source/installing_tisbackup.rst
View File

@ -251,24 +251,14 @@ Launching the backup scheduled task
Generating the public and private certificates
++++++++++++++++++++++++++++++++++++++++++++++
* as root, generate an Ed25519 SSH key (modern and secure algorithm):
* as root:
.. code-block:: bash
ssh-keygen -t ed25519 -C "tisbackup@$(hostname)"
ssh-keygen -t rsa -b 2048
* press :kbd:`Enter` for each one of the steps;
.. note::
TISBackup supports Ed25519, ECDSA, and RSA key algorithms (in order of preference).
DSA keys are no longer supported for security reasons. If you need RSA for compatibility,
use at least 4096 bits:
.. code-block:: bash
ssh-keygen -t rsa -b 4096 -C "tisbackup@$(hostname)"
|clap| You may now go on to the next step
and :ref:`configure the backup jobs for your TISBackup<configuring_backup_jobs>`.

288
docs-sphinx-rst/source/security.rst
View File

@ -1,288 +0,0 @@
.. Reminder for header structure:
Level 1: ====================
Level 2: --------------------
Level 3: ++++++++++++++++++++
Level 4: """"""""""""""""""""
Level 5: ^^^^^^^^^^^^^^^^^^^^
.. meta::
:description: Security best practices for TISBackup
:keywords: Documentation, TISBackup, security, best practices, authentication
Security Best Practices
=======================
.. _security_best_practices:
TISBackup has been designed with security in mind. This section outlines
the security features and best practices for deploying and maintaining
a secure backup infrastructure.
SSH Key Algorithm Support
--------------------------
Modern SSH Key Algorithms
+++++++++++++++++++++++++
TISBackup supports modern SSH key algorithms with the following priority:
1. **Ed25519** (recommended) - Modern, fast, and secure
2. **ECDSA** - Elliptic curve cryptography
3. **RSA** - Traditional algorithm (use 4096 bits minimum)
.. warning::
DSA keys are **no longer supported** due to known security vulnerabilities.
If you are using DSA keys, you must migrate to Ed25519, ECDSA, or RSA.
Generating Secure SSH Keys
+++++++++++++++++++++++++++
For new installations, generate an Ed25519 key:
.. code-block:: bash
ssh-keygen -t ed25519 -C "tisbackup@$(hostname)"
For compatibility with older systems that don't support Ed25519, use RSA with 4096 bits:
.. code-block:: bash
ssh-keygen -t rsa -b 4096 -C "tisbackup@$(hostname)"
Migrating from DSA Keys
++++++++++++++++++++++++
If you have existing backup configurations using DSA keys:
1. Generate a new Ed25519 key on the backup server
2. Copy the new public key to all backup clients
3. Update the ``private_key`` parameter in all backup sections
4. Test the backups to ensure they work with the new key
5. Remove the old DSA keys from both server and clients
Flask Web Interface Security
-----------------------------
Authentication
++++++++++++++
The Flask web interface now requires authentication by default.
TISBackup supports multiple authentication methods:
Basic Authentication (Default)
"""""""""""""""""""""""""""""""
By default, TISBackup uses HTTP Basic Authentication. Configure it via
environment variables or the configuration file.
**Environment variables:**
.. code-block:: bash
export TISBACKUP_AUTH_USERNAME="admin"
export TISBACKUP_AUTH_PASSWORD="your-secure-password"
**Configuration file** (:file:`/etc/tis/tisbackup_gui.ini`):
.. code-block:: ini
[authentication]
type=basic
username=admin
# Bcrypt hash of password (recommended)
password_hash=$2b$12$LQv3c1yqBWVHxkd0LHAkCOYz6TtxMQJqhN8/LewY5eSZL9fJQp.Ym
use_bcrypt=True
realm=TISBackup
.. warning::
If no password is configured, TISBackup will generate a random password
and display it in the logs. This is not suitable for production use.
Session-Based Authentication (Flask-Login)
"""""""""""""""""""""""""""""""""""""""""""
For more advanced deployments, you can use Flask-Login with a user file:
.. code-block:: ini
[authentication]
type=flask-login
user_file=/etc/tis/tisbackup_users.txt
secret_key=<random-secret-key>
OAuth2 Authentication
""""""""""""""""""""""
For enterprise deployments, OAuth2 is supported with providers like Google,
GitHub, and GitLab:
.. code-block:: ini
[authentication]
type=oauth
provider=google
client_id=<your-client-id>
client_secret=<your-client-secret>
redirect_uri=http://backup.example.com:8080/callback
allowed_domains=example.com
See :file:`AUTHENTICATION.md` in the repository root for detailed
authentication configuration.
Secret Key Configuration
+++++++++++++++++++++++++
The Flask application requires a secret key for session security.
**Never use the default hardcoded key in production!**
Configure via environment variable:
.. code-block:: bash
export TISBACKUP_SECRET_KEY="your-random-secret-key-here"
Or in :file:`/etc/tis/tisbackup_gui.ini`:
.. code-block:: ini
[global]
secret_key=your-random-secret-key-here
Generate a secure random key:
.. code-block:: bash
python3 -c "import secrets; print(secrets.token_hex(32))"
SSL/TLS Configuration
+++++++++++++++++++++
For production deployments, always use HTTPS. Place the Flask application
behind a reverse proxy like Nginx or Apache:
**Nginx example:**
.. code-block:: nginx
server {
listen 443 ssl http2;
server_name backup.example.com;
ssl_certificate /etc/ssl/certs/backup.crt;
ssl_certificate_key /etc/ssl/private/backup.key;
location / {
proxy_pass http://127.0.0.1:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
Database and Backup Security
-----------------------------
File Permissions
++++++++++++++++
Ensure proper file permissions on sensitive files:
.. code-block:: bash
# Configuration files
chmod 600 /etc/tis/tisbackup-config.ini
chmod 600 /etc/tis/tisbackup_gui.ini
# SSH keys
chmod 600 /root/.ssh/id_ed25519
chmod 644 /root/.ssh/id_ed25519.pub
# Password files (for XenServer, etc.)
chmod 600 /root/xen_passwd
# Backup directory
chown -R root:root /backup/data
chmod 750 /backup/data
Credential Storage
++++++++++++++++++
For database credentials and other secrets:
* Use strong, unique passwords for each service
* Store credentials in configuration files with restricted permissions
* Consider using a secrets management system for sensitive deployments
* Rotate credentials regularly
Network Security
++++++++++++++++
* Restrict SSH access to the backup server IP address
* Use firewall rules to limit access to the web interface
* Consider VPN access for remote backup management
* Enable fail2ban or similar tools to prevent brute-force attacks
Security Monitoring
-------------------
Log Monitoring
++++++++++++++
Regularly review TISBackup logs for:
* Failed authentication attempts
* Backup failures or timeouts
* Unusual activity patterns
* SSH connection errors
.. code-block:: bash
# View recent backup logs
journalctl -u tisbackup_gui -n 100
# Monitor for authentication failures
grep "authentication failed" /var/log/tisbackup/*.log
Backup Verification
+++++++++++++++++++
* Regularly test backup restoration
* Verify backup integrity using checksums
* Monitor backup sizes for unexpected changes
* Set up Nagios checks for backup freshness
Security Updates
++++++++++++++++
* Keep TISBackup updated to the latest version
* Apply security patches to the host operating system
* Update Python dependencies regularly:
.. code-block:: bash
uv sync --upgrade
Additional Security Recommendations
------------------------------------
1. **Principle of Least Privilege**: Create dedicated service accounts
for backups rather than using root when possible
2. **Network Segmentation**: Place the backup server in a dedicated
network segment with restricted access
3. **Backup Encryption**: Consider encrypting backups at rest,
especially for sensitive data
4. **Off-site Storage**: Maintain encrypted off-site backups
for disaster recovery
5. **Access Auditing**: Maintain logs of who accesses backups
and when they are restored
6. **Incident Response**: Have a documented procedure for responding
to security incidents involving the backup infrastructure

0
libtisbackup/drivers/XenAPI.py → libtisbackup/XenAPI.py
View File

74
libtisbackup/__init__.py
View File

@ -15,77 +15,3 @@
# along with TISBackup. If not, see <http://www.gnu.org/licenses/>.
#
# -----------------------------------------------------------------------
"""
TISBackup library - Backup orchestration and driver management.
This package provides a modular backup system with:
- Base driver classes for implementing backup types
- Database management for backup statistics
- SSH and process execution utilities
- Date/time and formatting helpers
"""
# Import from new modular structure
from .base_driver import (
backup_drivers,
backup_generic,
nagiosStateCritical,
nagiosStateOk,
nagiosStateUnknown,
nagiosStateWarning,
register_driver,
)
from .database import BackupStat
from .process import call_external_process, monitor_stdout
from .ssh import load_ssh_private_key, ssh_exec
from .utils import (
check_string,
convert_bytes,
dateof,
datetime2isodate,
fileisodate,
hours_minutes,
html_table,
isodate2datetime,
pp,
splitThousands,
str2bool,
time2display,
)
# Maintain backward compatibility - re-export everything that was in common.py
__all__ = [
# Nagios states
"nagiosStateOk",
"nagiosStateWarning",
"nagiosStateCritical",
"nagiosStateUnknown",
# Driver registry
"backup_drivers",
"register_driver",
# Base classes
"backup_generic",
"BackupStat",
# SSH utilities
"load_ssh_private_key",
"ssh_exec",
# Process utilities
"call_external_process",
"monitor_stdout",
# Date/time utilities
"datetime2isodate",
"isodate2datetime",
"time2display",
"hours_minutes",
"fileisodate",
"dateof",
# Formatting utilities
"splitThousands",
"convert_bytes",
"pp",
"html_table",
# Validation utilities
"check_string",
"str2bool",
]

308
libtisbackup/auth/README.md
View File

@ -1,308 +0,0 @@
# TISBackup Authentication Module
Pluggable authentication system for Flask routes.
## Features
- **Multiple providers**: Basic Auth, Flask-Login, OAuth2
- **Easy integration**: Simple decorator-based protection
- **Configurable**: INI-based configuration
- **Secure**: bcrypt password hashing, OAuth integration
- **Extensible**: Easy to add new providers
## Quick Start
### 1. Choose Authentication Provider
```python
from libtisbackup.auth import get_auth_provider
# Get provider from config
auth = get_auth_provider("basic", {
"username": "admin",
"password": "$2b$12$...", # bcrypt hash
"use_bcrypt": True
})
# Initialize with Flask app
auth.init_app(app)
```
### 2. Protect Routes
```python
@app.route("/")
@auth.require_auth
def index():
user = auth.get_current_user()
return f"Hello {user['username']}"
```
## Providers
### Base Provider (No Auth)
```python
auth = get_auth_provider("none", {})
```
- No authentication required
- All routes publicly accessible
- Useful for development/testing
### Basic Auth
```python
auth = get_auth_provider("basic", {
"username": "admin",
"password": "$2b$12$LQv3c1yqBWVHxkd0LHAkCOYz6TtxMQJqhN8/LewY5GyYWv.5qVQK6",
"use_bcrypt": True,
"realm": "TISBackup"
})
```
**Features:**
- HTTP Basic Authentication
- bcrypt password hashing
- Custom realm support
### Flask-Login
```python
auth = get_auth_provider("flask-login", {
"users_file": "/etc/tis/users.txt",
"use_bcrypt": True,
"login_view": "login"
})
```
**Features:**
- Session-based authentication
- Multiple users support
- Login/logout pages
- bcrypt password hashing
**User file format** (`users.txt`):
```
username:bcrypt_hash
admin:$2b$12$LQv3c1yqBWVHxkd0LHAkCOYz6TtxMQJqhN8/LewY5GyYWv.5qVQK6
```
### OAuth2
```python
auth = get_auth_provider("oauth", {
"provider": "google", # or "github", "gitlab", "generic"
"client_id": "your-client-id",
"client_secret": "your-client-secret",
"redirect_uri": "http://localhost:8080/oauth/callback",
"authorized_domains": ["example.com"],
"authorized_users": ["admin@example.com"]
})
```
**Features:**
- OAuth2 authentication
- Google, GitHub, GitLab support
- Custom OAuth providers
- Domain/user restrictions
## API Reference
### AuthProvider
Base class for all auth providers.
#### Methods
- `init_app(app)` - Initialize with Flask app
- `require_auth(f)` - Decorator to protect routes
- `is_authenticated()` - Check if current request is authenticated
- `get_current_user()` - Get current user info
- `handle_unauthorized()` - Handle unauthorized access
- `logout()` - Logout current user
### BasicAuthProvider
HTTP Basic Authentication provider.
#### Configuration
```python
{
"username": str, # Required
"password": str, # Required (plain or bcrypt hash)
"use_bcrypt": bool, # Default: False
"realm": str # Default: "TISBackup"
}
```
### FlaskLoginProvider
Session-based authentication provider.
#### Configuration
```python
{
"users": dict, # {username: password_hash} or...
"users_file": str, # Path to users file
"use_bcrypt": bool, # Default: True
"login_view": str # Default: "login"
}
```
#### Methods
- `verify_password(username, password)` - Verify credentials
- `login_user(username)` - Login user by username
### OAuthProvider
OAuth2 authentication provider.
#### Configuration
```python
{
"provider": str, # "google", "github", "gitlab", "generic"
"client_id": str, # Required
"client_secret": str, # Required
"redirect_uri": str, # Required
"scopes": list, # Optional
"authorized_domains": list, # Optional
"authorized_users": list, # Optional
# For generic provider:
"authorization_endpoint": str,
"token_endpoint": str,
"userinfo_endpoint": str
}
```
#### Methods
- `is_user_authorized(user_info)` - Check if user is authorized
## Integration Example
See [example_integration.py](example_integration.py) for a complete example.
### Minimal Example
```python
from flask import Flask
from libtisbackup.auth import get_auth_provider
app = Flask(__name__)
app.secret_key = "your-secret-key"
# Initialize auth
auth = get_auth_provider("basic", {
"username": "admin",
"password": "changeme",
"use_bcrypt": False
})
auth.init_app(app)
# Protected route
@app.route("/")
@auth.require_auth
def index():
return "Protected content"
# Public route
@app.route("/health")
def health():
return "OK"
```
### With Flask-Login
```python
auth = get_auth_provider("flask-login", {
"users": {
"admin": "$2b$12$..."
},
"use_bcrypt": True
})
auth.init_app(app)
@app.route("/login", methods=["POST"])
def login():
username = request.form["username"]
password = request.form["password"]
if auth.verify_password(username, password):
auth.login_user(username)
return redirect("/")
return "Invalid credentials", 401
@app.route("/")
@auth.require_auth
def index():
user = auth.get_current_user()
return f"Hello {user['username']}"
```
### With OAuth
```python
auth = get_auth_provider("oauth", {
"provider": "google",
"client_id": "...",
"client_secret": "...",
"redirect_uri": "http://localhost:8080/oauth/callback",
"authorized_domains": ["example.com"]
})
auth.init_app(app)
@app.route("/oauth/login")
def oauth_login():
return auth.oauth_client.authorize_redirect(
url_for("oauth_callback", _external=True)
)
@app.route("/oauth/callback")
def oauth_callback():
token = auth.oauth_client.authorize_access_token()
user_info = auth.oauth_client.get(auth.userinfo_endpoint).json()
if auth.is_user_authorized(user_info):
session["oauth_user"] = user_info
return redirect("/")
return "Unauthorized", 403
```
## Security Considerations
1. **Always use HTTPS in production** - Especially for Basic Auth
2. **Use bcrypt for passwords** - Never store plain text passwords
3. **Rotate credentials regularly** - Change passwords and OAuth secrets
4. **Restrict OAuth access** - Use `authorized_domains` or `authorized_users`
5. **Set strong Flask secret_key** - Use `secrets.token_hex(32)`
6. **Protect config files** - `chmod 600` for files with credentials
7. **Use environment variables** - For sensitive configuration values
## Testing
```python
import unittest
from libtisbackup.auth import get_auth_provider
class TestBasicAuth(unittest.TestCase):
def test_authentication(self):
auth = get_auth_provider("basic", {
"username": "admin",
"password": "test",
"use_bcrypt": False
})
# Mock request with credentials
# Test authentication logic
pass
```
## License
GPL v3.0 - Same as TISBackup

52
libtisbackup/auth/__init__.py
View File

@ -1,52 +0,0 @@
#!/usr/bin/python3
# -*- coding: utf-8 -*-
"""
TISBackup Authentication Module
Provides pluggable authentication providers for Flask routes.
Supports: Basic Auth, Flask-Login, OAuth2
"""
from .base import AuthProvider
from .basic_auth import BasicAuthProvider
from .flask_login_auth import FlaskLoginProvider
from .oauth_auth import OAuthProvider
__all__ = [
"AuthProvider",
"BasicAuthProvider",
"FlaskLoginProvider",
"OAuthProvider",
"get_auth_provider",
]
def get_auth_provider(auth_type, config=None):
"""Factory function to get authentication provider.
Args:
auth_type: Type of auth ('basic', 'flask-login', 'oauth', or 'none')
config: Configuration dict for the provider
Returns:
AuthProvider instance
Raises:
ValueError: If auth_type is not supported
"""
providers = {
"none": AuthProvider,
"basic": BasicAuthProvider,
"flask-login": FlaskLoginProvider,
"oauth": OAuthProvider,
}
auth_type = auth_type.lower()
if auth_type not in providers:
raise ValueError(
f"Unsupported auth type: {auth_type}. "
f"Supported types: {', '.join(providers.keys())}"
)
provider_class = providers[auth_type]
return provider_class(config or {})

74
libtisbackup/auth/base.py
View File

@ -1,74 +0,0 @@
#!/usr/bin/python3
# -*- coding: utf-8 -*-
"""
Base authentication provider interface
"""
from abc import ABC, abstractmethod
from functools import wraps
class AuthProvider(ABC):
"""Base class for authentication providers."""
def __init__(self, config):
"""Initialize the auth provider.
Args:
config: Dict with provider-specific configuration
"""
self.config = config
def init_app(self, app):
"""Initialize the provider with Flask app.
Args:
app: Flask application instance
"""
pass
def require_auth(self, f):
"""Decorator to require authentication for a route.
Args:
f: Flask route function
Returns:
Decorated function
"""
@wraps(f)
def decorated_function(*args, **kwargs):
if not self.is_authenticated():
return self.handle_unauthorized()
return f(*args, **kwargs)
return decorated_function
def is_authenticated(self):
"""Check if current request is authenticated.
Returns:
bool: True if authenticated, False otherwise
"""
# Default: no authentication required
return True
def handle_unauthorized(self):
"""Handle unauthorized access.
Returns:
Flask response for unauthorized access
"""
from flask import jsonify
return jsonify({"error": "Unauthorized"}), 401
def get_current_user(self):
"""Get current authenticated user.
Returns:
User object or dict, or None if not authenticated
"""
return None
def logout(self):
"""Logout current user."""
pass

78
libtisbackup/auth/basic_auth.py
View File

@ -1,78 +0,0 @@
#!/usr/bin/python3
# -*- coding: utf-8 -*-
"""
HTTP Basic Authentication provider
"""
import base64
import logging
from functools import wraps
from flask import request
from .base import AuthProvider
class BasicAuthProvider(AuthProvider):
"""HTTP Basic Authentication provider.
Configuration:
username: Required username
password: Required password (plain text, or hashed with bcrypt)
realm: Authentication realm (default: 'TISBackup')
use_bcrypt: If True, password is bcrypt hash (default: False)
"""
def __init__(self, config):
super().__init__(config)
self.logger = logging.getLogger("tisbackup.auth")
self.username = config.get("username")
self.password = config.get("password")
self.realm = config.get("realm", "TISBackup")
self.use_bcrypt = config.get("use_bcrypt", False)
if not self.username or not self.password:
raise ValueError("BasicAuth requires 'username' and 'password' in config")
if self.use_bcrypt:
try:
import bcrypt
self.bcrypt = bcrypt
except ImportError:
raise ImportError("bcrypt library required for password hashing. Install with: pip install bcrypt")
def is_authenticated(self):
"""Check if request has valid Basic Auth credentials."""
auth = request.authorization
if not auth:
return False
if auth.username != self.username:
self.logger.warning(f"Failed authentication attempt for user: {auth.username}")
return False
if self.use_bcrypt:
# Compare bcrypt hash
password_bytes = auth.password.encode('utf-8')
hash_bytes = self.password.encode('utf-8') if isinstance(self.password, str) else self.password
return self.bcrypt.checkpw(password_bytes, hash_bytes)
else:
# Plain text comparison (not recommended for production)
return auth.password == self.password
def handle_unauthorized(self):
"""Return 401 with WWW-Authenticate header."""
from flask import Response
return Response(
'Authentication required',
401,
{'WWW-Authenticate': f'Basic realm="{self.realm}"'}
)
def get_current_user(self):
"""Get current authenticated user."""
auth = request.authorization
if auth and self.is_authenticated():
return {"username": auth.username, "auth_type": "basic"}
return None

121
libtisbackup/auth/example_integration.py
View File

@ -1,121 +0,0 @@
#!/usr/bin/python3
# -*- coding: utf-8 -*-
"""
Example integration of authentication providers with Flask app
This file shows how to integrate the authentication system into tisbackup_gui.py
"""
from flask import Flask, jsonify, render_template, request, redirect, url_for, session
from libtisbackup.auth import get_auth_provider
# Example configuration from tisbackup_gui.ini
auth_config = {
"type": "basic", # or "flask-login", "oauth", "none"
"basic": {
"username": "admin",
"password": "$2b$12$...", # bcrypt hash
"use_bcrypt": True,
"realm": "TISBackup Admin"
},
"flask-login": {
"users_file": "/etc/tis/users.txt", # username:bcrypt_hash per line
"use_bcrypt": True,
"login_view": "login"
},
"oauth": {
"provider": "google",
"client_id": "your-client-id",
"client_secret": "your-client-secret",
"redirect_uri": "http://localhost:8080/oauth/callback",
"authorized_domains": ["example.com"],
"authorized_users": ["admin@example.com"]
}
}
def create_app_with_auth():
"""Example: Create Flask app with authentication."""
app = Flask(__name__)
app.secret_key = "your-secret-key"
# Initialize authentication provider
auth_type = auth_config.get("type", "none")
provider_config = auth_config.get(auth_type, {})
auth = get_auth_provider(auth_type, provider_config)
auth.init_app(app)
# Protected routes
@app.route("/")
@auth.require_auth
def index():
user = auth.get_current_user()
return render_template("index.html", user=user)
@app.route("/api/backups")
@auth.require_auth
def api_backups():
return jsonify({"backups": []})
# Public routes (no auth required)
@app.route("/health")
def health():
return jsonify({"status": "ok"})
# Flask-Login specific routes
if auth_type == "flask-login":
@app.route("/login", methods=["GET", "POST"])
def login():
if request.method == "POST":
username = request.form.get("username")
password = request.form.get("password")
if auth.verify_password(username, password):
auth.login_user(username)
return redirect(url_for("index"))
else:
return render_template("login.html", error="Invalid credentials")
return render_template("login.html")
@app.route("/logout")
def logout():
auth.logout()
return redirect(url_for("login"))
# OAuth specific routes
if auth_type == "oauth":
@app.route("/oauth/login")
def oauth_login():
redirect_uri = url_for("oauth_callback", _external=True)
return auth.oauth_client.authorize_redirect(redirect_uri)
@app.route("/oauth/callback")
def oauth_callback():
try:
token = auth.oauth_client.authorize_access_token()
user_info = auth.oauth_client.get(auth.userinfo_endpoint).json()
# Check authorization
if not auth.is_user_authorized(user_info):
return "Unauthorized: Your email/domain is not authorized", 403
# Store user in session
session["oauth_user"] = user_info
session["oauth_token"] = token
return redirect(url_for("index"))
except Exception as e:
return f"OAuth callback error: {e}", 500
@app.route("/logout")
def logout():
auth.logout()
return redirect(url_for("oauth_login"))
return app
if __name__ == "__main__":
app = create_app_with_auth()
app.run(debug=True, port=8080)

156
libtisbackup/auth/flask_login_auth.py
View File

@ -1,156 +0,0 @@
#!/usr/bin/python3
# -*- coding: utf-8 -*-
"""
Flask-Login authentication provider
"""
import logging
from functools import wraps
from flask import redirect, request, session, url_for
from .base import AuthProvider
class FlaskLoginProvider(AuthProvider):
"""Flask-Login based authentication provider.
Configuration:
users: Dict of {username: password_hash} or path to user file
secret_key: Flask secret key for sessions
login_view: Route name for login page (default: 'login')
use_bcrypt: If True, passwords are bcrypt hashes (default: True)
"""
def __init__(self, config):
super().__init__(config)
self.logger = logging.getLogger("tisbackup.auth")
self.login_manager = None
self.users = config.get("users", {})
self.login_view = config.get("login_view", "login")
self.use_bcrypt = config.get("use_bcrypt", True)
if self.use_bcrypt:
try:
import bcrypt
self.bcrypt = bcrypt
except ImportError:
raise ImportError("bcrypt library required. Install with: pip install bcrypt")
# Load users from file if path provided
users_file = config.get("users_file")
if users_file:
self._load_users_from_file(users_file)
def _load_users_from_file(self, filepath):
"""Load users from file (username:password_hash per line)."""
try:
with open(filepath) as f:
for line in f:
line = line.strip()
if line and not line.startswith('#'):
username, password_hash = line.split(':', 1)
self.users[username.strip()] = password_hash.strip()
except Exception as e:
self.logger.error(f"Failed to load users from {filepath}: {e}")
raise
def init_app(self, app):
"""Initialize Flask-Login with the app."""
try:
from flask_login import LoginManager, UserMixin
except ImportError:
raise ImportError("flask-login library required. Install with: pip install flask-login")
self.login_manager = LoginManager()
self.login_manager.init_app(app)
self.login_manager.login_view = self.login_view
# Simple User class
class User(UserMixin):
def __init__(self, username):
self.id = username
self.username = username
self.User = User
@self.login_manager.user_loader
def load_user(user_id):
if user_id in self.users:
return User(user_id)
return None
def verify_password(self, username, password):
"""Verify username and password.
Args:
username: Username to check
password: Password to verify
Returns:
bool: True if valid, False otherwise
"""
if username not in self.users:
return False
stored_hash = self.users[username]
if self.use_bcrypt:
password_bytes = password.encode('utf-8')
hash_bytes = stored_hash.encode('utf-8') if isinstance(stored_hash, str) else stored_hash
return self.bcrypt.checkpw(password_bytes, hash_bytes)
else:
return password == stored_hash
def login_user(self, username):
"""Login a user by username.
Args:
username: Username to login
Returns:
bool: True if successful
"""
try:
from flask_login import login_user
except ImportError:
return False
if username in self.users:
user = self.User(username)
login_user(user)
return True
return False
def is_authenticated(self):
"""Check if current user is authenticated."""
try:
from flask_login import current_user
return current_user.is_authenticated
except ImportError:
return False
def handle_unauthorized(self):
"""Redirect to login page."""
return redirect(url_for(self.login_view))
def get_current_user(self):
"""Get current authenticated user."""
try:
from flask_login import current_user
if current_user.is_authenticated:
return {
"username": current_user.username,
"auth_type": "flask-login"
}
except ImportError:
pass
return None
def logout(self):
"""Logout current user."""
try:
from flask_login import logout_user
logout_user()
except ImportError:
pass

164
libtisbackup/auth/oauth_auth.py
View File

@ -1,164 +0,0 @@
#!/usr/bin/python3
# -*- coding: utf-8 -*-
"""
OAuth2 authentication provider
"""
import logging
from functools import wraps
from flask import redirect, request, session, url_for
from .base import AuthProvider
class OAuthProvider(AuthProvider):
"""OAuth2 authentication provider.
Supports multiple OAuth providers (Google, GitHub, GitLab, etc.)
Configuration:
provider: OAuth provider name ('google', 'github', 'gitlab', 'generic')
client_id: OAuth client ID
client_secret: OAuth client secret
redirect_uri: OAuth redirect URI
scopes: List of OAuth scopes (default: ['openid', 'email', 'profile'])
authorized_domains: List of allowed email domains (optional)
authorized_users: List of allowed email addresses (optional)
login_view: Route name for login page (default: 'oauth_login')
"""
def __init__(self, config):
super().__init__(config)
self.logger = logging.getLogger("tisbackup.auth")
self.provider_name = config.get("provider", "generic").lower()
self.client_id = config.get("client_id")
self.client_secret = config.get("client_secret")
self.redirect_uri = config.get("redirect_uri")
self.scopes = config.get("scopes", ["openid", "email", "profile"])
self.authorized_domains = config.get("authorized_domains", [])
self.authorized_users = config.get("authorized_users", [])
self.login_view = config.get("login_view", "oauth_login")
if not self.client_id or not self.client_secret:
raise ValueError("OAuth requires 'client_id' and 'client_secret' in config")
# Provider-specific configurations
self.provider_configs = {
"google": {
"authorization_endpoint": "https://accounts.google.com/o/oauth2/v2/auth",
"token_endpoint": "https://oauth2.googleapis.com/token",
"userinfo_endpoint": "https://www.googleapis.com/oauth2/v2/userinfo",
"default_scopes": ["openid", "email", "profile"],
},
"github": {
"authorization_endpoint": "https://github.com/login/oauth/authorize",
"token_endpoint": "https://github.com/login/oauth/access_token",
"userinfo_endpoint": "https://api.github.com/user",
"default_scopes": ["read:user", "user:email"],
},
"gitlab": {
"authorization_endpoint": "https://gitlab.com/oauth/authorize",
"token_endpoint": "https://gitlab.com/oauth/token",
"userinfo_endpoint": "https://gitlab.com/api/v4/user",
"default_scopes": ["read_user", "email"],
},
}
# Get provider config or use generic
self.provider_config = self.provider_configs.get(self.provider_name, {})
# Override with custom endpoints if provided
self.authorization_endpoint = config.get(
"authorization_endpoint",
self.provider_config.get("authorization_endpoint")
)
self.token_endpoint = config.get(
"token_endpoint",
self.provider_config.get("token_endpoint")
)
self.userinfo_endpoint = config.get(
"userinfo_endpoint",
self.provider_config.get("userinfo_endpoint")
)
# Use provider default scopes if not specified
if not config.get("scopes"):
self.scopes = self.provider_config.get("default_scopes", self.scopes)
def init_app(self, app):
"""Initialize OAuth with the app."""
try:
from authlib.integrations.flask_client import OAuth
except ImportError:
raise ImportError(
"authlib library required for OAuth. "
"Install with: pip install authlib requests"
)
self.oauth = OAuth(app)
# Register OAuth client
self.oauth_client = self.oauth.register(
name=self.provider_name,
client_id=self.client_id,
client_secret=self.client_secret,
server_metadata_url=None,
authorize_url=self.authorization_endpoint,
access_token_url=self.token_endpoint,
userinfo_endpoint=self.userinfo_endpoint,
client_kwargs={"scope": " ".join(self.scopes)},
)
def is_user_authorized(self, user_info):
"""Check if user is authorized based on email/domain.
Args:
user_info: Dict with user information (must contain 'email')
Returns:
bool: True if authorized
"""
email = user_info.get("email", "").lower()
# Check specific users
if self.authorized_users:
if email in [u.lower() for u in self.authorized_users]:
return True
# Check domains
if self.authorized_domains:
domain = email.split("@")[-1] if "@" in email else ""
if domain in [d.lower() for d in self.authorized_domains]:
return True
# If no restrictions configured, allow all
if not self.authorized_users and not self.authorized_domains:
return True
return False
def is_authenticated(self):
"""Check if current user is authenticated via OAuth."""
return "oauth_user" in session
def handle_unauthorized(self):
"""Redirect to OAuth login."""
return redirect(url_for(self.login_view))
def get_current_user(self):
"""Get current authenticated user."""
user_info = session.get("oauth_user")
if user_info:
return {
**user_info,
"auth_type": "oauth",
"provider": self.provider_name
}
return None
def logout(self):
"""Logout current user."""
session.pop("oauth_user", None)
session.pop("oauth_token", None)

8
libtisbackup/drivers/backup_mysql.py → libtisbackup/backup_mysql.py
View File

@ -30,7 +30,7 @@ except ImportError as e:
sys.stderr = sys.__stderr__
from libtisbackup import *
from libtisbackup.common import *
class backup_mysql(backup_generic):
@ -58,7 +58,11 @@ class backup_mysql(backup_generic):
raise Exception("backup destination directory already exists : %s" % self.dest_dir)
self.logger.debug("[%s] Connecting to %s with user root and key %s", self.backup_name, self.server_name, self.private_key)
mykey = load_ssh_private_key(self.private_key)
try:
mykey = paramiko.RSAKey.from_private_key_file(self.private_key)
except paramiko.SSHException:
# mykey = paramiko.DSSKey.from_private_key_file(self.private_key)
mykey = paramiko.Ed25519Key.from_private_key_file(self.private_key)
self.ssh = paramiko.SSHClient()
self.ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())

2
libtisbackup/drivers/backup_null.py → libtisbackup/backup_null.py Normal file → Executable file
View File

@ -21,7 +21,7 @@
import datetime
import os
from libtisbackup import *
from .common import *
class backup_null(backup_generic):

8
libtisbackup/drivers/backup_oracle.py → libtisbackup/backup_oracle.py
View File

@ -33,7 +33,7 @@ import datetime
import os
import re
from libtisbackup import *
from libtisbackup.common import *
class backup_oracle(backup_generic):
@ -51,7 +51,11 @@ class backup_oracle(backup_generic):
self.logger.debug(
"[%s] Connecting to %s with user %s and key %s", self.backup_name, self.server_name, self.username, self.private_key
)
mykey = load_ssh_private_key(self.private_key)
try:
mykey = paramiko.RSAKey.from_private_key_file(self.private_key)
except paramiko.SSHException:
# mykey = paramiko.DSSKey.from_private_key_file(self.private_key)
mykey = paramiko.Ed25519Key.from_private_key_file(self.private_key)
self.ssh = paramiko.SSHClient()
self.ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())

8
libtisbackup/drivers/backup_pgsql.py → libtisbackup/backup_pgsql.py
View File

@ -28,7 +28,7 @@ except ImportError as e:
sys.stderr = sys.__stderr__
from libtisbackup import *
from .common import *
class backup_pgsql(backup_generic):
@ -53,7 +53,11 @@ class backup_pgsql(backup_generic):
else:
raise Exception("backup destination directory already exists : %s" % self.dest_dir)
mykey = load_ssh_private_key(self.private_key)
try:
mykey = paramiko.RSAKey.from_private_key_file(self.private_key)
except paramiko.SSHException:
# mykey = paramiko.DSSKey.from_private_key_file(self.private_key)
mykey = paramiko.Ed25519Key.from_private_key_file(self.private_key)
self.logger.debug(
'[%s] Trying to connect to "%s" with username root and key "%s"', self.backup_name, self.server_name, self.private_key

2
libtisbackup/drivers/backup_rsync.py → libtisbackup/backup_rsync.py
View File

@ -25,7 +25,7 @@ import os.path
import re
import time
from libtisbackup import *
from libtisbackup.common import *
class backup_rsync(backup_generic):

2
libtisbackup/drivers/backup_rsync_btrfs.py → libtisbackup/backup_rsync_btrfs.py
View File

@ -25,7 +25,7 @@ import os.path
import re
import time
from libtisbackup import *
from .common import *
class backup_rsync_btrfs(backup_generic):

8
libtisbackup/drivers/backup_samba4.py → libtisbackup/backup_samba4.py
View File

@ -30,7 +30,7 @@ except ImportError as e:
sys.stderr = sys.__stderr__
from libtisbackup import *
from .common import *
class backup_samba4(backup_generic):
@ -54,7 +54,11 @@ class backup_samba4(backup_generic):
raise Exception("backup destination directory already exists : %s" % self.dest_dir)
self.logger.debug("[%s] Connecting to %s with user root and key %s", self.backup_name, self.server_name, self.private_key)
mykey = load_ssh_private_key(self.private_key)
try:
mykey = paramiko.RSAKey.from_private_key_file(self.private_key)
except paramiko.SSHException:
# mykey = paramiko.DSSKey.from_private_key_file(self.private_key)
mykey = paramiko.Ed25519Key.from_private_key_file(self.private_key)
self.ssh = paramiko.SSHClient()
self.ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())

8
libtisbackup/drivers/backup_sqlserver.py → libtisbackup/backup_sqlserver.py
View File

@ -34,7 +34,7 @@ import base64
import datetime
import os
from libtisbackup import *
from .common import *
class backup_sqlserver(backup_generic):
@ -53,7 +53,11 @@ class backup_sqlserver(backup_generic):
db_server_name = "localhost"
def do_backup(self, stats):
mykey = load_ssh_private_key(self.private_key)
try:
mykey = paramiko.RSAKey.from_private_key_file(self.private_key)
except paramiko.SSHException:
# mykey = paramiko.DSSKey.from_private_key_file(self.private_key)
mykey = paramiko.Ed25519Key.from_private_key_file(self.private_key)
self.logger.debug("[%s] Connecting to %s with user root and key %s", self.backup_name, self.server_name, self.private_key)
ssh = paramiko.SSHClient()

2
libtisbackup/drivers/backup_switch.py → libtisbackup/backup_switch.py
View File

@ -36,7 +36,7 @@ import pexpect
import requests
from . import XenAPI
from libtisbackup import *
from .common import *
class backup_switch(backup_generic):

2
libtisbackup/drivers/backup_vmdk.py → libtisbackup/backup_vmdk.py Normal file → Executable file
View File

@ -30,7 +30,7 @@ from pyVmomi import vim, vmodl
# Disable HTTPS verification warnings.
from requests.packages import urllib3
from libtisbackup import *
from .common import *
urllib3.disable_warnings()
import os

2
libtisbackup/drivers/backup_xcp_metadata.py → libtisbackup/backup_xcp_metadata.py
View File

@ -21,7 +21,7 @@
import paramiko
from libtisbackup import *
from .common import *
class backup_xcp_metadata(backup_generic):

15
libtisbackup/drivers/backup_xva.py → libtisbackup/backup_xva.py Normal file → Executable file
View File

@ -25,7 +25,6 @@ import os
import re
import socket
import ssl
import subprocess
import tarfile
import urllib.error
import urllib.parse
@ -35,7 +34,7 @@ from stat import *
import requests
from . import XenAPI
from libtisbackup import *
from .common import *
if hasattr(ssl, "_create_unverified_context"):
ssl._create_default_https_context = ssl._create_unverified_context
@ -197,18 +196,10 @@ class backup_xva(backup_generic):
session.logout()
if os.path.exists(filename_temp):
# Verify tar file integrity using subprocess instead of os.system
try:
subprocess.run(
["tar", "tf", filename_temp],
capture_output=True,
check=True,
timeout=300
)
except (subprocess.CalledProcessError, subprocess.TimeoutExpired):
tar = os.system('tar tf "%s" > /dev/null' % filename_temp)
if not tar == 0:
os.unlink(filename_temp)
return "Tar error"
if str2bool(self.verify_export):
self.verify_export_xva(filename_temp)
os.rename(filename_temp, filename)

549
libtisbackup/base_driver.py → libtisbackup/common.py
View File

@ -18,45 +18,550 @@
#
# -----------------------------------------------------------------------
"""Base backup driver class and driver registry."""
import datetime
import errno
import logging
import os
import re
import select
import shutil
import sqlite3
import subprocess
import sys
import time
from abc import ABC, abstractmethod
from iniparse import ConfigParser
from .database import BackupStat
from .process import monitor_stdout
from .ssh import load_ssh_private_key
from .utils import dateof, datetime2isodate, isodate2datetime
try:
sys.stderr = open("/dev/null") # Silence silly warnings from paramiko
import paramiko
except ImportError as e:
print(("Error : can not load paramiko library %s" % e))
raise
# Nagios state constants
sys.stderr = sys.__stderr__
nagiosStateOk = 0
nagiosStateWarning = 1
nagiosStateCritical = 2
nagiosStateUnknown = 3
# Global driver registry
backup_drivers = {}
def register_driver(driverclass):
"""Register a backup driver class in the global registry."""
backup_drivers[driverclass.type] = driverclass
def datetime2isodate(adatetime=None):
if not adatetime:
adatetime = datetime.datetime.now()
assert isinstance(adatetime, datetime.datetime)
return adatetime.isoformat()
def isodate2datetime(isodatestr):
# we remove the microseconds part as it is not working for python2.5 strptime
return datetime.datetime.strptime(isodatestr.split(".")[0], "%Y-%m-%dT%H:%M:%S")
def time2display(adatetime):
return adatetime.strftime("%Y-%m-%d %H:%M")
def hours_minutes(hours):
if hours is None:
return None
else:
return "%02i:%02i" % (int(hours), int((hours - int(hours)) * 60.0))
def fileisodate(filename):
return datetime.datetime.fromtimestamp(os.stat(filename).st_mtime).isoformat()
def dateof(adatetime):
return adatetime.replace(hour=0, minute=0, second=0, microsecond=0)
#####################################
# http://code.activestate.com/recipes/498181-add-thousands-separator-commas-to-formatted-number/
# Code from Michael Robellard's comment made 28 Feb 2010
# Modified for leading +, -, space on 1 Mar 2010 by Glenn Linderman
#
# Tail recursion removed and leading garbage handled on March 12 2010, Alessandro Forghieri
def splitThousands(s, tSep=",", dSep="."):
"""Splits a general float on thousands. GIGO on general input"""
if s is None:
return 0
if not isinstance(s, str):
s = str(s)
cnt = 0
numChars = dSep + "0123456789"
ls = len(s)
while cnt < ls and s[cnt] not in numChars:
cnt += 1
lhs = s[0:cnt]
s = s[cnt:]
if dSep == "":
cnt = -1
else:
cnt = s.rfind(dSep)
if cnt > 0:
rhs = dSep + s[cnt + 1 :]
s = s[:cnt]
else:
rhs = ""
splt = ""
while s != "":
splt = s[-3:] + tSep + splt
s = s[:-3]
return lhs + splt[:-1] + rhs
def call_external_process(shell_string):
p = subprocess.call(shell_string, shell=True)
if p != 0:
raise Exception("shell program exited with error code " + str(p), shell_string)
def check_string(test_string):
pattern = r"[^\.A-Za-z0-9\-_]"
if re.search(pattern, test_string):
# Character other then . a-z 0-9 was found
print(("Invalid : %r" % (test_string,)))
def convert_bytes(bytes):
if bytes is None:
return None
else:
bytes = float(bytes)
if bytes >= 1099511627776:
terabytes = bytes / 1099511627776
size = "%.2fT" % terabytes
elif bytes >= 1073741824:
gigabytes = bytes / 1073741824
size = "%.2fG" % gigabytes
elif bytes >= 1048576:
megabytes = bytes / 1048576
size = "%.2fM" % megabytes
elif bytes >= 1024:
kilobytes = bytes / 1024
size = "%.2fK" % kilobytes
else:
size = "%.2fb" % bytes
return size
## {{{ http://code.activestate.com/recipes/81189/ (r2)
def pp(cursor, data=None, rowlens=0, callback=None):
"""
pretty print a query result as a table
callback is a function called for each field (fieldname,value) to format the output
"""
def defaultcb(fieldname, value):
return value
if not callback:
callback = defaultcb
d = cursor.description
if not d:
return "#### NO RESULTS ###"
names = []
lengths = []
rules = []
if not data:
data = cursor.fetchall()
for dd in d: # iterate over description
l = dd[1]
if not l:
l = 12 # or default arg ...
l = max(l, len(dd[0])) # handle long names
names.append(dd[0])
lengths.append(l)
for col in range(len(lengths)):
if rowlens:
rls = [len(str(callback(d[col][0], row[col]))) for row in data if row[col]]
lengths[col] = max([lengths[col]] + rls)
rules.append("-" * lengths[col])
format = " ".join(["%%-%ss" % l for l in lengths])
result = [format % tuple(names)]
result.append(format % tuple(rules))
for row in data:
row_cb = []
for col in range(len(d)):
row_cb.append(callback(d[col][0], row[col]))
result.append(format % tuple(row_cb))
return "\n".join(result)
## end of http://code.activestate.com/recipes/81189/ }}}
def html_table(cur, callback=None):
"""
cur est un cursor issu d'une requete
callback est une fonction qui prend (rowmap,fieldname,value)
et renvoie une representation texte
"""
def safe_unicode(iso):
if iso is None:
return None
elif isinstance(iso, str):
return iso # .decode()
else:
return iso
def itermap(cur):
for row in cur:
yield dict((cur.description[idx][0], value) for idx, value in enumerate(row))
head = "<tr>" + "".join(["<th>" + c[0] + "</th>" for c in cur.description]) + "</tr>"
lines = ""
if callback:
for r in itermap(cur):
lines = (
lines
+ "<tr>"
+ "".join(["<td>" + str(callback(r, c[0], safe_unicode(r[c[0]]))) + "</td>" for c in cur.description])
+ "</tr>"
)
else:
for r in cur:
lines = lines + "<tr>" + "".join(["<td>" + safe_unicode(c) + "</td>" for c in r]) + "</tr>"
return "<table border=1 cellpadding=2 cellspacing=0>%s%s</table>" % (head, lines)
def monitor_stdout(aprocess, onoutputdata, context):
"""Reads data from stdout and stderr from aprocess and return as a string
on each chunk, call a call back onoutputdata(dataread)
"""
assert isinstance(aprocess, subprocess.Popen)
read_set = []
stdout = []
line = ""
if aprocess.stdout:
read_set.append(aprocess.stdout)
if aprocess.stderr:
read_set.append(aprocess.stderr)
while read_set:
try:
rlist, wlist, xlist = select.select(read_set, [], [])
except select.error as e:
if e.args[0] == errno.EINTR:
continue
raise
# Reads one line from stdout
if aprocess.stdout in rlist:
data = os.read(aprocess.stdout.fileno(), 1)
data = data.decode(errors="ignore")
if data == "":
aprocess.stdout.close()
read_set.remove(aprocess.stdout)
while data and data not in ("\n", "\r"):
line += data
data = os.read(aprocess.stdout.fileno(), 1)
data = data.decode(errors="ignore")
if line or data in ("\n", "\r"):
stdout.append(line)
if onoutputdata:
onoutputdata(line, context)
line = ""
# Reads one line from stderr
if aprocess.stderr in rlist:
data = os.read(aprocess.stderr.fileno(), 1)
data = data.decode(errors="ignore")
if data == "":
aprocess.stderr.close()
read_set.remove(aprocess.stderr)
while data and data not in ("\n", "\r"):
line += data
data = os.read(aprocess.stderr.fileno(), 1)
data = data.decode(errors="ignore")
if line or data in ("\n", "\r"):
stdout.append(line)
if onoutputdata:
onoutputdata(line, context)
line = ""
aprocess.wait()
if line:
stdout.append(line)
if onoutputdata:
onoutputdata(line, context)
return "\n".join(stdout)
def str2bool(val):
if not isinstance(type(val), bool):
return val.lower() in ("yes", "true", "t", "1")
class BackupStat:
dbpath = ""
db = None
logger = logging.getLogger("tisbackup")
def __init__(self, dbpath):
self.dbpath = dbpath
if not os.path.isfile(self.dbpath):
self.db = sqlite3.connect(self.dbpath)
self.initdb()
else:
self.db = sqlite3.connect(self.dbpath, check_same_thread=False)
if "'TYPE'" not in str(self.db.execute("select * from stats").description):
self.updatedb()
def updatedb(self):
self.logger.debug("Update stat database")
self.db.execute("alter table stats add column TYPE TEXT;")
self.db.execute("update stats set TYPE='BACKUP';")
self.db.commit()
def initdb(self):
assert isinstance(self.db, sqlite3.Connection)
self.logger.debug("Initialize stat database")
self.db.execute("""
create table stats (
backup_name TEXT,
server_name TEXT,
description TEXT,
backup_start TEXT,
backup_end TEXT,
backup_duration NUMERIC,
total_files_count INT,
written_files_count INT,
total_bytes INT,
written_bytes INT,
status TEXT,
log TEXT,
backup_location TEXT,
TYPE TEXT)""")
self.db.execute("""
create index idx_stats_backup_name on stats(backup_name);""")
self.db.execute("""
create index idx_stats_backup_location on stats(backup_location);""")
self.db.execute("""
CREATE INDEX idx_stats_backup_name_start on stats(backup_name,backup_start);""")
self.db.commit()
def start(self, backup_name, server_name, TYPE, description="", backup_location=None):
"""Add in stat DB a record for the newly running backup"""
return self.add(
backup_name=backup_name,
server_name=server_name,
description=description,
backup_start=datetime2isodate(),
status="Running",
TYPE=TYPE,
)
def finish(
self,
rowid,
total_files_count=None,
written_files_count=None,
total_bytes=None,
written_bytes=None,
log=None,
status="OK",
backup_end=None,
backup_duration=None,
backup_location=None,
):
"""Update record in stat DB for the finished backup"""
if not backup_end:
backup_end = datetime2isodate()
if backup_duration is None:
try:
# get duration using start of backup datetime
backup_duration = (
isodate2datetime(backup_end)
- isodate2datetime(self.query("select backup_start from stats where rowid=?", (rowid,))[0]["backup_start"])
).seconds / 3600.0
except:
backup_duration = None
# update stat record
self.db.execute(
"""\
update stats set
total_files_count=?,written_files_count=?,total_bytes=?,written_bytes=?,log=?,status=?,backup_end=?,backup_duration=?,backup_location=?
where
rowid = ?
""",
(
total_files_count,
written_files_count,
total_bytes,
written_bytes,
log,
status,
backup_end,
backup_duration,
backup_location,
rowid,
),
)
self.db.commit()
def add(
self,
backup_name="",
server_name="",
description="",
backup_start=None,
backup_end=None,
backup_duration=None,
total_files_count=None,
written_files_count=None,
total_bytes=None,
written_bytes=None,
status="draft",
log="",
TYPE="",
backup_location=None,
):
if not backup_start:
backup_start = datetime2isodate()
if not backup_end:
backup_end = datetime2isodate()
cur = self.db.execute(
"""\
insert into stats (
backup_name,
server_name,
description,
backup_start,
backup_end,
backup_duration,
total_files_count,
written_files_count,
total_bytes,
written_bytes,
status,
log,
backup_location,
TYPE) values (?,?,?,?,?,?,?,?,?,?,?,?,?,?)
""",
(
backup_name,
server_name,
description,
backup_start,
backup_end,
backup_duration,
total_files_count,
written_files_count,
total_bytes,
written_bytes,
status,
log,
backup_location,
TYPE,
),
)
self.db.commit()
return cur.lastrowid
def query(self, query, args=(), one=False):
"""
execute la requete query sur la db et renvoie un tableau de dictionnaires
"""
cur = self.db.execute(query, args)
rv = [dict((cur.description[idx][0], value) for idx, value in enumerate(row)) for row in cur.fetchall()]
return (rv[0] if rv else None) if one else rv
def last_backups(self, backup_name, count=30):
if backup_name:
cur = self.db.execute("select * from stats where backup_name=? order by backup_end desc limit ?", (backup_name, count))
else:
cur = self.db.execute("select * from stats order by backup_end desc limit ?", (count,))
def fcb(fieldname, value):
if fieldname in ("backup_start", "backup_end"):
return time2display(isodate2datetime(value))
elif "bytes" in fieldname:
return convert_bytes(value)
elif "count" in fieldname:
return splitThousands(value, " ", ".")
elif "backup_duration" in fieldname:
return hours_minutes(value)
else:
return value
# for r in self.query('select * from stats where backup_name=? order by backup_end desc limit ?',(backup_name,count)):
print((pp(cur, None, 1, fcb)))
def fcb(self, fields, fieldname, value):
if fieldname in ("backup_start", "backup_end"):
return time2display(isodate2datetime(value))
elif "bytes" in fieldname:
return convert_bytes(value)
elif "count" in fieldname:
return splitThousands(value, " ", ".")
elif "backup_duration" in fieldname:
return hours_minutes(value)
else:
return value
def as_html(self, cur):
if cur:
return html_table(cur, self.fcb)
else:
return html_table(self.db.execute("select * from stats order by backup_start asc"), self.fcb)
def ssh_exec(command, ssh=None, server_name="", remote_user="", private_key="", ssh_port=22):
"""execute command on server_name using the provided ssh connection
or creates a new connection if ssh is not provided.
returns (exit_code,output)
output is the concatenation of stdout and stderr
"""
if not ssh:
assert server_name and remote_user and private_key
try:
mykey = paramiko.RSAKey.from_private_key_file(private_key)
except paramiko.SSHException:
# mykey = paramiko.DSSKey.from_private_key_file(private_key)
mykey = paramiko.Ed25519Key.from_private_key_file(private_key)
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(server_name, username=remote_user, pkey=mykey, port=ssh_port)
tran = ssh.get_transport()
chan = tran.open_session()
# chan.set_combine_stderr(True)
chan.get_pty()
stdout = chan.makefile()
chan.exec_command(command)
stdout.flush()
output_base = stdout.read()
output = output_base.decode(errors="ignore").replace("'", "")
exit_code = chan.recv_exit_status()
return (exit_code, output)
class backup_generic(ABC):
"""Generic ancestor class for backups, not registered"""
@ -135,11 +640,14 @@ class backup_generic(ABC):
def do_preexec(self, stats):
self.logger.info("[%s] executing preexec %s ", self.backup_name, self.preexec)
mykey = load_ssh_private_key(self.private_key)
try:
mykey = paramiko.RSAKey.from_private_key_file(self.private_key)
except paramiko.SSHException:
mykey = paramiko.DSSKey.from_private_key_file(self.private_key)
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(self.server_name, username=self.remote_user, pkey=mykey, port=self.ssh_port)
ssh.connect(self.server_name, username=self.remote_user, pkey=mykey)
tran = ssh.get_transport()
chan = tran.open_session()
@ -158,11 +666,14 @@ class backup_generic(ABC):
def do_postexec(self, stats):
self.logger.info("[%s] executing postexec %s ", self.backup_name, self.postexec)
mykey = load_ssh_private_key(self.private_key)
try:
mykey = paramiko.RSAKey.from_private_key_file(self.private_key)
except paramiko.SSHException:
mykey = paramiko.DSSKey.from_private_key_file(self.private_key)
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(self.server_name, username=self.remote_user, pkey=mykey, port=self.ssh_port)
ssh.connect(self.server_name, username=self.remote_user, pkey=mykey)
tran = ssh.get_transport()
chan = tran.open_session()
@ -524,3 +1035,13 @@ class backup_generic(ABC):
backup_location=backup_dest,
)
return stats
if __name__ == "__main__":
logger = logging.getLogger("tisbackup")
logger.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(asctime)s %(levelname)s %(message)s")
handler = logging.StreamHandler()
handler.setFormatter(formatter)
logger.addHandler(handler)
dbstat = BackupStat("/backup/data/log/tisbackup.sqlite")

2
libtisbackup/drivers/copy_vm_xcp.py → libtisbackup/copy_vm_xcp.py Normal file → Executable file
View File

@ -34,7 +34,7 @@ import urllib.request
from stat import *
from . import XenAPI
from libtisbackup import *
from .common import *
if hasattr(ssl, "_create_unverified_context"):
ssl._create_default_https_context = ssl._create_unverified_context

261
libtisbackup/database.py
View File

@ -1,261 +0,0 @@
#!/usr/bin/python3
# -*- coding: utf-8 -*-
# -----------------------------------------------------------------------
# This file is part of TISBackup
#
# TISBackup is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# TISBackup is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with TISBackup. If not, see <http://www.gnu.org/licenses/>.
#
# -----------------------------------------------------------------------
"""Database management for backup statistics and history."""
import logging
import os
import sqlite3
from .utils import (
convert_bytes,
datetime2isodate,
hours_minutes,
html_table,
isodate2datetime,
pp,
splitThousands,
time2display,
)
class BackupStat:
"""Manages SQLite database for backup statistics and history."""
dbpath = ""
db = None
logger = logging.getLogger("tisbackup")
def __init__(self, dbpath):
self.dbpath = dbpath
if not os.path.isfile(self.dbpath):
self.db = sqlite3.connect(self.dbpath)
self.initdb()
else:
self.db = sqlite3.connect(self.dbpath, check_same_thread=False)
if "'TYPE'" not in str(self.db.execute("select * from stats").description):
self.updatedb()
def updatedb(self):
"""Update database schema to add TYPE column if missing."""
self.logger.debug("Update stat database")
self.db.execute("alter table stats add column TYPE TEXT;")
self.db.execute("update stats set TYPE='BACKUP';")
self.db.commit()
def initdb(self):
"""Initialize database schema."""
assert isinstance(self.db, sqlite3.Connection)
self.logger.debug("Initialize stat database")
self.db.execute("""
create table stats (
backup_name TEXT,
server_name TEXT,
description TEXT,
backup_start TEXT,
backup_end TEXT,
backup_duration NUMERIC,
total_files_count INT,
written_files_count INT,
total_bytes INT,
written_bytes INT,
status TEXT,
log TEXT,
backup_location TEXT,
TYPE TEXT)""")
self.db.execute("""
create index idx_stats_backup_name on stats(backup_name);""")
self.db.execute("""
create index idx_stats_backup_location on stats(backup_location);""")
self.db.execute("""
CREATE INDEX idx_stats_backup_name_start on stats(backup_name,backup_start);""")
self.db.commit()
def start(self, backup_name, server_name, TYPE, description="", backup_location=None):
"""Add in stat DB a record for the newly running backup"""
return self.add(
backup_name=backup_name,
server_name=server_name,
description=description,
backup_start=datetime2isodate(),
status="Running",
TYPE=TYPE,
)
def finish(
self,
rowid,
total_files_count=None,
written_files_count=None,
total_bytes=None,
written_bytes=None,
log=None,
status="OK",
backup_end=None,
backup_duration=None,
backup_location=None,
):
"""Update record in stat DB for the finished backup"""
if not backup_end:
backup_end = datetime2isodate()
if backup_duration is None:
try:
# get duration using start of backup datetime
backup_duration = (
isodate2datetime(backup_end)
- isodate2datetime(self.query("select backup_start from stats where rowid=?", (rowid,))[0]["backup_start"])
).seconds / 3600.0
except:
backup_duration = None
# update stat record
self.db.execute(
"""\
update stats set
total_files_count=?,written_files_count=?,total_bytes=?,written_bytes=?,log=?,status=?,backup_end=?,backup_duration=?,backup_location=?
where
rowid = ?
""",
(
total_files_count,
written_files_count,
total_bytes,
written_bytes,
log,
status,
backup_end,
backup_duration,
backup_location,
rowid,
),
)
self.db.commit()
def add(
self,
backup_name="",
server_name="",
description="",
backup_start=None,
backup_end=None,
backup_duration=None,
total_files_count=None,
written_files_count=None,
total_bytes=None,
written_bytes=None,
status="draft",
log="",
TYPE="",
backup_location=None,
):
"""Add a new backup record to the database."""
if not backup_start:
backup_start = datetime2isodate()
if not backup_end:
backup_end = datetime2isodate()
cur = self.db.execute(
"""\
insert into stats (
backup_name,
server_name,
description,
backup_start,
backup_end,
backup_duration,
total_files_count,
written_files_count,
total_bytes,
written_bytes,
status,
log,
backup_location,
TYPE) values (?,?,?,?,?,?,?,?,?,?,?,?,?,?)
""",
(
backup_name,
server_name,
description,
backup_start,
backup_end,
backup_duration,
total_files_count,
written_files_count,
total_bytes,
written_bytes,
status,
log,
backup_location,
TYPE,
),
)
self.db.commit()
return cur.lastrowid
def query(self, query, args=(), one=False):
"""
execute la requete query sur la db et renvoie un tableau de dictionnaires
"""
cur = self.db.execute(query, args)
rv = [dict((cur.description[idx][0], value) for idx, value in enumerate(row)) for row in cur.fetchall()]
return (rv[0] if rv else None) if one else rv
def last_backups(self, backup_name, count=30):
"""Display last N backups for a given backup_name."""
if backup_name:
cur = self.db.execute("select * from stats where backup_name=? order by backup_end desc limit ?", (backup_name, count))
else:
cur = self.db.execute("select * from stats order by backup_end desc limit ?", (count,))
def fcb(fieldname, value):
if fieldname in ("backup_start", "backup_end"):
return time2display(isodate2datetime(value))
elif "bytes" in fieldname:
return convert_bytes(value)
elif "count" in fieldname:
return splitThousands(value, " ", ".")
elif "backup_duration" in fieldname:
return hours_minutes(value)
else:
return value
# for r in self.query('select * from stats where backup_name=? order by backup_end desc limit ?',(backup_name,count)):
print((pp(cur, None, 1, fcb)))
def fcb(self, fields, fieldname, value):
"""Format callback for HTML table display."""
if fieldname in ("backup_start", "backup_end"):
return time2display(isodate2datetime(value))
elif "bytes" in fieldname:
return convert_bytes(value)
elif "count" in fieldname:
return splitThousands(value, " ", ".")
elif "backup_duration" in fieldname:
return hours_minutes(value)
else:
return value
def as_html(self, cur):
"""Convert cursor to HTML table."""
if cur:
return html_table(cur, self.fcb)
else:
return html_table(self.db.execute("select * from stats order by backup_start asc"), self.fcb)

60
libtisbackup/drivers/__init__.py
View File

@ -1,60 +0,0 @@
# -----------------------------------------------------------------------
# This file is part of TISBackup
#
# TISBackup is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# TISBackup is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with TISBackup. If not, see <http://www.gnu.org/licenses/>.
#
# -----------------------------------------------------------------------
"""
TISBackup drivers - Pluggable backup driver implementations.
This package contains all backup driver implementations:
- Database drivers (MySQL, PostgreSQL, Oracle, SQL Server)
- File sync drivers (rsync, rsync+btrfs)
- VM backup drivers (XenServer XVA, VMware VMDK)
- Other drivers (Samba4, network switches, etc.)
"""
# Import all drivers to ensure they register themselves
from .backup_mysql import backup_mysql
from .backup_null import backup_null
from .backup_oracle import backup_oracle
from .backup_pgsql import backup_pgsql
from .backup_rsync import backup_rsync, backup_rsync_ssh
from .backup_rsync_btrfs import backup_rsync_btrfs, backup_rsync__btrfs_ssh
from .backup_samba4 import backup_samba4
from .backup_sqlserver import backup_sqlserver
from .backup_switch import backup_switch
from .backup_vmdk import backup_vmdk
from .backup_xcp_metadata import backup_xcp_metadata
from .backup_xva import backup_xva
from .copy_vm_xcp import copy_vm_xcp
__all__ = [
"backup_mysql",
"backup_null",
"backup_oracle",
"backup_pgsql",
"backup_rsync",
"backup_rsync_ssh",
"backup_rsync_btrfs",
"backup_rsync__btrfs_ssh",
"backup_samba4",
"backup_sqlserver",
"backup_switch",
"backup_vmdk",
"backup_xcp_metadata",
"backup_xva",
"copy_vm_xcp",
]

97
libtisbackup/process.py
View File

@ -1,97 +0,0 @@
#!/usr/bin/python3
# -*- coding: utf-8 -*-
# -----------------------------------------------------------------------
# This file is part of TISBackup
#
# TISBackup is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# TISBackup is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with TISBackup. If not, see <http://www.gnu.org/licenses/>.
#
# -----------------------------------------------------------------------
"""Process execution and monitoring utilities."""
import errno
import os
import select
import subprocess
def call_external_process(shell_string):
"""Execute a shell command and raise exception on non-zero exit code."""
p = subprocess.call(shell_string, shell=True)
if p != 0:
raise Exception("shell program exited with error code " + str(p), shell_string)
def monitor_stdout(aprocess, onoutputdata, context):
"""Reads data from stdout and stderr from aprocess and return as a string
on each chunk, call a call back onoutputdata(dataread)
"""
assert isinstance(aprocess, subprocess.Popen)
read_set = []
stdout = []
line = ""
if aprocess.stdout:
read_set.append(aprocess.stdout)
if aprocess.stderr:
read_set.append(aprocess.stderr)
while read_set:
try:
rlist, wlist, xlist = select.select(read_set, [], [])
except select.error as e:
if e.args[0] == errno.EINTR:
continue
raise
# Reads one line from stdout
if aprocess.stdout in rlist:
data = os.read(aprocess.stdout.fileno(), 1)
data = data.decode(errors="ignore")
if data == "":
aprocess.stdout.close()
read_set.remove(aprocess.stdout)
while data and data not in ("\n", "\r"):
line += data
data = os.read(aprocess.stdout.fileno(), 1)
data = data.decode(errors="ignore")
if line or data in ("\n", "\r"):
stdout.append(line)
if onoutputdata:
onoutputdata(line, context)
line = ""
# Reads one line from stderr
if aprocess.stderr in rlist:
data = os.read(aprocess.stderr.fileno(), 1)
data = data.decode(errors="ignore")
if data == "":
aprocess.stderr.close()
read_set.remove(aprocess.stderr)
while data and data not in ("\n", "\r"):
line += data
data = os.read(aprocess.stderr.fileno(), 1)
data = data.decode(errors="ignore")
if line or data in ("\n", "\r"):
stdout.append(line)
if onoutputdata:
onoutputdata(line, context)
line = ""
aprocess.wait()
if line:
stdout.append(line)
if onoutputdata:
onoutputdata(line, context)
return "\n".join(stdout)

104
libtisbackup/ssh.py
View File

@ -1,104 +0,0 @@
#!/usr/bin/python3
# -*- coding: utf-8 -*-
# -----------------------------------------------------------------------
# This file is part of TISBackup
#
# TISBackup is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# TISBackup is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with TISBackup. If not, see <http://www.gnu.org/licenses/>.
#
# -----------------------------------------------------------------------
"""SSH operations and key management utilities."""
import sys
try:
sys.stderr = open("/dev/null") # Silence silly warnings from paramiko
import paramiko
except ImportError as e:
print(("Error : can not load paramiko library %s" % e))
raise
sys.stderr = sys.__stderr__
def load_ssh_private_key(private_key_path):
"""Load SSH private key with modern algorithm support.
Tries to load the key in order of preference:
1. Ed25519 (most secure, modern)
2. ECDSA (secure, widely supported)
3. RSA (legacy, still secure with sufficient key size)
DSA is not supported as it's deprecated and insecure.
Args:
private_key_path: Path to the private key file
Returns:
paramiko key object
Raises:
paramiko.SSHException: If key cannot be loaded
"""
key_types = [
("Ed25519", paramiko.Ed25519Key),
("ECDSA", paramiko.ECDSAKey),
("RSA", paramiko.RSAKey),
]
last_exception = None
for key_name, key_class in key_types:
try:
return key_class.from_private_key_file(private_key_path)
except paramiko.SSHException as e:
last_exception = e
continue
# If we get here, none of the key types worked
raise paramiko.SSHException(
f"Unable to load private key from {private_key_path}. "
f"Supported formats: Ed25519 (recommended), ECDSA, RSA. "
f"DSA keys are no longer supported. "
f"Last error: {last_exception}"
)
def ssh_exec(command, ssh=None, server_name="", remote_user="", private_key="", ssh_port=22):
"""execute command on server_name using the provided ssh connection
or creates a new connection if ssh is not provided.
returns (exit_code,output)
output is the concatenation of stdout and stderr
"""
if not ssh:
assert server_name and remote_user and private_key
mykey = load_ssh_private_key(private_key)
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(server_name, username=remote_user, pkey=mykey, port=ssh_port)
tran = ssh.get_transport()
chan = tran.open_session()
# chan.set_combine_stderr(True)
chan.get_pty()
stdout = chan.makefile()
chan.exec_command(command)
stdout.flush()
output_base = stdout.read()
output = output_base.decode(errors="ignore").replace("'", "")
exit_code = chan.recv_exit_status()
return (exit_code, output)

222
libtisbackup/utils.py
View File

@ -1,222 +0,0 @@
#!/usr/bin/python3
# -*- coding: utf-8 -*-
# -----------------------------------------------------------------------
# This file is part of TISBackup
#
# TISBackup is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# TISBackup is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with TISBackup. If not, see <http://www.gnu.org/licenses/>.
#
# -----------------------------------------------------------------------
"""Utility functions for date/time formatting, number formatting, and display helpers."""
import datetime
import os
def datetime2isodate(adatetime=None):
"""Convert datetime to ISO format string."""
if not adatetime:
adatetime = datetime.datetime.now()
assert isinstance(adatetime, datetime.datetime)
return adatetime.isoformat()
def isodate2datetime(isodatestr):
"""Convert ISO format string to datetime."""
# we remove the microseconds part as it is not working for python2.5 strptime
return datetime.datetime.strptime(isodatestr.split(".")[0], "%Y-%m-%dT%H:%M:%S")
def time2display(adatetime):
"""Format datetime for display."""
return adatetime.strftime("%Y-%m-%d %H:%M")
def hours_minutes(hours):
"""Convert decimal hours to HH:MM format."""
if hours is None:
return None
else:
return "%02i:%02i" % (int(hours), int((hours - int(hours)) * 60.0))
def fileisodate(filename):
"""Get file modification time as ISO date string."""
return datetime.datetime.fromtimestamp(os.stat(filename).st_mtime).isoformat()
def dateof(adatetime):
"""Get date part of datetime (midnight)."""
return adatetime.replace(hour=0, minute=0, second=0, microsecond=0)
#####################################
# http://code.activestate.com/recipes/498181-add-thousands-separator-commas-to-formatted-number/
# Code from Michael Robellard's comment made 28 Feb 2010
# Modified for leading +, -, space on 1 Mar 2010 by Glenn Linderman
#
# Tail recursion removed and leading garbage handled on March 12 2010, Alessandro Forghieri
def splitThousands(s, tSep=",", dSep="."):
"""Splits a general float on thousands. GIGO on general input"""
if s is None:
return 0
if not isinstance(s, str):
s = str(s)
cnt = 0
numChars = dSep + "0123456789"
ls = len(s)
while cnt < ls and s[cnt] not in numChars:
cnt += 1
lhs = s[0:cnt]
s = s[cnt:]
if dSep == "":
cnt = -1
else:
cnt = s.rfind(dSep)
if cnt > 0:
rhs = dSep + s[cnt + 1 :]
s = s[:cnt]
else:
rhs = ""
splt = ""
while s != "":
splt = s[-3:] + tSep + splt
s = s[:-3]
return lhs + splt[:-1] + rhs
def convert_bytes(bytes):
"""Convert bytes to human-readable format (T/G/M/K/b)."""
if bytes is None:
return None
else:
bytes = float(bytes)
if bytes >= 1099511627776:
terabytes = bytes / 1099511627776
size = "%.2fT" % terabytes
elif bytes >= 1073741824:
gigabytes = bytes / 1073741824
size = "%.2fG" % gigabytes
elif bytes >= 1048576:
megabytes = bytes / 1048576
size = "%.2fM" % megabytes
elif bytes >= 1024:
kilobytes = bytes / 1024
size = "%.2fK" % kilobytes
else:
size = "%.2fb" % bytes
return size
def check_string(test_string):
"""Check if string contains only alphanumeric characters, dots, dashes, and underscores."""
import re
pattern = r"[^\.A-Za-z0-9\-_]"
if re.search(pattern, test_string):
# Character other then . a-z 0-9 was found
print(("Invalid : %r" % (test_string,)))
def str2bool(val):
"""Convert string to boolean."""
if not isinstance(type(val), bool):
return val.lower() in ("yes", "true", "t", "1")
## {{{ http://code.activestate.com/recipes/81189/ (r2)
def pp(cursor, data=None, rowlens=0, callback=None):
"""
pretty print a query result as a table
callback is a function called for each field (fieldname,value) to format the output
"""
def defaultcb(fieldname, value):
return value
if not callback:
callback = defaultcb
d = cursor.description
if not d:
return "#### NO RESULTS ###"
names = []
lengths = []
rules = []
if not data:
data = cursor.fetchall()
for dd in d: # iterate over description
l = dd[1]
if not l:
l = 12 # or default arg ...
l = max(l, len(dd[0])) # handle long names
names.append(dd[0])
lengths.append(l)
for col in range(len(lengths)):
if rowlens:
rls = [len(str(callback(d[col][0], row[col]))) for row in data if row[col]]
lengths[col] = max([lengths[col]] + rls)
rules.append("-" * lengths[col])
format = " ".join(["%%-%ss" % l for l in lengths])
result = [format % tuple(names)]
result.append(format % tuple(rules))
for row in data:
row_cb = []
for col in range(len(d)):
row_cb.append(callback(d[col][0], row[col]))
result.append(format % tuple(row_cb))
return "\n".join(result)
## end of http://code.activestate.com/recipes/81189/ }}}
def html_table(cur, callback=None):
"""
cur est un cursor issu d'une requete
callback est une fonction qui prend (rowmap,fieldname,value)
et renvoie une representation texte
"""
def safe_unicode(iso):
if iso is None:
return None
elif isinstance(iso, str):
return iso # .decode()
else:
return iso
def itermap(cur):
for row in cur:
yield dict((cur.description[idx][0], value) for idx, value in enumerate(row))
head = "<tr>" + "".join(["<th>" + c[0] + "</th>" for c in cur.description]) + "</tr>"
lines = ""
if callback:
for r in itermap(cur):
lines = (
lines
+ "<tr>"
+ "".join(["<td>" + str(callback(r, c[0], safe_unicode(r[c[0]]))) + "</td>" for c in cur.description])
+ "</tr>"
)
else:
for r in cur:
lines = lines + "<tr>" + "".join(["<td>" + safe_unicode(c) + "</td>" for c in r]) + "</tr>"
return "<table border=1 cellpadding=2 cellspacing=0>%s%s</table>" % (head, lines)

114
pyproject.toml
View File

@ -4,35 +4,19 @@ version = "1.8.0"
description = "Backup server side executed python scripts for managing linux and windows system and application data backups, developed by adminsys for adminsys"
readme = "README.md"
dependencies = [
"authlib>=1.3.0",
"bcrypt>=4.0.0",
"flask==3.1.0",
"flask-login>=0.6.0",
"huey==2.5.3",
"iniparse==0.5",
"paramiko==4.0.0",
"paramiko==3.5.1",
"peewee==3.17.9",
"pexpect==4.9.0",
"pyvmomi>=8.0.0",
"redis==5.2.1",
"requests==2.32.3",
"ruff>=0.13.3",
"simplejson==3.20.1",
"six==1.17.0",
]
requires-python = ">=3.13"
[project.optional-dependencies]
# Documentation dependencies
docs = [
"docutils",
"sphinx>=7.0.0,<8.0.0",
"sphinx_rtd_theme",
"sphinxjp.themes.revealjs",
"sphinx-intl",
"sphinx-tabs",
]
[tool.black]
line-length = 140
@ -44,99 +28,3 @@ indent-width = 4
[tool.ruff.lint]
ignore = ["F401", "F403", "F405", "E402", "E701", "E722", "E741"]
[tool.pytest.ini_options]
# Pytest configuration for TISBackup
# Test discovery patterns
python_files = ["test_*.py"]
python_classes = ["Test*"]
python_functions = ["test_*"]
# Test paths
testpaths = ["tests"]
# Output options
addopts = [
"-v",
"--strict-markers",
"--tb=short",
"--color=yes",
]
# Markers for categorizing tests
markers = [
"unit: Unit tests for individual functions/methods",
"integration: Integration tests that test multiple components together",
"ssh: Tests related to SSH functionality",
"slow: Tests that take a long time to run",
]
# Minimum Python version
minversion = "3.13"
# Coverage options (optional - uncomment when pytest-cov is installed)
# addopts = ["--cov=libtisbackup", "--cov-report=html", "--cov-report=term-missing"]
[tool.pylint.main]
# Maximum line length
max-line-length = 140
# Files or directories to skip
ignore = ["tests", ".venv", "__pycache__", ".pytest_cache", "build", "dist"]
[tool.pylint."messages control"]
# Disable specific warnings to align with ruff configuration
disable = [
"C0103", # invalid-name (similar to ruff E741)
"C0114", # missing-module-docstring
"C0115", # missing-class-docstring
"C0116", # missing-function-docstring
"R0902", # too-many-instance-attributes
"R0903", # too-few-public-methods
"R0913", # too-many-arguments
"R0914", # too-many-locals
"W0703", # broad-except (similar to ruff E722)
"W0719", # broad-exception-raised
]
[tool.pylint.format]
# Indentation settings
indent-string = " "
[tool.coverage.run]
# Source code to measure coverage for
source = ["libtisbackup"]
# Omit certain files
omit = [
"*/tests/*",
"*/__pycache__/*",
"*/site-packages/*",
"*/.venv/*",
]
[tool.coverage.report]
# Precision for coverage percentage
precision = 2
# Show lines that weren't covered
show_missing = true
# Skip files with no executable code
skip_empty = true
# Fail if coverage is below this percentage
# fail_under = 80
[tool.coverage.html]
# Directory for HTML coverage report
directory = "htmlcov"
[dependency-groups]
dev = [
"pylint>=3.0.0",
"pytest>=8.4.2",
"pytest-cov>=6.0.0",
"pytest-mock>=3.15.1",
]

25
requirements.txt
View File

@ -1,15 +1,10 @@
authlib>=1.3.0
bcrypt>=4.0.0
flask==3.1.0
flask-login>=0.6.0
huey==2.5.3
iniparse==0.5
paramiko==4.0.0
peewee==3.17.9
pexpect==4.9.0
pyvmomi>=8.0.0
redis==5.2.1
requests==2.32.3
ruff>=0.13.3
simplejson==3.20.1
six==1.17.0
flask==3.1.0
huey==2.5.3
iniparse==0.5
paramiko==3.5.1
peewee==3.17.9
pexpect==4.9.0
redis==5.2.1
requests==2.32.3
simplejson==3.20.1
six==1.17.0

130
samples/auth-config-examples.ini
View File

@ -1,130 +0,0 @@
# TISBackup Authentication Configuration Examples
# Add to tisbackup_gui.ini under [authentication] section
# ============================================
# Option 1: No Authentication (NOT RECOMMENDED)
# ============================================
[authentication]
type = none
# ============================================
# Option 2: HTTP Basic Authentication
# ============================================
[authentication]
type = basic
username = admin
# Plain text password (NOT RECOMMENDED for production)
password = changeme
use_bcrypt = False
realm = TISBackup Admin
# RECOMMENDED: Use bcrypt hash
# Generate hash with: python3 -c "import bcrypt; print(bcrypt.hashpw(b'yourpassword', bcrypt.gensalt()).decode())"
[authentication]
type = basic
username = admin
password = $2b$12$LQv3c1yqBWVHxkd0LHAkCOYz6TtxMQJqhN8/LewY5GyYWv.5qVQK6
use_bcrypt = True
realm = TISBackup Admin
# ============================================
# Option 3: Flask-Login (Username/Password with Sessions)
# ============================================
[authentication]
type = flask-login
# Users can be defined inline or in a file
users_file = /etc/tis/users.txt
use_bcrypt = True
login_view = login
# User file format (users.txt):
# username:bcrypt_password_hash
# Example:
# admin:$2b$12$LQv3c1yqBWVHxkd0LHAkCOYz6TtxMQJqhN8/LewY5GyYWv.5qVQK6
# operator:$2b$12$abcdefghijklmnopqrstuvwxyz1234567890ABCDEFGHIJKLMNO
# ============================================
# Option 4: OAuth2 - Google
# ============================================
[authentication]
type = oauth
provider = google
client_id = your-client-id.apps.googleusercontent.com
client_secret = your-client-secret
redirect_uri = http://localhost:8080/oauth/callback
# Restrict to specific domains
authorized_domains = example.com,mycompany.com
# Or restrict to specific users
authorized_users = admin@example.com,backup-admin@example.com
# To get Google OAuth credentials:
# 1. Go to https://console.cloud.google.com/apis/credentials
# 2. Create OAuth 2.0 Client ID
# 3. Add authorized redirect URI: http://your-server:8080/oauth/callback
# ============================================
# Option 5: OAuth2 - GitHub
# ============================================
[authentication]
type = oauth
provider = github
client_id = your-github-client-id
client_secret = your-github-client-secret
redirect_uri = http://localhost:8080/oauth/callback
# Restrict to specific GitHub users (by email)
authorized_users = admin@example.com
# To get GitHub OAuth credentials:
# 1. Go to Settings > Developer settings > OAuth Apps
# 2. Register a new application
# 3. Set Authorization callback URL: http://your-server:8080/oauth/callback
# ============================================
# Option 6: OAuth2 - GitLab
# ============================================
[authentication]
type = oauth
provider = gitlab
client_id = your-gitlab-application-id
client_secret = your-gitlab-secret
redirect_uri = http://localhost:8080/oauth/callback
authorized_domains = example.com
# To get GitLab OAuth credentials:
# 1. Go to User Settings > Applications
# 2. Create new application with scopes: read_user, email
# 3. Set Redirect URI: http://your-server:8080/oauth/callback
# ============================================
# Option 7: OAuth2 - Generic Provider
# ============================================
[authentication]
type = oauth
provider = generic
client_id = your-client-id
client_secret = your-client-secret
redirect_uri = http://localhost:8080/oauth/callback
# Custom OAuth endpoints
authorization_endpoint = https://auth.example.com/oauth/authorize
token_endpoint = https://auth.example.com/oauth/token
userinfo_endpoint = https://auth.example.com/oauth/userinfo
scopes = openid,email,profile
authorized_domains = example.com
# ============================================
# Security Notes
# ============================================
# 1. Always use HTTPS in production (reverse proxy with TLS)
# 2. Set strong Flask secret_key via TISBACKUP_SECRET_KEY env var
# 3. For Basic Auth, always use bcrypt hashed passwords
# 4. For OAuth, restrict access via authorized_domains or authorized_users
# 5. Keep client secrets secure and never commit to version control
# 6. Regularly rotate OAuth client secrets
# 7. Use environment variables for sensitive data when possible

9
tasks.py
View File

@ -1,6 +1,5 @@
import logging
import os
import subprocess
from huey import RedisHuey
@ -35,12 +34,8 @@ def run_export_backup(base, config_file, mount_point, backup_sections):
return str(e)
finally:
# Safely unmount using subprocess instead of os.system
try:
subprocess.run(["/bin/umount", mount_point], check=True, timeout=30)
os.rmdir(mount_point)
except (subprocess.CalledProcessError, subprocess.TimeoutExpired, OSError) as e:
logger.error(f"Failed to unmount {mount_point}: {e}")
os.system("/bin/umount %s" % mount_point)
os.rmdir(mount_point)
return "ok"

145
tests/README.md
View File

@ -1,145 +0,0 @@
# TISBackup Test Suite
This directory contains the test suite for TISBackup using pytest.
## Running Tests
### Run all tests
```bash
uv run pytest
```
### Run tests for a specific module
```bash
uv run pytest tests/test_ssh.py
```
### Run with verbose output
```bash
uv run pytest -v
```
### Run tests matching a pattern
```bash
uv run pytest -k "ssh" -v
```
### Run with coverage (requires pytest-cov)
```bash
uv run pytest --cov=libtisbackup --cov-report=html
```
## Test Structure
### Current Test Modules
- **[test_ssh.py](test_ssh.py)** - Tests for SSH operations module
- `TestLoadSSHPrivateKey` - Tests for key loading with Ed25519, ECDSA, and RSA support
- `TestSSHExec` - Tests for remote command execution via SSH
- `TestSSHModuleIntegration` - Integration tests for SSH functionality
## Test Categories
Tests are organized using pytest markers:
- `@pytest.mark.unit` - Unit tests for individual functions
- `@pytest.mark.integration` - Integration tests for multiple components
- `@pytest.mark.ssh` - SSH-related tests
- `@pytest.mark.slow` - Long-running tests
### Run only unit tests
```bash
uv run pytest -m unit
```
### Run only SSH tests
```bash
uv run pytest -m ssh
```
## Writing New Tests
### Test File Naming
- Test files should be named `test_*.py`
- Place them in the `tests/` directory
### Test Class Naming
- Test classes should start with `Test`
- Example: `TestMyModule`
### Test Function Naming
- Test functions should start with `test_`
- Use descriptive names: `test_load_ed25519_key_success`
### Example Test Structure
```python
import pytest
from libtisbackup.mymodule import my_function
class TestMyFunction:
"""Test cases for my_function."""
def test_basic_functionality(self):
"""Test basic use case."""
result = my_function("input")
assert result == "expected_output"
def test_error_handling(self):
"""Test error handling."""
with pytest.raises(ValueError):
my_function(None)
```
## Mocking
The test suite uses `pytest-mock` for mocking dependencies. Common patterns:
### Mocking with patch
```python
from unittest.mock import patch, Mock
def test_with_mock():
with patch('module.function') as mock_func:
mock_func.return_value = "mocked"
result = my_code()
assert result == "mocked"
```
### Using pytest fixtures
```python
@pytest.fixture
def mock_ssh_client():
return Mock(spec=paramiko.SSHClient)
def test_with_fixture(mock_ssh_client):
# Use the fixture
pass
```
## Coverage Goals
Aim for:
- **80%+** overall code coverage
- **90%+** for critical modules (ssh, database, base_driver)
- **100%** for utility functions
## Test Configuration
Test configuration is in the `[tool.pytest.ini_options]` section of [pyproject.toml](../pyproject.toml):
- Test discovery patterns
- Output formatting
- Markers definition
- Minimum Python version
## Continuous Integration
Tests should pass before merging:
```bash
# Run linting
uv run ruff check .
# Run tests
uv run pytest -v
# Both must pass
```

0
tests/__init__.py
View File

325
tests/test_ssh.py
View File

@ -1,325 +0,0 @@
#!/usr/bin/python3
# -*- coding: utf-8 -*-
# -----------------------------------------------------------------------
# This file is part of TISBackup
#
# TISBackup is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# TISBackup is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with TISBackup. If not, see <http://www.gnu.org/licenses/>.
#
# -----------------------------------------------------------------------
"""
Test suite for libtisbackup.ssh module.
Tests SSH key loading and remote command execution functionality.
"""
import os
import tempfile
from unittest.mock import Mock, patch
import paramiko
import pytest
from libtisbackup.ssh import load_ssh_private_key, ssh_exec
class TestLoadSSHPrivateKey:
"""Test cases for load_ssh_private_key() function."""
def test_load_ed25519_key_success(self):
"""Test loading a valid Ed25519 key."""
with patch.object(paramiko.Ed25519Key, "from_private_key_file") as mock_ed25519:
mock_key = Mock()
mock_ed25519.return_value = mock_key
result = load_ssh_private_key("/path/to/ed25519_key")
assert result == mock_key
mock_ed25519.assert_called_once_with("/path/to/ed25519_key")
def test_load_ecdsa_key_fallback(self):
"""Test loading ECDSA key when Ed25519 fails."""
with patch.object(paramiko.Ed25519Key, "from_private_key_file") as mock_ed25519, patch.object(
paramiko.ECDSAKey, "from_private_key_file"
) as mock_ecdsa:
# Ed25519 fails, ECDSA succeeds
mock_ed25519.side_effect = paramiko.SSHException("Not Ed25519")
mock_key = Mock()
mock_ecdsa.return_value = mock_key
result = load_ssh_private_key("/path/to/ecdsa_key")
assert result == mock_key
mock_ecdsa.assert_called_once_with("/path/to/ecdsa_key")
def test_load_rsa_key_fallback(self):
"""Test loading RSA key when Ed25519 and ECDSA fail."""
with patch.object(paramiko.Ed25519Key, "from_private_key_file") as mock_ed25519, patch.object(
paramiko.ECDSAKey, "from_private_key_file"
) as mock_ecdsa, patch.object(paramiko.RSAKey, "from_private_key_file") as mock_rsa:
# Ed25519 and ECDSA fail, RSA succeeds
mock_ed25519.side_effect = paramiko.SSHException("Not Ed25519")
mock_ecdsa.side_effect = paramiko.SSHException("Not ECDSA")
mock_key = Mock()
mock_rsa.return_value = mock_key
result = load_ssh_private_key("/path/to/rsa_key")
assert result == mock_key
mock_rsa.assert_called_once_with("/path/to/rsa_key")
def test_load_key_all_formats_fail(self):
"""Test that appropriate error is raised when all key formats fail."""
with patch.object(paramiko.Ed25519Key, "from_private_key_file") as mock_ed25519, patch.object(
paramiko.ECDSAKey, "from_private_key_file"
) as mock_ecdsa, patch.object(paramiko.RSAKey, "from_private_key_file") as mock_rsa:
# All key types fail
error_msg = "Invalid key format"
mock_ed25519.side_effect = paramiko.SSHException(error_msg)
mock_ecdsa.side_effect = paramiko.SSHException(error_msg)
mock_rsa.side_effect = paramiko.SSHException(error_msg)
with pytest.raises(paramiko.SSHException) as exc_info:
load_ssh_private_key("/path/to/invalid_key")
assert "Unable to load private key" in str(exc_info.value)
assert "Ed25519 (recommended), ECDSA, RSA" in str(exc_info.value)
assert "DSA keys are no longer supported" in str(exc_info.value)
def test_load_key_with_real_ed25519_key(self):
"""Test loading a real Ed25519 private key file."""
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import ed25519
# Create a temporary Ed25519 key for testing
with tempfile.TemporaryDirectory() as tmpdir:
key_path = os.path.join(tmpdir, "test_ed25519_key")
# Generate a real Ed25519 key using cryptography library
private_key = ed25519.Ed25519PrivateKey.generate()
# Write the key in OpenSSH format (required for paramiko)
pem = private_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.OpenSSH,
encryption_algorithm=serialization.NoEncryption()
)
with open(key_path, 'wb') as f:
f.write(pem)
# Load the key with our function
loaded_key = load_ssh_private_key(key_path)
assert isinstance(loaded_key, paramiko.Ed25519Key)
def test_load_key_with_real_rsa_key(self):
"""Test loading a real RSA private key file."""
with tempfile.TemporaryDirectory() as tmpdir:
key_path = os.path.join(tmpdir, "test_rsa_key")
# Generate a real RSA key
key = paramiko.RSAKey.generate(2048)
key.write_private_key_file(key_path)
# Load the key
loaded_key = load_ssh_private_key(key_path)
assert isinstance(loaded_key, paramiko.RSAKey)
class TestSSHExec:
"""Test cases for ssh_exec() function."""
def test_ssh_exec_with_existing_connection(self):
"""Test executing command with an existing SSH connection."""
# Mock SSH client and channel
mock_ssh = Mock(spec=paramiko.SSHClient)
mock_transport = Mock()
mock_channel = Mock()
mock_stdout = Mock()
mock_ssh.get_transport.return_value = mock_transport
mock_transport.open_session.return_value = mock_channel
mock_channel.makefile.return_value = mock_stdout
mock_stdout.read.return_value = b"command output\n"
mock_channel.recv_exit_status.return_value = 0
exit_code, output = ssh_exec("ls -la", ssh=mock_ssh)
assert exit_code == 0
assert "command output" in output
mock_channel.exec_command.assert_called_once_with("ls -la")
def test_ssh_exec_creates_new_connection(self):
"""Test that ssh_exec creates a new connection when ssh parameter is None."""
with patch("libtisbackup.ssh.load_ssh_private_key") as mock_load_key, patch(
"libtisbackup.ssh.paramiko.SSHClient"
) as mock_ssh_client_class:
# Setup mocks
mock_key = Mock()
mock_load_key.return_value = mock_key
mock_ssh = Mock()
mock_ssh_client_class.return_value = mock_ssh
mock_transport = Mock()
mock_channel = Mock()
mock_stdout = Mock()
mock_ssh.get_transport.return_value = mock_transport
mock_transport.open_session.return_value = mock_channel
mock_channel.makefile.return_value = mock_stdout
mock_stdout.read.return_value = b"test output"
mock_channel.recv_exit_status.return_value = 0
# Execute
exit_code, output = ssh_exec(
command="whoami", server_name="testserver", remote_user="testuser", private_key="/path/to/key", ssh_port=22
)
# Verify
assert exit_code == 0
assert "test output" in output
mock_load_key.assert_called_once_with("/path/to/key")
mock_ssh.set_missing_host_key_policy.assert_called_once()
mock_ssh.connect.assert_called_once_with("testserver", username="testuser", pkey=mock_key, port=22)
def test_ssh_exec_with_non_zero_exit_code(self):
"""Test handling of commands that exit with non-zero status."""
mock_ssh = Mock(spec=paramiko.SSHClient)
mock_transport = Mock()
mock_channel = Mock()
mock_stdout = Mock()
mock_ssh.get_transport.return_value = mock_transport
mock_transport.open_session.return_value = mock_channel
mock_channel.makefile.return_value = mock_stdout
mock_stdout.read.return_value = b"error: command failed\n"
mock_channel.recv_exit_status.return_value = 1
exit_code, output = ssh_exec("false", ssh=mock_ssh)
assert exit_code == 1
assert "error: command failed" in output
def test_ssh_exec_with_custom_port(self):
"""Test ssh_exec with custom SSH port."""
with patch("libtisbackup.ssh.load_ssh_private_key") as mock_load_key, patch(
"libtisbackup.ssh.paramiko.SSHClient"
) as mock_ssh_client_class:
mock_key = Mock()
mock_load_key.return_value = mock_key
mock_ssh = Mock()
mock_ssh_client_class.return_value = mock_ssh
mock_transport = Mock()
mock_channel = Mock()
mock_stdout = Mock()
mock_ssh.get_transport.return_value = mock_transport
mock_transport.open_session.return_value = mock_channel
mock_channel.makefile.return_value = mock_stdout
mock_stdout.read.return_value = b"output"
mock_channel.recv_exit_status.return_value = 0
ssh_exec(command="ls", server_name="server", remote_user="user", private_key="/key", ssh_port=2222)
mock_ssh.connect.assert_called_once_with("server", username="user", pkey=mock_key, port=2222)
def test_ssh_exec_output_decoding(self):
"""Test that ssh_exec properly decodes output and handles special characters."""
mock_ssh = Mock(spec=paramiko.SSHClient)
mock_transport = Mock()
mock_channel = Mock()
mock_stdout = Mock()
mock_ssh.get_transport.return_value = mock_transport
mock_transport.open_session.return_value = mock_channel
mock_channel.makefile.return_value = mock_stdout
# Output with single quotes that should be removed
mock_stdout.read.return_value = b"output with 'quotes' included"
mock_channel.recv_exit_status.return_value = 0
exit_code, output = ssh_exec("echo test", ssh=mock_ssh)
assert exit_code == 0
# ssh_exec removes single quotes from output
assert "output with quotes included" == output
def test_ssh_exec_empty_output(self):
"""Test handling of commands with no output."""
mock_ssh = Mock(spec=paramiko.SSHClient)
mock_transport = Mock()
mock_channel = Mock()
mock_stdout = Mock()
mock_ssh.get_transport.return_value = mock_transport
mock_transport.open_session.return_value = mock_channel
mock_channel.makefile.return_value = mock_stdout
mock_stdout.read.return_value = b""
mock_channel.recv_exit_status.return_value = 0
exit_code, output = ssh_exec("true", ssh=mock_ssh)
assert exit_code == 0
assert output == ""
def test_ssh_exec_requires_connection_params(self):
"""Test that ssh_exec requires connection parameters when ssh is None."""
# This should raise an assertion error because we don't provide ssh connection
# and don't provide the required parameters
with pytest.raises(AssertionError):
ssh_exec(command="ls")
class TestSSHModuleIntegration:
"""Integration tests for SSH module functionality."""
def test_load_and_use_key_in_connection(self):
"""Test the flow of loading a key and using it in ssh_exec."""
with tempfile.TemporaryDirectory() as tmpdir:
key_path = os.path.join(tmpdir, "test_key")
# Generate a real RSA key (more compatible across paramiko versions)
key = paramiko.RSAKey.generate(2048)
key.write_private_key_file(key_path)
# Mock the SSH connection part
with patch("libtisbackup.ssh.paramiko.SSHClient") as mock_ssh_client_class:
mock_ssh = Mock()
mock_ssh_client_class.return_value = mock_ssh
mock_transport = Mock()
mock_channel = Mock()
mock_stdout = Mock()
mock_ssh.get_transport.return_value = mock_transport
mock_transport.open_session.return_value = mock_channel
mock_channel.makefile.return_value = mock_stdout
mock_stdout.read.return_value = b"success"
mock_channel.recv_exit_status.return_value = 0
# Execute with real key file
exit_code, output = ssh_exec(
command="echo hello", server_name="localhost", remote_user="testuser", private_key=key_path, ssh_port=22
)
assert exit_code == 0
assert output == "success"
# Verify that connect was called with a real RSAKey
connect_call = mock_ssh.connect.call_args
assert connect_call[1]["username"] == "testuser"
assert isinstance(connect_call[1]["pkey"], paramiko.RSAKey)

471
tests/test_utils.py
View File

@ -1,471 +0,0 @@
#!/usr/bin/python3
# -*- coding: utf-8 -*-
# -----------------------------------------------------------------------
# This file is part of TISBackup
#
# TISBackup is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# TISBackup is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with TISBackup. If not, see <http://www.gnu.org/licenses/>.
#
# -----------------------------------------------------------------------
"""
Test suite for libtisbackup.utils module.
Tests utility functions for date/time formatting, number formatting, and display helpers.
"""
import datetime
import os
import tempfile
from unittest.mock import Mock
import pytest
from libtisbackup.utils import (
check_string,
convert_bytes,
dateof,
datetime2isodate,
fileisodate,
hours_minutes,
html_table,
isodate2datetime,
pp,
splitThousands,
str2bool,
time2display,
)
class TestDateTimeFunctions:
"""Test cases for date/time utility functions."""
def test_datetime2isodate_with_datetime(self):
"""Test converting a datetime to ISO format."""
dt = datetime.datetime(2025, 10, 5, 14, 30, 45, 123456)
result = datetime2isodate(dt)
assert result == "2025-10-05T14:30:45.123456"
def test_datetime2isodate_without_datetime(self):
"""Test converting current datetime to ISO format."""
result = datetime2isodate()
# Should return a valid ISO format string
assert "T" in result
assert len(result) >= 19 # At least YYYY-MM-DDTHH:MM:SS
def test_datetime2isodate_with_none(self):
"""Test that None triggers default datetime.now() behavior."""
result = datetime2isodate(None)
assert isinstance(result, str)
assert "T" in result
def test_isodate2datetime_basic(self):
"""Test converting ISO date string to datetime."""
iso_str = "2025-10-05T14:30:45"
result = isodate2datetime(iso_str)
assert result == datetime.datetime(2025, 10, 5, 14, 30, 45)
def test_isodate2datetime_with_microseconds(self):
"""Test that microseconds are stripped during conversion."""
iso_str = "2025-10-05T14:30:45.123456"
result = isodate2datetime(iso_str)
# Microseconds should be ignored
assert result == datetime.datetime(2025, 10, 5, 14, 30, 45)
def test_isodate2datetime_roundtrip(self):
"""Test roundtrip conversion datetime -> ISO -> datetime."""
original = datetime.datetime(2025, 10, 5, 14, 30, 45)
iso_str = datetime2isodate(original)
result = isodate2datetime(iso_str)
assert result == original
def test_time2display(self):
"""Test formatting datetime for display."""
dt = datetime.datetime(2025, 10, 5, 14, 30, 45)
result = time2display(dt)
assert result == "2025-10-05 14:30"
def test_time2display_different_times(self):
"""Test time2display with various datetime values."""
test_cases = [
(datetime.datetime(2025, 1, 1, 0, 0, 0), "2025-01-01 00:00"),
(datetime.datetime(2025, 12, 31, 23, 59, 59), "2025-12-31 23:59"),
(datetime.datetime(2025, 6, 15, 12, 30, 45), "2025-06-15 12:30"),
]
for dt, expected in test_cases:
assert time2display(dt) == expected
def test_dateof(self):
"""Test getting date part of datetime (midnight)."""
dt = datetime.datetime(2025, 10, 5, 14, 30, 45, 123456)
result = dateof(dt)
assert result == datetime.datetime(2025, 10, 5, 0, 0, 0, 0)
def test_dateof_already_midnight(self):
"""Test dateof with a datetime already at midnight."""
dt = datetime.datetime(2025, 10, 5, 0, 0, 0, 0)
result = dateof(dt)
assert result == dt
def test_fileisodate(self):
"""Test getting file modification time as ISO date."""
with tempfile.NamedTemporaryFile(delete=False) as tmp:
tmp_path = tmp.name
tmp.write(b"test content")
try:
result = fileisodate(tmp_path)
# Should return a valid ISO format string
assert "T" in result
# Verify it's a parseable datetime
parsed = isodate2datetime(result)
assert isinstance(parsed, datetime.datetime)
finally:
os.unlink(tmp_path)
class TestHoursMinutes:
"""Test cases for hours_minutes function."""
def test_hours_minutes_whole_hours(self):
"""Test converting whole hours."""
assert hours_minutes(1.0) == "01:00"
assert hours_minutes(5.0) == "05:00"
assert hours_minutes(10.0) == "10:00"
def test_hours_minutes_with_minutes(self):
"""Test converting hours with minutes."""
assert hours_minutes(1.5) == "01:30"
assert hours_minutes(2.25) == "02:15"
assert hours_minutes(3.75) == "03:45"
def test_hours_minutes_less_than_one_hour(self):
"""Test converting less than one hour."""
assert hours_minutes(0.5) == "00:30"
assert hours_minutes(0.25) == "00:15"
assert hours_minutes(0.75) == "00:45"
def test_hours_minutes_zero(self):
"""Test converting zero hours."""
assert hours_minutes(0) == "00:00"
def test_hours_minutes_none(self):
"""Test that None returns None."""
assert hours_minutes(None) is None
def test_hours_minutes_large_values(self):
"""Test converting large hour values."""
assert hours_minutes(24.0) == "24:00"
assert hours_minutes(100.5) == "100:30"
class TestSplitThousands:
"""Test cases for splitThousands function."""
def test_splitThousands_integer(self):
"""Test formatting integer numbers."""
assert splitThousands("1000") == "1,000"
assert splitThousands("1000000") == "1,000,000"
assert splitThousands("123456789") == "123,456,789"
def test_splitThousands_float(self):
"""Test formatting float numbers."""
assert splitThousands("1000.50") == "1,000.50"
assert splitThousands("1234567.89") == "1,234,567.89"
def test_splitThousands_number_types(self):
"""Test that numeric types are converted to string."""
assert splitThousands(1000) == "1,000"
assert splitThousands(1000000) == "1,000,000"
def test_splitThousands_none(self):
"""Test that None returns 0."""
assert splitThousands(None) == 0
def test_splitThousands_small_numbers(self):
"""Test numbers that don't need separators."""
assert splitThousands("100") == "100"
assert splitThousands("999") == "999"
def test_splitThousands_custom_separators(self):
"""Test with custom thousand and decimal separators."""
assert splitThousands("1000.50", tSep=" ", dSep=".") == "1 000.50"
assert splitThousands("1000,50", tSep=".", dSep=",") == "1.000,50"
def test_splitThousands_with_leading_characters(self):
"""Test numbers with leading characters."""
assert splitThousands("+1000") == "+1,000"
assert splitThousands("-1000000") == "-1,000,000"
class TestConvertBytes:
"""Test cases for convert_bytes function."""
def test_convert_bytes_none(self):
"""Test that None returns None."""
assert convert_bytes(None) is None
def test_convert_bytes_bytes(self):
"""Test converting byte values."""
assert convert_bytes(0) == "0.00b"
assert convert_bytes(500) == "500.00b"
assert convert_bytes(1023) == "1023.00b"
def test_convert_bytes_kilobytes(self):
"""Test converting to kilobytes."""
assert convert_bytes(1024) == "1.00K"
assert convert_bytes(1024 * 5) == "5.00K"
assert convert_bytes(1024 * 100) == "100.00K"
def test_convert_bytes_megabytes(self):
"""Test converting to megabytes."""
assert convert_bytes(1048576) == "1.00M"
assert convert_bytes(1048576 * 10) == "10.00M"
assert convert_bytes(1048576 * 500) == "500.00M"
def test_convert_bytes_gigabytes(self):
"""Test converting to gigabytes."""
assert convert_bytes(1073741824) == "1.00G"
assert convert_bytes(1073741824 * 5) == "5.00G"
assert convert_bytes(1073741824 * 100) == "100.00G"
def test_convert_bytes_terabytes(self):
"""Test converting to terabytes."""
assert convert_bytes(1099511627776) == "1.00T"
assert convert_bytes(1099511627776 * 2) == "2.00T"
assert convert_bytes(1099511627776 * 10) == "10.00T"
def test_convert_bytes_string_input(self):
"""Test that string numbers are converted to float."""
assert convert_bytes("1024") == "1.00K"
assert convert_bytes("1048576") == "1.00M"
class TestCheckString:
"""Test cases for check_string function."""
def test_check_string_valid(self):
"""Test valid strings (alphanumeric, dots, dashes, underscores)."""
# These should not print anything
check_string("valid_string")
check_string("valid-string")
check_string("valid.string")
check_string("ValidString123")
def test_check_string_invalid(self, capsys):
"""Test invalid strings print error message."""
check_string("invalid string with spaces")
captured = capsys.readouterr()
assert "Invalid" in captured.out
assert "invalid string with spaces" in captured.out
def test_check_string_special_characters(self, capsys):
"""Test strings with special characters."""
check_string("invalid@string")
captured = capsys.readouterr()
assert "Invalid" in captured.out
class TestStr2Bool:
"""Test cases for str2bool function."""
def test_str2bool_true_values(self):
"""Test strings that should convert to True."""
assert str2bool("yes") is True
assert str2bool("YES") is True
assert str2bool("true") is True
assert str2bool("TRUE") is True
assert str2bool("t") is True
assert str2bool("T") is True
assert str2bool("1") is True
def test_str2bool_false_values(self):
"""Test strings that should convert to False."""
assert str2bool("no") is False
assert str2bool("NO") is False
assert str2bool("false") is False
assert str2bool("FALSE") is False
assert str2bool("f") is False
assert str2bool("F") is False
assert str2bool("0") is False
def test_str2bool_mixed_case(self):
"""Test mixed case strings."""
assert str2bool("Yes") is True
assert str2bool("True") is True
assert str2bool("No") is False
assert str2bool("False") is False
class TestPrettyPrint:
"""Test cases for pp (pretty print) function."""
def test_pp_basic(self):
"""Test basic pretty printing of cursor results."""
# Mock cursor
mock_cursor = Mock()
mock_cursor.description = [("id", 10), ("name", 20)]
mock_cursor.fetchall.return_value = [(1, "Alice"), (2, "Bob")]
result = pp(mock_cursor)
assert "id" in result
assert "name" in result
assert "Alice" in result
assert "Bob" in result
assert "---" in result # Should have separator line
def test_pp_no_description(self):
"""Test pp with no cursor description."""
mock_cursor = Mock()
mock_cursor.description = None
result = pp(mock_cursor)
assert result == "#### NO RESULTS ###"
def test_pp_with_callback(self):
"""Test pp with custom callback for formatting."""
mock_cursor = Mock()
mock_cursor.description = [("count", 10)]
mock_cursor.fetchall.return_value = [(1000,), (2000,)]
def format_callback(fieldname, value):
if fieldname == "count":
return str(value * 2)
return value
result = pp(mock_cursor, callback=format_callback)
assert "2000" in result # 1000 * 2
assert "4000" in result # 2000 * 2
def test_pp_with_provided_data(self):
"""Test pp with data provided instead of fetching."""
mock_cursor = Mock()
mock_cursor.description = [("id", 10), ("value", 20)]
data = [(1, "test1"), (2, "test2")]
result = pp(mock_cursor, data=data)
assert "test1" in result
assert "test2" in result
# fetchall should not be called
mock_cursor.fetchall.assert_not_called()
class TestHtmlTable:
"""Test cases for html_table function."""
def test_html_table_basic(self):
"""Test basic HTML table generation."""
mock_cursor = Mock()
mock_cursor.description = [("id",), ("name",)]
mock_cursor.__iter__ = Mock(return_value=iter([("1", "Alice"), ("2", "Bob")]))
result = html_table(mock_cursor)
assert "<table" in result
assert "<tr>" in result
assert "<th>id</th>" in result
assert "<th>name</th>" in result
assert "<td>1</td>" in result
assert "<td>Alice</td>" in result
assert "<td>2</td>" in result
assert "<td>Bob</td>" in result
def test_html_table_with_callback(self):
"""Test HTML table with custom formatting callback."""
mock_cursor = Mock()
mock_cursor.description = [("count",)]
# Create an iterator that yields tuples (for non-callback path)
mock_cursor.__iter__ = Mock(return_value=iter([("1000",), ("2000",)]))
result = html_table(mock_cursor)
assert "<table" in result
assert "1000" in result
assert "2000" in result
def test_html_table_with_none_values(self):
"""Test HTML table handles None values."""
mock_cursor = Mock()
mock_cursor.description = [("id",), ("value",)]
# Use empty string instead of None to avoid TypeError
mock_cursor.__iter__ = Mock(return_value=iter([("1", ""), ("2", "test")]))
result = html_table(mock_cursor)
assert "<table" in result
assert "<td>1</td>" in result
assert "<td>test</td>" in result
def test_html_table_structure(self):
"""Test that HTML table has proper structure."""
mock_cursor = Mock()
mock_cursor.description = [("col1",)]
mock_cursor.__iter__ = Mock(return_value=iter([("1",)]))
result = html_table(mock_cursor)
# Should have table tag with attributes
assert result.startswith("<table border=1")
assert result.endswith("</table>")
assert "cellpadding=2" in result
assert "cellspacing=0" in result
class TestUtilsIntegration:
"""Integration tests for utilities working together."""
def test_datetime_conversion_chain(self):
"""Test complete datetime conversion workflow."""
# Create a datetime
original = datetime.datetime(2025, 10, 5, 14, 30, 45)
# Convert to ISO
iso_str = datetime2isodate(original)
# Convert back
restored = isodate2datetime(iso_str)
# Display format
display = time2display(restored)
# Get date only
date_only = dateof(restored)
assert restored == original
assert display == "2025-10-05 14:30"
assert date_only == datetime.datetime(2025, 10, 5, 0, 0, 0, 0)
def test_number_formatting_chain(self):
"""Test number formatting utilities together."""
# Convert bytes to human readable
bytes_val = 1073741824 # 1 GB
readable = convert_bytes(bytes_val)
assert readable == "1.00G"
# Format with thousands separator
large_num = 1234567
formatted = splitThousands(large_num)
assert formatted == "1,234,567"
def test_time_duration_formatting(self):
"""Test formatting time durations."""
# Different durations in hours
durations = [0.5, 1.25, 2.75, 10.5]
expected = ["00:30", "01:15", "02:45", "10:30"]
for duration, expected_format in zip(durations, expected):
assert hours_minutes(duration) == expected_format

20
tisbackup.py
View File

@ -34,9 +34,23 @@ from optparse import OptionParser
from iniparse import ConfigParser, ini
# Import all backup drivers - this registers them with the driver registry
from libtisbackup.drivers import *
from libtisbackup import *
from libtisbackup.backup_mysql import backup_mysql
# from libtisbackup.backup_vmdk import backup_vmdk
# from libtisbackup.backup_switch import backup_switch
from libtisbackup.backup_null import backup_null
from libtisbackup.backup_pgsql import backup_pgsql
from libtisbackup.backup_rsync import backup_rsync, backup_rsync_ssh
# from libtisbackup.backup_oracle import backup_oracle
from libtisbackup.backup_rsync_btrfs import backup_rsync__btrfs_ssh, backup_rsync_btrfs
# from libtisbackup.backup_sqlserver import backup_sqlserver
from libtisbackup.backup_samba4 import backup_samba4
from libtisbackup.backup_xcp_metadata import backup_xcp_metadata
from libtisbackup.backup_xva import backup_xva
from libtisbackup.common import *
from libtisbackup.copy_vm_xcp import copy_vm_xcp
__version__ = "2.0"

229
tisbackup_gui.py
View File

@ -30,17 +30,15 @@ import glob
import json
import logging
import re
import shutil
import subprocess
import time
from shutil import *
from urllib.parse import urlparse
from flask import Flask, Response, abort, appcontext_pushed, flash, g, jsonify, redirect, render_template, request, session, url_for
from iniparse import ConfigParser, RawConfigParser
from config import huey
from libtisbackup import *
from libtisbackup.auth import get_auth_provider
from libtisbackup.common import *
from tasks import get_task, run_export_backup, set_task
from tisbackup import tis_backup
@ -63,86 +61,9 @@ mindate = None
error = None
info = None
app = Flask(__name__)
# Load secret key from environment variable or generate a secure random one
SECRET_KEY = os.environ.get("TISBACKUP_SECRET_KEY")
if not SECRET_KEY:
# Generate a secure random secret key if not provided
import secrets
SECRET_KEY = secrets.token_hex(32)
# Warn if using a random key (sessions won't persist across restarts)
logging.warning(
"TISBACKUP_SECRET_KEY environment variable not set. Using a randomly generated secret key. "
"Sessions will not persist across application restarts. "
"Set TISBACKUP_SECRET_KEY environment variable for production use."
)
app.secret_key = SECRET_KEY
app.secret_key = "fsiqefiuqsefARZ4Zfesfe34234dfzefzfe"
app.config["PROPAGATE_EXCEPTIONS"] = True
# Initialize authentication
auth_config = {}
try:
# Read authentication config from tisbackup_gui.ini
cp_gui = ConfigParser()
cp_gui.read("/etc/tis/tisbackup_gui.ini")
if cp_gui.has_section("authentication"):
auth_type = cp_gui.get("authentication", "type", fallback="basic")
# Load auth provider config
for key, value in cp_gui.items("authentication"):
if key != "type":
auth_config[key] = value
else:
# Default to Basic Auth if no config section
auth_type = "basic"
# Get credentials from environment or use defaults
default_username = os.environ.get("TISBACKUP_AUTH_USERNAME", "admin")
default_password = os.environ.get("TISBACKUP_AUTH_PASSWORD")
if not default_password:
# Generate random password if not set
import secrets
default_password = secrets.token_urlsafe(16)
logging.warning(
f"TISBACKUP_AUTH_PASSWORD not set. Generated temporary password for user '{default_username}': {default_password}"
)
logging.warning(
"Set TISBACKUP_AUTH_USERNAME and TISBACKUP_AUTH_PASSWORD environment variables, "
"or add [authentication] section to tisbackup_gui.ini for production use."
)
auth_config = {
"username": default_username,
"password": default_password,
"use_bcrypt": False, # Plain text for auto-generated password
"realm": "TISBackup"
}
except Exception as e:
# Fallback to basic auth on error
logging.error(f"Error loading authentication config: {e}. Using default Basic Auth.")
auth_type = "basic"
auth_config = {
"username": os.environ.get("TISBACKUP_AUTH_USERNAME", "admin"),
"password": os.environ.get("TISBACKUP_AUTH_PASSWORD", "changeme"),
"use_bcrypt": False,
"realm": "TISBackup"
}
# Initialize auth provider
try:
auth = get_auth_provider(auth_type, auth_config)
auth.init_app(app)
logging.info(f"Authentication initialized: {auth_type}")
except Exception as e:
logging.error(f"Failed to initialize authentication: {e}")
# Fallback to no auth
auth = get_auth_provider("none", {})
logging.warning("Authentication disabled due to initialization error")
tasks_db = os.path.join(tisbackup_root_dir, "tasks.sqlite")
@ -156,10 +77,9 @@ def read_all_configs(base_dir):
raw_configs.append(join(base_dir, file))
for elem in raw_configs:
with open(elem) as f:
line = f.readline()
if "global" in line:
list_config.append(elem)
line = open(elem).readline()
if "global" in line:
list_config.append(elem)
backup_dict = {}
backup_dict["rsync_ssh_list"] = []
@ -327,7 +247,6 @@ def read_config():
@app.route("/")
@auth.require_auth
def backup_all():
backup_dict = read_config()
return render_template("backups.html", backup_list=backup_dict)
@ -335,7 +254,6 @@ def backup_all():
@app.route("/config_number/")
@app.route("/config_number/<int:id>")
@auth.require_auth
def set_config_number(id=None):
if id is not None and len(CONFIG) > id:
global config_number
@ -345,7 +263,6 @@ def set_config_number(id=None):
@app.route("/all_json")
@auth.require_auth
def backup_all_json():
backup_dict = read_all_configs(BASE_DIR)
return json.dumps(
@ -362,7 +279,6 @@ def backup_all_json():
@app.route("/json")
@auth.require_auth
def backup_json():
backup_dict = read_config()
return json.dumps(
@ -380,26 +296,12 @@ def backup_json():
def check_usb_disk():
"""This method returns the mounts point of FIRST external disk"""
# disk_name = []
usb_disk_list = []
for name in glob.glob("/dev/sd[a-z]"):
# Validate device name to prevent command injection
if not re.match(r"^/dev/sd[a-z]$", name):
continue
try:
result = subprocess.run(
["udevadm", "info", "-q", "env", "-n", name],
capture_output=True,
text=True,
check=True,
timeout=5
)
for line in result.stdout.splitlines():
if re.match("ID_PATH=.*usb.*", line):
usb_disk_list.append(name)
break
except (subprocess.CalledProcessError, subprocess.TimeoutExpired):
continue
for line in os.popen("udevadm info -q env -n %s" % name):
if re.match("ID_PATH=.*usb.*", line):
usb_disk_list += [name]
if len(usb_disk_list) == 0:
raise_error("Cannot find any external usb disk", "You should plug the usb hard drive into the server")
@ -408,27 +310,14 @@ def check_usb_disk():
usb_partition_list = []
for usb_disk in usb_disk_list:
partition = usb_disk + "1"
# Validate partition name
if not re.match(r"^/dev/sd[a-z]1$", partition):
continue
cmd = "udevadm info -q path -n %s" % usb_disk + "1"
output = os.popen(cmd).read()
print("cmd : " + cmd)
print("output : " + output)
try:
result = subprocess.run(
["udevadm", "info", "-q", "path", "-n", partition],
capture_output=True,
text=True,
check=True,
timeout=5
)
output = result.stdout
print("partition check: " + partition)
print("output : " + output)
if "/devices/pci" in output:
usb_partition_list.append(partition)
except (subprocess.CalledProcessError, subprocess.TimeoutExpired):
continue
if "/devices/pci" in output:
# flash("partition found: %s1" % usb_disk)
usb_partition_list.append(usb_disk + "1")
print(usb_partition_list)
@ -441,22 +330,9 @@ def check_usb_disk():
tisbackup_partition_list = []
for usb_partition in usb_partition_list:
# Validate partition name to prevent command injection
if not re.match(r"^/dev/sd[a-z]1$", usb_partition):
continue
try:
result = subprocess.run(
["/sbin/dumpe2fs", "-h", usb_partition],
capture_output=True,
text=True,
timeout=5
)
if "tisbackup" in result.stdout.lower():
flash("tisbackup backup partition found: %s" % usb_partition)
tisbackup_partition_list.append(usb_partition)
except (subprocess.CalledProcessError, subprocess.TimeoutExpired):
continue
if "tisbackup" in os.popen("/sbin/dumpe2fs -h %s 2>&1 |/bin/grep 'volume name'" % usb_partition).read().lower():
flash("tisbackup backup partition found: %s" % usb_partition)
tisbackup_partition_list.append(usb_partition)
print(tisbackup_partition_list)
@ -475,64 +351,27 @@ def check_usb_disk():
def check_already_mount(partition_name, refresh):
# Validate partition name to prevent path traversal
if not re.match(r"^/dev/[a-z0-9]+$", partition_name):
raise_error("Invalid partition name", "Partition name contains invalid characters")
return ""
with open("/proc/mounts") as f:
mount_point = ""
for line in f.readlines():
if line.startswith(partition_name):
mount_point = line.split(" ")[1]
if not refresh and mount_point:
try:
subprocess.run(["/bin/umount", mount_point], check=True, timeout=30)
os.rmdir(mount_point)
except (subprocess.CalledProcessError, subprocess.TimeoutExpired, OSError) as e:
raise_error(f"Failed to unmount {mount_point}", str(e))
if not refresh:
run_command("/bin/umount %s" % mount_point)
os.rmdir(mount_point)
return mount_point
def run_command(cmd_list, info=""):
"""Execute a command safely using subprocess.run with list arguments.
def run_command(cmd, info=""):
flash("Executing: %s" % cmd)
from subprocess import CalledProcessError, check_output
Args:
cmd_list: List of command arguments (or string for backward compatibility)
info: Additional info message on error
"""
# Handle legacy string commands by converting to list
if isinstance(cmd_list, str):
flash(f"Executing (legacy): {cmd_list}")
# This should be refactored - shell=True is unsafe
try:
result = subprocess.run(
cmd_list,
capture_output=True,
text=True,
shell=True,
timeout=30
)
if result.returncode != 0:
raise_error(result.stderr or result.stdout, info)
return result.stdout
except subprocess.TimeoutExpired:
raise_error("Command timeout", info)
else:
flash(f"Executing: {' '.join(cmd_list)}")
try:
result = subprocess.run(
cmd_list,
capture_output=True,
text=True,
check=True,
timeout=30
)
return result.stdout
except subprocess.CalledProcessError as e:
raise_error(e.stderr or e.stdout, info)
except subprocess.TimeoutExpired:
raise_error("Command timeout", info)
result = ""
try:
result = check_output(cmd, stderr=subprocess.STDOUT, shell=True)
except CalledProcessError:
raise_error(result, info)
return result
def check_mount_disk(partition_name, refresh):
@ -551,7 +390,6 @@ def check_mount_disk(partition_name, refresh):
@app.route("/status.json")
@auth.require_auth
def export_backup_status():
exports = dbstat.query('select * from stats where TYPE="EXPORT" and backup_start>="%s"' % mindate)
error = ""
@ -572,21 +410,18 @@ def runnings_backups():
@app.route("/backups.json")
@auth.require_auth
def last_backup_json():
exports = dbstat.query('select * from stats where TYPE="BACKUP" ORDER BY backup_start DESC ')
return Response(response=json.dumps(exports), status=200, mimetype="application/json")
@app.route("/last_backups")
@auth.require_auth
def last_backup():
exports = dbstat.query('select * from stats where TYPE="BACKUP" ORDER BY backup_start DESC LIMIT 20 ')
return render_template("last_backups.html", backups=exports)
@app.route("/export_backup")
@auth.require_auth
def export_backup():
raise_error("", "")
backup_dict = read_config()

875
uv.lock generated
View File

File diff suppressed because it is too large Load Diff