docs: update README with comprehensive feature list, installation guide, and architectural documentation
This commit is contained in:
Rizqi
parent
f6f6158811
commit
63ddc74f7e
105
README.md
105
README.md
@ -1,37 +1,68 @@
|
||||
# vSphere Backup Manager
|
||||
|
||||
A web interface and CLI tool to automate, schedule, and manage snapshot-based backups for virtual machines on VMware vCenter/ESXi.
|
||||
An enterprise-ready web interface and CLI tool to automate, schedule, and manage snapshot-based backups for virtual machines on VMware vCenter/ESXi. Designed for performance, reliability, and security, it includes advanced features such as Change-based checksumming, automated retention policies, and grouped batch executions.
|
||||
|
||||
---
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Grouped Sequential Batch Backups**: Select multiple VMs to execute sequentially in a single job. This protects vCenter/ESXi storage datastores from network I/O congestion and merges execution logs and progress indicators into a single view.
|
||||
- **SHA-256 Checksum Verification & Cataloging**: Computes SHA-256 signatures immediately after each VMDK/VMX file download and generates a machine-readable `manifest.json` catalog alongside each backup run.
|
||||
- **Pre-Upload Validation**: Automatically validates local checksums prior to remote transfers (e.g., SFTP) to protect storage vaults against silent write errors or network package loss.
|
||||
- **On-the-Fly ZST Verification**: Supports stream-decompression on the fly to verify `.zst` archives against original manifest signatures without needing local disk extraction.
|
||||
- **Safe Force Stop (Cancellation)**: Safely halt running backups via the Web UI. The engine immediately aborts socket downloads and **automatically cleans up the VM snapshot** on the ESXi host before gracefully terminating.
|
||||
- **Automated Retention Policies**: Define count-based (`keep_count` to keep the last $N$ backups) or age-based (`keep_days` to clean up backups older than $N$ days) retention policies per VM to manage storage space automatically.
|
||||
- **Resilient Scheduling**: Uses APScheduler to schedule daily, weekly, monthly (with specific weekday or day number rules), or interval backups. Schedules are written to disk (`jobs.json`) and automatically re-registered upon app restarts.
|
||||
- **Integrated NFS Mount Manager**: View, mount, and manage NFS/CIFS shares directly from the Web GUI, showing real-time mount statuses, total size, used capacity, and free disk space.
|
||||
|
||||
---
|
||||
|
||||
## Requirements
|
||||
|
||||
Requirements
|
||||
- Python 3.8+
|
||||
- See `requirements.txt` (pyvmomi, requests, paramiko, zstandard)
|
||||
- System packages listed in `requirements.txt`:
|
||||
- `pyvmomi` (VMware vSphere API Python SDK)
|
||||
- `requests` (vCenter HTTPS folder API transfers)
|
||||
- `paramiko` (SFTP remote storage replication)
|
||||
- `zstandard` (High-ratio backup compression)
|
||||
- `APScheduler` (Recurring backups scheduling)
|
||||
|
||||
Basic usage
|
||||
---
|
||||
|
||||
```bash
|
||||
python vsphere_backup.py --host vc.example.local --user administrator@vsphere.local --vm MyVM --dest /backups/MyVM --compress
|
||||
```
|
||||
## Installation
|
||||
|
||||
Optional SFTP upload
|
||||
1. **Clone the repository**:
|
||||
```bash
|
||||
git clone <repository_url>
|
||||
cd backupvmware
|
||||
```
|
||||
|
||||
```bash
|
||||
python vsphere_backup.py --host vc.example.local --user admin --vm MyVM --dest /tmp/backups --sftp-host backup.example.com --sftp-user backup --sftp-password secret
|
||||
```
|
||||
2. **Set up a Python Virtual Environment**:
|
||||
- **Linux**:
|
||||
```bash
|
||||
python3 -m venv venv
|
||||
source venv/bin/activate
|
||||
```
|
||||
- **Windows**:
|
||||
```powershell
|
||||
python -m venv venv
|
||||
.\venv\Scripts\Activate.ps1
|
||||
```
|
||||
|
||||
Notes & caveats
|
||||
- The script creates a snapshot on the VM and downloads the VM's `.vmdk` and `.vmx` files from the datastore while the snapshot exists — do NOT copy `.vmdk` without snapshot.
|
||||
- The script attempts to use `zstd -19` if available; otherwise it falls back to Python `zstandard`.
|
||||
- SSL verification is disabled with `--no-verify-ssl` for convenience with self-signed vCenter/ESXi certs.
|
||||
- Test carefully in dev before using in production. This is a minimal DIY backup tool and does not replace a full backup product.
|
||||
3. **Install dependencies**:
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## Web GUI
|
||||
---
|
||||
|
||||
A Flask-based web interface to manage your backups, schedules, and NFS mounts.
|
||||
## Web GUI Setup
|
||||
|
||||
### Running with PM2 (Recommended for Linux production)
|
||||
A Flask-based web interface utilizing a premium glassmorphic dark theme to manage backups, schedules, mounts, and real-time logs.
|
||||
|
||||
PM2 natively supports Python applications and can keep the server running across restarts.
|
||||
### Running with PM2 (Recommended for Production)
|
||||
|
||||
PM2 natively supports Python applications and keeps the server running across restarts or process crashes.
|
||||
|
||||
1. **Install PM2** (requires Node.js):
|
||||
```bash
|
||||
@ -44,16 +75,38 @@ PM2 natively supports Python applications and can keep the server running across
|
||||
pm2 start ecosystem.config.js
|
||||
```
|
||||
|
||||
*(Optional)* If you are running inside a Python virtual environment (e.g. `venv`), edit `ecosystem.config.js` to uncomment and point the `interpreter` to your venv's python executable:
|
||||
*(Optional)* If you are running inside a Python virtual environment (e.g. `venv`), edit `ecosystem.config.js` to point the `interpreter` to your venv's python executable:
|
||||
```javascript
|
||||
interpreter: './venv/bin/python3'
|
||||
```
|
||||
|
||||
3. **Useful PM2 Commands**:
|
||||
- Status: `pm2 status`
|
||||
- Logs: `pm2 logs vsphere-backup-manager`
|
||||
- Restart: `pm2 restart vsphere-backup-manager`
|
||||
- Stop: `pm2 stop vsphere-backup-manager`
|
||||
- Setup auto-start on server boot: `pm2 startup` and then run the command it outputs, followed by `pm2 save`.
|
||||
- **Status Dashboard**: `pm2 status`
|
||||
- **Real-time Console Logs**: `pm2 logs vsphere-backup-manager`
|
||||
- **Restart Application**: `pm2 restart vsphere-backup-manager`
|
||||
- **Stop Application**: `pm2 stop vsphere-backup-manager`
|
||||
- **Enable Auto-start on Boot**: Run `pm2 startup` and execute the command it prints, followed by `pm2 save`.
|
||||
|
||||
---
|
||||
|
||||
## CLI Usage
|
||||
|
||||
You can also execute standalone backups directly from the command line:
|
||||
|
||||
### Basic Backup
|
||||
```bash
|
||||
python vsphere_backup.py --host vc.example.com --user administrator@vsphere.local --vm MyVM --dest /mnt/nfs-backup --compress
|
||||
```
|
||||
|
||||
### Backup with Remote SFTP Replication
|
||||
```bash
|
||||
python vsphere_backup.py --host vc.example.com --user administrator@vsphere.local --vm MyVM --dest /tmp/backups --sftp-host backup-vault.local --sftp-user vault-user --sftp-password vault-pass
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Safety & Architecture
|
||||
|
||||
1. **Snapshot Isolation**: The backup engine creates a temporary snapshot on the target VM, downloads the locked base files (such as `.vmdk` descriptors, `-flat.vmdk` disk data, and `.vmx` configurations) directly from the Datastore HTTP gateway, and deletes the snapshot immediately afterwards.
|
||||
2. **SSL Configuration**: Custom certificate verification options (`--no-verify-ssl` or Web checkbox) allow connecting to environments using self-signed vCenter certificates.
|
||||
3. **Database Integrity**: Job records, statuses, and scheduling data are written safely using thread-safe synchronization locks to prevent state corruption.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user