Aegis

Aegis is a CLI tool for archiving files in a Merkle DAG across various storage backends. It's designed for archiving large files to multiple backup locations while preventing silent data loss through explicit review and snapshot workflows.

Core Concepts

Merkle DAG Storage

Aegis stores your files in a content-addressed Merkle DAG (Directed Acyclic Graph). Every file and directory is identified by its SHA-256 checksum, which provides:

• Deduplication: Identical files are stored only once, even across different snapshots
• Integrity verification: Any corruption can be detected by comparing checksums
• Efficient storage: Only changed files need to be uploaded when creating new snapshots

Backends

A backend is where Aegis stores your archived data. Backends can be:

• Storage backends: Local filesystem, Backblaze B2, SFTP servers
• Transform backends: Encryption (wraps another backend)
• Multiplexing backends: Tee (writes to multiple backends simultaneously)

Backends can be composed together. For example, you can store your data unencrypted on your local filesystem, but also have encrypted backups on B2 and an SFTP server:

      ┌─────┐
      │ tee │
      └─┬─┬─┘
        │ │
┌────┐  │ │  ┌───────┐
│ fs │◄─┘ └─►│encrypt│
└────┘       └───┬───┘
                 │
                 ▼
              ┌─────┐
              │ tee │
              └─┬─┬─┘
                │ │
        ┌────┐  │ │  ┌──────┐
        │ b2 │◄─┘ └─►│ sftp │
        └────┘       └──────┘

Snapshots

A snapshot is a point-in-time record of your entire directory tree. Each snapshot contains:

• A timestamp of when it was created
• A reference to the root directory checksum
• A blob namespace (used by transform backends to prevent collisions when storing a transformed and non-transformed blobs in the same storage backend)

Explicit Review Workflow

Unlike tools that silently sync changes, Aegis requires you to explicitly review and confirm changes before they're archived. This prevents accidental data loss from inadvertent deletions or modifications.

Quick Start

Create a Repository

Create an aegis.toml file in the root directory you want to archive:

name = "photos"
default_backend = "local"

[backends.local]
type = "fs"
dir = "/path/to/local/backup"

This configures a repository named "photos", and a filesystem backend that stores data in /path/to/local/backup.

The name uniquely identifies this repo across the configured backends. You can configure multiple repos to backup to the same backend, their blobs will be deduplicated. Snapshots will be partitioned by name.

Create a Snapshot

Archive the current state of your files:

aegis snapshot

Aegis will show you the changes and ask for confirmation before uploading.

Make Changes and Snapshot Again

After modifying files, you can run status to see what changed:

aegis status

Then create another snapshot:

aegis snapshot

Only the changed files will be uploaded.

Verify Your Backup

At some point you may want to verify the integrity of your snapshot. You can use verify to download and confirm the checksum of all files in a snapshot:

aegis verify

Note: This can take some time for large backups, and be costly if verifying a backend that charges for download bandwidth.

Restore Files

If you accidentally delete or modify a file, restore it from the latest snapshot:

aegis restore path/to/file

Configuration Reference

The aegis.toml file configures your repository:

# Required: A unique name for this repository
name = "photos"

# Required: Which backend to use by default
default_backend = "local"

# Backend configurations (at least one required)
[backends.local]
type = "fs"
dir = "/path/to/local/backup"

You can define multiple backends and switch between them using --backend:

aegis status --backend cloud

See the backends documentation for all available backend types and their configuration options.