backup_for_luciano

Simple backup program tailored for me Luciano. Made with rust, git-bash and powershell.
version: 2026.306.1339 date: 2026-03-06 author: bestia.dev repository: codeberg.org

Hashtags: #tutorial #rust #git-bash #powershell
My projects on Codeberg are more like a tutorial than a finished product: bestia-dev tutorials.

Backup strategy

My working folder is on the internal drive 1TB. Now it is just 116GB, because internal SSD can be really small.
My "primary external drive" is 2TB Samsung T7Shield SSD. It is small, fast, resilient and expensive. My original files that cannot be stored on my laptop's internal drive will be stored on this drive.
My "backup external drive" is 2TB Transcend StoreJet 25H3 ruggedized portable hard drive. It has an old school HDD spinning disk, but it is pretty tough.
I have two 1TB storage boxes online on Hetzner called Hetzner_02 and Hetzner_04. They don't have an offering of 2TB, just 1TB and then 5TB. So I need to split my files accordingly.
I have now 3 box_original folders. One on my laptop and 2 on my "primary external SSD".
My backup strategy is described in this ascii art:

     ┌──────────────┐                       ┌─────────┐                ┌──────────┐           ┌─────────┐          ┌──────────┐
     │internal_drive│                       │T7_Shield│                │Hetzner_04│           │Transcend│          │Hetzner_02│
     └───────┬──────┘                       └────┬────┘                └─────┬────┘           └────┬────┘          └─────┬────┘
             │backup_1_of_box_original_1 (116GB) │                           │                     │                     │     
             │──────────────────────────────────>│                           │                     │                     │     
             │                                   │                           │                     │                     │     
             │                  backup_2_of_box_original_1                   │                     │                     │     
             │──────────────────────────────────────────────────────────────>│                     │                     │     
             │                                   │                           │                     │                     │     
             │                             backup_3_of_box_original_1        │                     │                     │     
             │────────────────────────────────────────────────────────────────────────────────────>│                     │     
             │                                   │                           │                     │                     │     
             │                                   │       backup_1_of_box_original_2 (369GB)        │                     │     
             │                                   │────────────────────────────────────────────────>│                     │     
             │                                   │                           │                     │                     │     
             │                                   │backup_2_of_box_original_2 │                     │                     │     
             │                                   │──────────────────────────>│                     │                     │     
             │                                   │                           │                     │                     │     
             │                                   │       backup_1_of_box_original_3 (636GB)        │                     │     
             │                                   │────────────────────────────────────────────────>│                     │     
             │                                   │                           │                     │                     │     
             │                                   │                      backup_2_of_box_original_3 │                     │     
             │                                   │──────────────────────────────────────────────────────────────────────>│     
     ┌───────┴──────┐                       ┌────┴────┐                ┌─────┴────┐           ┌────┴────┐          ┌─────┴────┐
     │internal_drive│                       │T7_Shield│                │Hetzner_04│           │Transcend│          │Hetzner_02│
     └──────────────┘                       └─────────┘                └──────────┘           └─────────┘          └──────────┘

I am using https://editor.plantuml.com to draw the diagram.

@startuml
internal_drive -> T7_Shield : backup_1_of_box_original_1 (116GB)
internal_drive -> Hetzner_04 : backup_2_of_box_original_1
internal_drive -> Transcend : backup_3_of_box_original_1

T7_Shield ->  Transcend : backup_1_of_box_original_2 (369GB)
T7_Shield ->  Hetzner_04 : backup_2_of_box_original_2

T7_Shield ->  Transcend : backup_1_of_box_original_3 (636GB)
T7_Shield ->  Hetzner_02 : backup_2_of_box_original_3
@enduml

rsync

I would love to use rsync for my backups. But git-bash on Windows is super slow to work with Windows files. I need to invent a workaround.

Hetzner Storage box

I pay for 2 storage boxes of 1TB each on Hetzner around 8€/month. It is not too expensive.

sshadd hetzner
ssh -p 23 -i //wsl.localhost/Debian/home/luciano/.ssh/hetzner_debian_1_luciano_ssh_1 u543604@u543604.your-storagebox.de ls
# backup_2_of_box_original_1
# backup_2_of_box_original_2

sshadd hetzner
ssh -p 23 -i //wsl.localhost/Debian/home/luciano/.ssh/hetzner_debian_1_luciano_ssh_1 u543602@u543602.your-storagebox.de ls
# backup_2_of_box_original_3

File list from Hetzner

Hetzner shell have very limited functionality, but it has the tree command. Very fast! So I can get the file list simply. Add to ~/.ssh/config file:

Host hetzner_storage_box_02
    HostName u543602.your-storagebox.de
    Port 23
    User u543602
    IdentityFile //wsl.localhost/Debian/home/luciano/.ssh/hetzner_debian_1_luciano_ssh_1 
    IdentitiesOnly yes

Host hetzner_storage_box_04
    HostName u543604.your-storagebox.de
    Port 23
    User u543604
    IdentityFile //wsl.localhost/Debian/home/luciano/.ssh/hetzner_debian_1_luciano_ssh_1 
    IdentitiesOnly yes

cd /c/Users/luciano/git-bash/rustprojects/backup_for_luciano
sshadd hetzner
ssh hetzner_storage_box_04   'tree -i -p -s --timefmt "%Y-%m-%d %H:%M:%S" -f backup_2_of_box_original_1' > backup_2_of_box_original_1.txt
# 4 seconds
# 4206 directories, 33874 files

cd /c/Users/luciano/git-bash/rustprojects/backup_for_luciano
sshadd hetzner
ssh hetzner_storage_box_04   'tree -i -p -s --timefmt "%Y-%m-%d %H:%M:%S" -f backup_2_of_box_original_2' > backup_2_of_box_original_2.txt
# 4 seconds
# 1741 directories, 52403 files

cd /c/Users/luciano/git-bash/rustprojects/backup_for_luciano
sshadd hetzner
ssh hetzner_storage_box_02   'tree -i -p -s --timefmt "%Y-%m-%d %H:%M:%S" -f backup_2_of_box_original_3' > backup_2_of_box_original_3.txt
# 4 seconds
# 3465 directories, 57843 files

The format of the resulting files is:

[drwxr-xr-x           6 2026-01-17 11:44:03]  backup_2_of_box_original_1/0_temp/peacefulview_export
[-rw-r--r--   517201991 2026-01-17 11:40:23]  backup_2_of_box_original_1/0_temp/peacefulview_export/peacefulview_eu_public_html.zip

File list from local originals

I will use a powershell command from git-bash because it is fast, really fast:

pwsh -Command 'dir -r d:\box_original_1  | % { if ($_.PsIsContainer) {"d`t" + "0" + "`t" + "2026-01-01 01:01:01" + "`t" + $_.FullName } else {"f`t" + $_.Length + "`t" + $_.LastWriteTime.ToString("yyyy-MM-dd hh:mm:ss") + "`t" +$_.FullName } } ' > box_original_1.txt
# 5 seconds
# 69591 lines

pwsh -Command 'dir -r e:\box_original_2  | % { if ($_.PsIsContainer) {"d`t" + "0" + "`t" + "2026-01-01 01:01:01" + "`t" + $_.FullName } else {"f`t" + $_.Length + "`t" + $_.LastWriteTime.ToString("yyyy-MM-dd hh:mm:ss") + "`t" +$_.FullName } } ' > box_original_2.txt
# 5 seconds
# 55015 lines

pwsh -Command 'dir -r e:\box_original_3  | % { if ($_.PsIsContainer) {"d`t" + "0" + "`t" + "2026-01-01 01:01:01" + "`t" + $_.FullName } else {"f`t" + $_.Length + "`t" + $_.LastWriteTime.ToString("yyyy-MM-dd hh:mm:ss") + "`t" +$_.FullName } } ' > box_original_3.txt
# 5 seconds
# 60865 lines

The format of the resulting files is:

d	0	2026-01-01 01:01:01	D:\box_original_1\0_temp\peacefulview_export
f	517201991	2026-01-17 12:40:23	D:\box_original_1\0_temp\peacefulview_export\peacefulview_eu_public_html.zip

Compare file lists

Now I need a rust program to parse these files, compare and find the differences.
This is strictly one-way sync, from local to remote.
Basic rules:

same name, size and timestamp around 3 seconds is the same file. Ignore.
different size or timestamp or more than 3 seconds difference goes into the list for upload
not-existing on remote goes into the list for upload
not existing on local goes into the list for deletion

The 3 seconds rule is stupid because Microsoft's exFAT has a resolution of 2 seconds for the file timestamp. How stupid is that in 2026! Back to the future 1980 - Hooray Microsoft.

First I need to sort the files with the same sort order. Sometimes different OS have different sort orders for non alphabetic symbols.

rust to the rescue

I created a rust CLI program in Linux, cross-compiled to Windows.

clear && cargo fmt &&  cargo build --release --target x86_64-pc-windows-gnu

It can be run inside git-bash.

cd git-bash/rustprojects/backup_for_luciano/

scp crustde:/home/rustdevuser/rustprojects/backup_for_luciano/target/x86_64-pc-windows-gnu/release/backup_for_luciano.exe backup_for_luciano.exe && BACKUP_FOR_LUCIANO_LOG="debug" ./backup_for_luciano -h

The program has a few commands:

remote_file_list
local_file_list
compare
upload_added
upload_changed
delete_removed
backup_local

Development

File systems are a disaster. I had problems with UTF8 encoding, hidden files, escaping quote and double quote,... To not mention the Microsoft exFAT 2 second resolution.
Incredibly frustrating.
In desperation I just deleted some files with strange characters that were not supported. They were non important files.

TODO

use cross_platform_file_path TUI, on the top shows the commands as menu or buttons. On the button shows the terminal. New automation code for codeberg instead of github add datetime when presenting the error to the user. So he can communicate to the developer the datetime and the developer can find what happened in the log.

Open-source and free as a beer

My open-source projects are free as a beer (MIT license).
I just love programming.
But I need also to drink. If you find my projects and tutorials helpful, please buy me a beer by donating to my PayPal.
You know the price of a beer in your local bar ;-)
So I can drink a free beer for your health :-)
Na zdravje! Alla salute! Prost! Nazdravlje! 🍻

//bestia.dev
//codeberg.org/bestia-dev
//github.com/bestia-dev
//bestiadev.substack.com
//youtube.com/@bestia-dev-tutorials