Back to overview
Downtime

Alonso Comatose

Apr 11, 2024 at 6:50pm UTC
Affected services
alonso & wilf pulse
borg: alonso-full
borg: wilf-vault
plex
overseerr requests

Resolved
May 9, 2024 at 3:10pm UTC

Backfill is complete and requests app is synced. We're so back

Updated
May 3, 2024 at 8:14pm UTC

TV Shows continue downloading, Movies are queued, Requests is still disconnected and will re-sync once downloads are complete

Updated
May 1, 2024 at 7:59pm UTC

Plex library has been reset, backfill is downloading

Updated
May 1, 2024 at 4:32pm UTC

New zpool is up and running, the Plex library will be resetting shortly.

Updated
May 1, 2024 at 1:28pm UTC

New disk burn-in is complete, we are now building the new pool and attaching to services.

Updated
Apr 21, 2024 at 4:47pm UTC

New disk burn-in testing is estimated to take ~1 week

Created
Apr 11, 2024 at 6:50pm UTC

A sudden power outage has fatally wounded one of Alonso's media disks, resulting in a total data loss for the whole pool. A new larger and more resilient pool is being built: ~34tb of space with a tolerance for 2 full disk failures. This is expected to take about two weeks. When Alonso returns, current shows will automatically be redownloaded, but older movies and TV will need to be re-requested.