Whoops don't be mad guys
Sorry doesn't even begin to cover the boo-boo I've inflicted upon WSGF today. Getting the site ready for a drupal mutli-site setup for a cool rewards/bounty system and moving the store out to a subdomain, I accidentlied a majority of the images uploaded. How might you ask? Why was I so careless? Well, I'm actually super careful because moments like this aren't battle scares you show people. These moments are the ones you regret forever. How it happened was cleaning up some older directories that got copied around during the preperation for mutlisite, a symlink was copied over and when you remove a directory that contains a symlink, by default, it will follow the symlink and continue to delete files in the directory the symlink is pointed to. Backups? Well, funny you should mention that. We'd discussed using object storage such as S3. Developing a backup plan so files aren't deleted too soon between backups was something we were trying to figure out with object storage. So yeah, backups were on the list, we had to delete our local (same disk) backups because of space constraints a few months ago and the sites been running commando ever since. Before you ask, filesystem undelete was already attempted, but in our wisdom, we forgot to prepend the drupal node id to the beginning of the filenames, so now I have 1000 copies of boxart.jpg.#### which is far from helpful.
Hopefully when my faithful counterpart in the UK wakes up he'll tell me he had a total site backup and I can just delete this post...
Source:Tags: outage