Ron Burgundy Posted October 29, 2008 Report Share Posted October 29, 2008 I lost all of my highscores... o well.. and my screen name.. Quote Link to comment Share on other sites More sharing options...
cattmouch33 Posted October 29, 2008 Report Share Posted October 29, 2008 So did the system crash because of all the hits, what server are you running, and what do you use for backing up your data. We just lost a server at work and had to have it back up and running that day. I feel your pain. Quote Link to comment Share on other sites More sharing options...
jon_SV650 Posted October 29, 2008 Report Share Posted October 29, 2008 So did the system crash because of all the hits, what server are you running, and what do you use for backing up your data. We just lost a server at work and had to have it back up and running that day. I feel your pain.Oh yes, i have felt the server pain many days before...Satan, if you need help with anything, let me know... Quote Link to comment Share on other sites More sharing options...
cbrjess0815 Posted October 29, 2008 Report Share Posted October 29, 2008 Oh noez! That was pretty quick with the comeback though! At least OR is running!damn, my post count was way higher tooooo! lol Quote Link to comment Share on other sites More sharing options...
e-flores Posted October 29, 2008 Report Share Posted October 29, 2008 ohioriders is back!!! Thanks for the work ben. I thought I would be bored all day today but I am happy to say I will not be getting any work done today. Quote Link to comment Share on other sites More sharing options...
flounder Posted October 29, 2008 Report Share Posted October 29, 2008 My questions are these from an IT standpoint1. Why the hell wasnt the OR host performing completion checks on backups. (Shitty host to me) Not to mention that they are not doing adequate backups. 2 fucking months, are you kidding me. They should be running full backups weekly and validating those and performing incrementals on another agreed upon basis. Sounds like they are doing full backups monthly and no incrementals at all. I would be getting my money back and find a different host. 2 Does the host ever do periodic restores to test their backup media for problems just like this. 3. What caused the crash in the first place. Quote Link to comment Share on other sites More sharing options...
Dweezel Posted October 29, 2008 Report Share Posted October 29, 2008 server was DUI... Quote Link to comment Share on other sites More sharing options...
redbarron77 Posted October 29, 2008 Report Share Posted October 29, 2008 here's a for Satan. Quote Link to comment Share on other sites More sharing options...
jon_SV650 Posted October 29, 2008 Report Share Posted October 29, 2008 here's a for Satan.i second that one... Quote Link to comment Share on other sites More sharing options...
redbarron77 Posted October 29, 2008 Report Share Posted October 29, 2008 ...wait a minute.....I gotta check "Scrabble".....and the boobies and butts posts....please, for the love of God Satan, tell me the pics are still salvageable!!!!(does anyone else see the irony of that last statement???) Quote Link to comment Share on other sites More sharing options...
Ron Burgundy Posted October 29, 2008 Report Share Posted October 29, 2008 doobie doo so I been absent for a week or two anyway. Does this mean if I had any private messages or anything they gone now?Not that I would but. meh. Gotta post something. 1st day back in what.. two weeks....?LOL back to noob status. FRELL Quote Link to comment Share on other sites More sharing options...
fusion Posted October 29, 2008 Report Share Posted October 29, 2008 Thanks for all the work getting OR back up Ben! Quote Link to comment Share on other sites More sharing options...
mrmako777 Posted October 29, 2008 Report Share Posted October 29, 2008 Quote Link to comment Share on other sites More sharing options...
zerocrash Posted October 29, 2008 Report Share Posted October 29, 2008 ...wait a minute.....I gotta check "Scrabble".....and the boobies and butts posts....please, for the love of God Satan, tell me the pics are still salvageable!!!!(does anyone else see the irony of that last statement???) Quote Link to comment Share on other sites More sharing options...
yotaman88210 Posted October 29, 2008 Report Share Posted October 29, 2008 Damn, shitty. Well thanks for gettin her back up Ben!! Quote Link to comment Share on other sites More sharing options...
mrmako777 Posted October 29, 2008 Report Share Posted October 29, 2008 what is rep power:0 on the main page next to my user name? never noticed that before Quote Link to comment Share on other sites More sharing options...
ninjachk08 Posted October 29, 2008 Report Share Posted October 29, 2008 oh damn this sucks!! lol i know its not your fault ben, but it still sucks. had to reset my password to sign in, then i lost all my friends and albums and avatar..lol it'll take me awhile to get it back where i had it.sounds like we need a bigger server/host... Quote Link to comment Share on other sites More sharing options...
Casper Posted October 29, 2008 Author Report Share Posted October 29, 2008 To answer a few questions....What happend?Several really bad things happened all at the same time. I have a virtual dedicated server hosted out on the east coast. Apparently, they do not do backups of the virtual dedicated servers. I didn't know this. I've been slacking on verifying my own backups. I had it set to run a backup every night, then FTP the backup to another server. Apparently the SFTP server had a harddrive fill up to 75% and stop accepting incoming files. Had I been checking this, I would've known. It was my own fault. I'm still not 100% sure what happened to the database to begin with. While dropping a table, somehow I managed to drop the database. I went over the code a million times. I still have no idea how the database got dropped. However, usually this wouldn't be a big deal. I just restore to the last backup then roll forward the binary logs. All is good. Unfortunately, the backups were not working properly, and the binary log directory wasn't logging the correct database. Why not the correct database? Well back when I upgraded the vBulletin software in August, I had made a copy of the database to use for testing. I went ahead and used the test database for the production site. Somehow, the test database never got added to the list of MySQL databases in the my.cnf file to be logged. It should've been automatic, but I didn't check it. Had the server been here locally, I would've been able to pull the data off the harddrive with no problems. But since it's not here, and it's virtual, there was no way for me to do this. So, several bad things all led to one big boo boo. How do we keep it from happening again?For the immediate solution, I have written a shell script which is ran in CRON every day at 4a. The script does a MySQL dump of the database, compresses and encrypts it, makes two local copies of it, then SFTPs it to my remote server. My remote server then emails me when the file is received. For a long term solution, I'm looking into hosting the server here locally. Had the hardware been here, or at least not virtual, I would've been able to unmount the device, run LSOF, and pull the files off the disk. Unfortunately, I have no control over the hardware the way things are now. I'm looking into either a local co-lo or possibly hosting it myself. What about my post count?Everybody's posts from the past two months are gone. I can add to your post count, but the next time I rebuild/optimize the database your post count will go back down. Seems quite pointless to me. Just post more. Haha.PMs, posts, pictures, etc all gone?Yes. EVERYTHING from the past two months is gone. Name changes, private messages, profile messages, friend lists, social groups, etc...... I'm just thankful we didn't lose more than we did.Cliff's Notes:Shit's gone. Ain't coming back. I suck. Shouldn't happen again. I'm sorry. Quote Link to comment Share on other sites More sharing options...
Dweezel Posted October 29, 2008 Report Share Posted October 29, 2008 eh, you don't suck. You impress the shit out of me, it's up, and running. what more could you ask for? untill post counts can buy me a new bike who really cares? Quote Link to comment Share on other sites More sharing options...
jeremygsxr Posted October 29, 2008 Report Share Posted October 29, 2008 ^ what he said! Quote Link to comment Share on other sites More sharing options...
redbarron77 Posted October 29, 2008 Report Share Posted October 29, 2008 .... I suck....That's what we like about you.....and what your wife LOVES about you!If anyone's livelihood depended on the data, then you can beat yourself up about it. I mean, you aren't getting paid for OR, and it's being done in your spare time, so we get to rehash 2 months of political debates, and have to repost a bunch of porn.......life goes on. Quote Link to comment Share on other sites More sharing options...
cattmouch33 Posted October 29, 2008 Report Share Posted October 29, 2008 At work we have a poweredge 2650 if you are possibly interested in hosting locally. You would need to get bigger harddrives as it only has 5 16gb drives in it now. PM me if you would like to know more about it. It is a rack mount server though.Thank you for getting the site back up. Quote Link to comment Share on other sites More sharing options...
exSRAaron Posted October 29, 2008 Report Share Posted October 29, 2008 I think we all deserve a free round of beers!! Jk, thanks for getting it back up Ben. Now I have to figure out what the hell happen in the last 2 months of my life, thanks Quote Link to comment Share on other sites More sharing options...
bigbluebird Posted October 29, 2008 Report Share Posted October 29, 2008 shit happens, :cheers:thanks for all the work and effort getting or back up Quote Link to comment Share on other sites More sharing options...
SchmuckGirl Posted October 29, 2008 Report Share Posted October 29, 2008 I missed you guys! lol, funny how much I log into the site. Anyway, thank you Ben for getting us back to our post-whoring or lurking or picture ogling or whatever it is each of us do! Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.