So I learned a pretty tough lesson today. I was doing some research for a project at work (which was going to be the original topic of today’s post) on which required me to setup a SOCKs proxy through AWS. I logged onto AWS, configured a small instance and went to spin it up when I got an error message saying that I didn’t have permissions to do this. That’s odd, I had used this account a lot when I was getting ready for ACSC so I knew I did.
I started to do some digging, and noticed a bunch of instances that were shut down that I didn’t recall making. I decided to go to the billing page and was absolutely floored when I saw this.
Yep, that’s a $4,218 bill. I went to open a support case and discovered that AWS had actually tried to alert me several days earlier (going to an account that I don’t regularly check), and eventually locked the account when I didn’t reply. I started trying to figure out what the hell had happened. I changed the password, but kept digging. Eventually I look at my API keys and see one had been used 8 days ago. This was a test key, I was using for some development stuff. Apparently I never bother to delete it. OK, got it, but how did the API key get stolen?
The testing I was doing was when I was building out stuff for ACSC. I was using Ansible to spin up a lot of the stuff, and had posted the yaml to my github. Checked there, no luck. I had sanitized the script before I posted it. Only other place I could think was on the ACSC repo itself. Sure as hell….
Looks like I had put my script in that git also, without first sanitizing it. So there we have it. I deleted the API key from the account, deleted all of the VMs, and everything else (nearly 100 VMs across 15 regions if I remember correctly). I have a request in with AWS to see if I am going to have to actually pay for this thing (fingers crossed)