Wednesday, July 10, 2013

Names changed to protect the guilty

Being a moderately-talented programmer, I am a big fan of the Daily WTF. Since I am also a shitty writer, I submit the following, in accordance with the style of the site.

Part 1: The Interview
Nathan arrived at the group interview for a webmaster / developer position with ACME Corporation. The phone interview had gone well, and he was one of 4 people selected for the final group interview. During the phone interview, the owner of ACME, Bob, briefly explained that they were looking for someone with expertise in HTML and XML to develop a new web app for their site, and gave Nathan the time & place to show up. Arriving at the office, there were two other applicants, an older gentleman in a suit, and a college aged guy. The final applicant never showed up. Since the position was offering far more than the average market value for salary, Nathan assumed the competition for the job would be stiff.

All 3 applicants were brought to a room, and Bob spoke about the company history, as well as what he was looking for in future development with the website. Then he started asking the 3 candidates questions. The older gentleman went first, and began his tale of woe about his failed business teaching people Java. After about 20 minutes, he finished by saying that he couldn't understand why he never attracted many clients, and was looking to get into "web stuff". The college kid hadn't had a web development job before, but he was currently running a Linux server out of his mom's basement hosting some sites for his friends. He didn't know XML, but he felt confidant that he would be able to master classic ASP well enough to do a bang-up job on ACME's projects. He lived two counties away, and assured Bob that his first priority upon being hired would be to buy a reliable car so he would be able to commute to the office regularly.

5 minutes into the older man's speech on "Object Orientated Programming", Nathan started idly doodling on the notepad he habitually carried around. When the young man finished, he looked at Bob and stated "I don't know if there's a new spec out, but all my Java books have the words 'Object Oriented' on the cover. I live around the corner from the office, and I've spent the last decade working in web development for [corporations you would recognize]." He slid his notebook across the table to Bob. "Here's a rough sketch of the XML and transactions to set up the service, based on what we've talked about. I would need some more detail on the points marked on that page, but it shouldn't be too tough to build."
Bob asked Nathan a few more questions, and the interview ended.

Part 2: A few months later
Nathan had learned over the course of the last few months that "HTML and XML" were indeed part of the requirements for the job. Also, managing ACME's SQL server, web server, Exchange server, as well as maintaining the site written in classic ASP using splashes of XML, JavaScript, VB Script, coded by roughly 10 people over the last few years with varying levels of documentation. And troubleshooting the PCs and printers used by the office staff. That along with a lengthy list of new app and feature requests made for a broad set of tasks each day. The company website was also the backbone of their operations, all work done at ACME was through the site. The office staff would log in every morning, and begin entering orders the less technical customers had faxed in, as well as processing the orders they had received. The front end of the site was the tip of a large iceberg of back end custom features to support this.
Around 11PM one Saturday night, Nathan's phone began ringing while he was in the middle of a 'liquid therapy' session at the local bar. Bob's voice was frantic: "The login page on our site doesn't work! We need this fixed now!" Nathan, drunk and feeling no urgency to overhaul a login page on a weekend, sighed, grabbed his laptop and checked the site.

Sure enough, his login failed when trying to access the site. He also noticed some strange script text showing up on various pages on the site. Odd, where was these strings coming from? In the back of his well lubricated mind, alarm klaxons started blaring. Logging in to the web server and SQL server, Nathan noticed a high volume of traffic coming from an IP in Russia. And that the table for user accounts now had all-new information instead of the encrypted usernames and passwords it was supposed to contain. Username fields now contained the same script tags he'd seen on other parts of the site. Knowing that this was bad, and not feeling up to resolving the obvious issue, Nathan set up a blanket ban for all IPs outside of the U.S., reset all user accounts and changed all of the passwords, and removed the login form from the site. Not wanting to try to explain this while drunk, he called Bob back and notified him that he located the problem, and that logging in to the site would be disabled until he had a chance to review the issue more thoroughly in the morning. Satisfied that whatever script kiddies were dicking around with SQL injection were thwarted for the time being, he headed home to catch a little sleep, sober up, and begin proper investigation.

A few hours later, examining the database revealed that 3 tables had data overwritten. Two of those had basic information that would be piped out to the site upon request, and the 3rd was the table of user data. Nathan restored the two tables with a backup from the previous week, since nothing there changed much and started going through the code on the site looking for vulnerabilities. Each passing hour spiked his blood pressure a little more. Finally, he called Bob. "Who set up the security on the site?" Knowing that the answer would be Local IT Outsourcing Shop. "The local shop did it. We had them run tests on the site, and patch all the holes." "Do we have any documentation on what was found and what they did?" Bob replied that there was indeed documentation, as ACME had paid handsomely for such a service. Nathan agreed to meet Bob at the office immediately to see that doc, as it appeared that there were still some outstanding issues.
Bob proudly handed over a 20 page document of noted security issues with the site, and a form verifying that they had all been addressed. Nathan checked each one in turn, noting that SQL injection was one of the issues tagged as 'high priority' on the very first page. After reviewing all of the issues and the code on the site, he walked into Bob's office.

"So, about those security issues in the report."
"Yes?"
"How much did we pay to have those fixed?"
"[large number of dollars]"
"You might have overpaid. The thing that broke the login page trashed some of our database, and was not fixed. Neither was anything else listed on that report. None of the security fixes were put into place."
Bob stared in disbelief. "What do you mean, nothing?"
"Not a single goddamned thing was ever patched. I checked the whole report, and the major flaws were not addressed at all. So . . . . I'm going to take a couple days to address these items, but here's a copy of the site files from the day I started. I imagine this will be useful to the lawyers."


No comments:

Post a Comment