Friday, July 25, 2008

Software Quality Assurance: SDL (Secure Development Lifecycle

At work, I've been helping one of my teams implement portions of Microsoft's Secure Development Lifecycle (SDL). SDL is more than just plinking around attempting some penetration testing--it's a committed approach to secure software design. Here are some of the takeaways from our work:

  1. The first point I made with my team is that security features DO NOT EQUAL secure features. Having SSL encrypted communications does not make a web application secure! It just means you have an encrypted communications channel. Secure software isn't secure because of features such as Acegi security, RSA encryption or anything like it. Secure software is produced when developers think secure from the start. Secure software comes when code is written safely, when developers write solid, secure code, and when testers are helping with the planning and testing of the software with a secure focus.
  2. Next point we emphasized: threat modeling. As we wrapped up a two-hour threat modeling session, one of the developers commented "Why didn't we do this months ago, before our code was written!". Good point!! It's never too early to threat model. Analyze your product, paying special attention to where data crosses boundaries: user to Internet, Internet to server, server to database, etc. Model threats, wild and crazy or down-to-earth. Our threat modeling has resulted in 9 potential threats so far, and we expect several more as we continue.
  3. Security comes in layers. Back when I lived in India, I toured the Delhi fort. This fort was built by professionals. It has a deep moat around it. Tall, thick walls surround an inner wall, and inside that inner wall lies the fort. That's how our code should be! OK - so you're authenticated via Acegi and LDAP. You're encrypted with SSL. That's great - but what if someone logs in with a valid account, then tries to hijack another session? Your layered security will catch hacks like this--authorize anytime someone tries to access sensitive data. Even if you've already authenticated and authorized, do it again! Layers bring security.
  4. Reduce your footprint: in Agra, where the Taj Majal is, there's another fort (these Moguls were building forts everywhere!). This fort is on the edge of town, in the hills. Compared to the Taj, the fort is tiny. This is referred to as attack surface reduction. Only allow public access to a few of your resources. If you have features which suffer from weak security, disable them by default or remove them completely. Give hackers as little space as you can.
  5. Train your engineers (dev and test). There's common training needed by both (elements of secure design, running a threat model, etc.) and there are discipline-specific trainings such as penetration testing or the application of specific technologies. The SDL is called a lifecycle because it's a continuous process. Lather, rinse, repeat and all that.

Our training has produced benefits. For starters, developers have a new security-focused mind set. We've found a few security bugs already, and our threat model has exposed some potential issues. This is great progress, and it comes from just one day of work. Imagine what we'll be like in a few months after a day or two of training and a complete milestone with security in mind!

It's never too early or too late to take a step back and start thinking security. I designed our course based on my experience at Microsoft, which was neatly documented in Michael Howard's new book "SDL: Secure Development Lifecycle". I cannot recommend reading this book enough!

Got a security question? Post it here...

Tuesday, July 15, 2008

Software Testing Patterns: Sessions

One of the most common things to have to test is sessions, cookies, and login/logout. This is the start of a pattern for that kind of testing.




First, some background:




Test Patterns:





  • Session hijacking: create several cookies for your site (varying login sessions). Tests: 1) can you substitute the session id from one session and use it in another session. 2) what happens with invalid session IDs? 3) Prevention: specify that cookies must be https-based (specify the secure flag when creating the cookie.


  • Cookie poisoning: change values in a cookie - for instance, if a cookie stores the check-out value in a shopping cart, the user could change that value before checking out.


  • Back button: do something that adds/modifies a cookie, then click 'back'. The state could then be out of sync (being on a page which expects you to NOT have a value in your cookie which you actually do have).


  • Expiration: cookies w/o expirations are considered 'not persistent' and should be destroyed at the end of the session. Persistent cookies represent a security risk due to how long they are persistent; make sure the expiration of a cookie is reasonable (30 min?) based on typical user scenarios.

  • Log in, thereby picking up a sessionID. Make note of the sessionid. Close the browser, and then log in as someone else. Finally, update the cookie to be the id you noted previously, and hit refresh. Does the session change?

Software Testing Patterns

I'm convinced there are a series of testing patterns we could use. I haven't found any on the Internet, which is surprising, but I figure I'll start documenting them here. Look for blogs with "Test Pattern" in the title.

Don't be shy about contributing, please!

First of all, some links:

Quality Assurance or Testing Seminars In India

I've often wondered if there's a market for QA/quality assurance/testing seminars in India. I've thought of setting up seminars for how to test in cities like Mumbai, Bangalore, Delhi, Chennai. One day seminars where people pay a flat fee to get in and we spend the day really digging into testing. Some presentations, some 'labs' and a lot of Q/A.

If you think there's value, post a comment. If you'd be interested in partnering, hit me in an e-mail joatgm at gmail dot com.

Should Freshers be Testing Software?

I have seen an alarming trend in India and worldwide. "Test engineers sought - freshers" (for those not familiar with the Indian IT scene, a 'fresher' is someone fresh out of college - 0 to 1 year experience, maybe even a little more).

Seems like everyone with an open tester position wants to throw a fresher at it. This is a huge mistake! What does a fresher bring to the table?
  • Enthusiasm
  • Lots of book learning; in India, that means rote memorization
  • Some programming experience.
I once helped a large, failing consumer electronics company move work offshore to save money (offshoring to save money is a BAAAADDDD idea--ask me why or wait for a blog on it). The development 'partner' on the project brought in a ton of freshers and some somewhat seasoned testers. What did we get? We had test engineers who didn't know what a defect report was, who'd never used a defect tracking tool, who had absolutely no domain knowledge. All they had to contribute was 1) a warm body and 2) some experience in Java. This is a recipe for disaster.

Is there room for freshers in an IT organization? Absolutely! Microsoft focuses most of its recruiting in the college space, but that's because the ratio of new hire to full timer (at least in the US) is probably 10:1. They have people all around them who can mentor and grow them, and they are NOT encouraged/forced to do mundane and boring tasks. Well, not all the time... As a new hire at Microsoft, I was the release lead for German Mac Office 98, with 18 months experience in the company! But that was after 18 months of incredible mentoring and learning.

For you people hiring for testers, don't think "Well, no one wants to be a tester - let's find someone who wants a foot in the door and get them in here." You're doing yourself a disservice, you're doing your customer/project a disservice, and you're really not helping the fresher either. Look for some passionate QA/testing professionals. Bring them in and let them establish a world-class test organization. Then hire freshers, train them up, and keep growing your organization.

OK - off my soapbox for now...