Recap Of WHaTDa And QA Or The Highway

Monday and Tuesday of this week (02/16 and 02/17/2015) had me in Columbus, Ohio for the Workshop on Teaching Test Design and QA Or The Highway. QA Or The Highway is a regional conference designed for software testers. This is the second running, and the second time it has sold out. This time to a bigger crowd than last year.

QA Or The Highway was great again this year. I didn’t attend a whole lot of track sessions, but the ones I did get to were high quality. Joe Ours puts on a good conference.

I did speak at the conference this year which was different from last. I did a talk on testing APIs that included and intro to REST, information on the testing VS checking dilemma, and a live demo with some code I wrote using frisbyjs. Public speaking is a new craft for me, and I’m growing into is and learning from others. Overall, I feel like it went pretty well.

WHaTDa was a one day workshop focused on teaching test design. It reminded me a lot of WHOSE, that is a good thing. We started the workshop at 9am with introductions; a little about who we were, what we were working on, and how we were planning to contribute to the workshop.

The goal, as with most Excelon Development workshops was to produce something useful to the testing community by the end of the day. Usually about halfway into the day, I start getting the feeling that producing something quickly is completely impossible.

After lunch, we split into groups and focused on an exercise we wanted to build and contribute to the wider testing community. I paired, or maybe grouped is a better word, up with Paul Harju, Megan Studzenski, and Dwayne Green.

A new Test Challenge

The 4 of us built something, and we hope it is useful to you.

We build a testing exercise based on a program that determines whether the text you entered into a field is a palindrome or not. It sounds simple, and it is, but there are a number of ways you can frame this simple program into a test exercise.

Here is the Palindrome Challenge. Feel free to use it however you like with credit to the authors.

Here are some framing examples you can use. This is what we came up with during the workshop, there are of course many other ways you can run the challenge.

For the person giving the challenge: you have to address what the scope of the exercise should be.

Here is one possibility:
1 – Design a test strategy / how would you test this (actually write the strategy down)
2 – Test for a couple of minutes using strategy
3 – What did you find? Did the stuff you found matter? Why?
4 – If you had more time, what tests would you run?
5 – Debrief

You could also do a survey of test techniques.

— Domain
— Function
— Risk
— Load
— Security

Maybe run the exercise a couple times, and see how the test strategy differs based on the identified technique.

Seeding test ideas:
I know there is a bug in X area, how would you test for that?

Megan’s Gambit:
For any given exercise, you can make another exercise by having the student identify what they were doing and why.

Test exercises are everywhere, framing and scoping are the hard part.

Have fun!

QAOrTheHighway 2014

QAOrTheHighway was held this past Tuesday in Dublin, Ohio, a city just outside of Columbus, Ohio. For the most part, this was a very regional thing. A lot of people I spoke with were from Columbus or very near by. That was sort of surprising considering the speakers (keynote and presentation) they managed to get. Despite the small/regional conference feel, there were a pretty good number of attendees present.


I was planning to get there early on Monday, hangout with friends and work maybe a little, but it turns out February is not a great time for flying. My layover was cancelled and I didn’t end up at the hotel till after 11:30pm. Luckily a few friendly faces were still up to chat and catch up despite the late hour on the day before giving a talk.

The day started at 6:30 am with breakfast in the hotel lobby and then rushing off to lean coffee. Lean coffee is a testing conference fixture at this point. People ask for it by name. This one was really enjoyable as usual. I left with some useful notes on preparing to give a conference talk. Oh, did I mention that I’ll be speaking at CAST 2014 in New York, NY?

The conference was kicked off with a keynote by Joseph Ours. The theme was some ways you could tell if you (the tester) were undervalued within your organization and some things you might do to change that. Joseph did a great job and presented some old ideas with a fresh perspective. Some things I thought were interesting was the way he thinks of testers as information brokers and also something he calls the OURS method: Observe, Understand, Review, Serve. One important thing to note is that Joseph was not the scheduled keynote speaker. Keith Klain was scheduled to talk but could not make it and Joseph did a fantastic impromptu keynote.

The first session I went to was The New Tester Skill Set by Matthew Eakin. There was some stuff in this presentation that I didn’t necessarily agree with such as an emphasis on documentation, an emphasis on tools, and very little about testing skill or how that fits into agile but I think Matt had some great points elsewhere. Mainly in emphasizing restricting WIP to be very small at any given time, and also something he mentioned about how a powerful test might tell you specifically where a problem is.

Session two was by JeanAnn Harrison on A Debate on the Merits of Mobile Software Test Automation. JeanAnn is a great speaker and conversationalist, I thought this was a fun talk. This was sort of a socratic talk, a lot of the content was posed as questions for the attendees to consider. I really enjoy this style. Some of the questions were around the idea of defining best, defining need, and asking if the project is worth the investment. She also mentioned that she doesn’t highly rely on domain expertise in new hired because that can usually be picked up quickly. I generally agree with that sentiment.

Session three was about Disintegration Testing presented by David Hoppe. This is another session that I thought was very good. The content was useful and engaging. David talked about the value of looking at problems in isolation as opposed to completely integrated products. He did this via stories about automotive repair, scenarios focusing on how a person might test their amazon home page, and also a live demo of a test program he wrote. The presentation has a little bit of everything and the attendees really seemed to respond to and enjoy that.

My last session of the day was by Scahin Mulik, Four Questions Every CXX Should Ask About Testing. This started off by modeling testing questions around Maslows Hierarchy of Needs. I thought it was quite interesting. The four questions Sachin came up with based on the hierarchy were: Is the software not doing what it is not supposed to do; It is secure, fast enough, reliable enough; Is it loved by its intended audience; Is it faster, cheaper. After this there was as a bit on testing measurements with absolutely no foundation and no reference for where the numbers came from. I wish I had written down the measurements he referenced. One I do remember was defect removal efficiency. There were also some measurements that were somehow supposed to represent industry maturity. This part left me really dissatisfied with the talk.

After this was a closing keynote given by Matt Heusser. Regretfully, I had to miss this to catch a flight back to Nashville.

I was in Columbus for about 20 hours total, not even a full day. If I go next year, I’ll try to hit the 24 hour mark.