Latest Entries »

Thoughts from #telsum

Filed Under: Community, News by Trish Khoo — Leave a comment
May 20, 2013

A few weeks ago I one of a lucky few to have the chance to go to the Telerik Testing Summit (#telsum). It’s an invite-only annual peer conference held in Austin, Texas and I was in good company with fellow attendees Paul Carvalho, Adam Goucher, AlanPage, Matt Brandt, Selena Delesie, Matt Barcomb, Jeff Morgan, Marlena Compton, Steven Vore, Chris McMahon and of course our excellent host Jim Holmes.

I’d wanted to meet many of the other attendees in person for a long time so it was awesome to finally do so! What followed were several days of intense discussion about a range of topics. Here are my highlights:

Testing is an activity, not a role:

No matter what we discussed, we always seemed to come back to this. Whether it’s testers writing more code, or developers doing more testing, teams continue to benefit from multi-skilled individuals. I hear a lot about this approach from other Google Test Engineers as well. Some describe the Test Engineer role as more of a Test Management role, even though there may only be one Test Engineer on a team. They are often leading the development team towards better quality testing activities rather than doing all of the testing activities themselves all of the time.

All test metrics suck:

There’s a long history of measuring testing-related metrics in isolation from whole-team metrics, which can lead to undesirable results. Ultimately it comes down to measuring the outcomes of our testing (delivering better features more quickly) rather than the means of our testing (how many test cases have we automated) in order to track more valuable and meaningful results.

Good test design and mind maps:

I was reminded of the usefulness of mind maps for test planning. Alan said that he uses mind maps to plan his testing initially, shares the map with the development team, then they work on testing the feature together, automate whatever makes sense and then never have to look at the map again.

I think the best thing about this is that it forces you out of the all-too-common process of writing a series of manual step-by-step test scripts and translating them directly into automated tests. Instead, it really highlights what you should actually be doing with “automated testing” – building a program that is designed to give you important information about the system under test. The information that you really want to know will drive the features that you add to this program.

What I took away:

I felt that the most important point that we kept coming back to was that we’re not in the software testing business – we’re in the software development business. We’re skilled at testing as an activity, but our goal is to deliver software. The testing activity is an important part of that process, but it shouldn’t be isolated to select individuals within a team. It’s something that all team members should be able to do. Alan Page wrote a great post about it which you should go and read now, and a second one which you should read right after that. It would be somewhat revolutionary, if it weren’t already happening. I’ve talked about blurring the line between testers and developers before, in terms of how having skills that are typically associated with a “software engineer” role (for example, programming) can benefit testers. It definitely goes both ways – having skills that are typically associated with a “software tester” role can also benefit developers.

When I shared this view with my team at work, I had an interesting discussion with Marco DeLaurenti who works as DevOps on my team. He said that there are similar discussions happening in the world of DevOps – an increasing demand to blur the line between DevOps and Developer. If you’re interested, there’s a fascinating article here by Mantas Klasavicius about Metrics-Driven Development. Like testing, systems monitoring is an activity that benefits the whole team and it’s easier to achieve useful results if the whole team knows something about it and is willing to make it part of their software development process. If you think about it, software testing is like another kind of systems monitoring – they both ask a question of the system and provide more insight to the team. Sometimes they’re even the same questions!

Many thanks to Telerik, Jim Holmes and the team for making a fantastic event, and to my fellow Telerik attendees for giving me so much inspiration.

Comment

One simple suggestion to encourage more women into IT

Filed Under: Community, Random musings by Trish Khoo — 1 Comment
May 1, 2013

Last week I wrote a blog post about a conversation I had with a young girl about women who work in technology. I did not anticipate such a massive response! #1 on Hacker News and almost 40,000 page visits in 24 hours. I’m encouraged to know that so many people are thinking about this topic.

In case a few people are still listening, I have one suggestion to make. Now, I know that this is a complex issue and there isn’t an easy solution. So instead of offering a solution, I would like to offer a small, positive step.

So here it is.

If you are a person – of any gender! – who works closely with technology, make friends with a woman who does not know much about technology.

If you are a woman who does not know much about technology, make friends with a person who works closely with technology.

My hope is that this will encourage more women to talk about technology. Then perhaps their daughters and your daughters will see this interaction and it will become ordinary for them. Then maybe one day it will become ordinary for women to talk about technology. And one day maybe it will become ordinary for women to work in technology too.

And hey, even if it doesn’t end up solving the big issue, at least you’ve made a new friend! I’m sure many of you will say that you already have friends like these, which is wonderful. I do too. I could use a few more.

Comment

I knew exactly how she felt

Filed Under: Random musings by Trish Khoo — 44 Comments
April 26, 2013

I went to a Women in IT event last night at Google Campus London designed to connect professional women in the IT industry to high school girls. Most of the girls I talked to were either interested in a different field, or were toying with the idea of computing as a backup plan. Then at the end of the evening, one girl pushed through the crowd to talk to me.

“Hi! You said you were an engineer?” she asked. I said yeah, a Test Engineer at Google. “That’s so cool! Have you ever seen Star Wars?” I looked really shocked and said of course I had. “Oh! I didn’t mean to offend you!” she said. I said I wasn’t offended, it’s just that nobody had ever asked me that before.

“I’m so glad to meet an engineer,” she enthused, “all the other women I talked to here were in marketing or law or something. I thought it was meant to be about IT.” I said that’s what happens when we hold events for women in IT when there aren’t that many women in this industry – we tend to broaden the definition of women in IT.

She then talked to me about robotics and physics and programming and how none of the other girls in her class understood anything she was talking about. They thought she wanted to build the next Terminator or hack into Government systems, when she just wanted to build self-driving cars.

All the other girls would talk about fashion labels and gossip and she would be bored out of her mind, wondering why nobody else watched Doctor Who. None of them understood the joke on her Darth Vader t-shirt. One of them listened to her podcast and asked her what “Marvel” was. One of them told her to stop making them feel stupid all the time.

She said that she really feels at home with other geeks, so she ends up hanging out with other geeks and they’re all boys at her school. I told her I knew exactly how she felt. I introduced her to a friend of mine who’s a software engineer and we told her that if she went into IT that it would be full of our people and that it’s great. She was really happy about that.

It’s funny because the evening started out with a lady giving a speech about how the IT industry is alienating women. But for some girls, it feels like women are alienating the girls who want to be in IT.

Comment

Testing Positive

Filed Under: Community, News, Random musings by Trish Khoo — Leave a comment
April 24, 2013

I gave a talk at the Sydney Testers Meetup last year called something very New-Age-sounding like “The Power of Positive Testing”. It was a short talk with a positive message so I thought I’d spruce it up a bit and present it to the London Tester Gathering now that I live on this side of the pond. The old slides looked a bit boring though, so I decided to make them look a bit more interesting.

Then I had a mad idea. Hey, how about I code my presentation!

So I dug up an old Smashing Magazine free HTML5 parallax effect template and several hours and one small coding tantrum later, I had this little beauty. Pretty cool, right?

Those of you who are viewing it on anything other than a standard 13″ Macbook Air resolution can already see the hilarious flaw in my plan.

Yes, of course I tested it and I knew it would have issues in different screen resolutions. But like a true laidback Aussie stereotype, I thought she’ll be right, MVP mate, I just need to get it working on my laptop and mirror the screen on the projector. At least, that’s what I’d do if any of the screen output adaptors I had worked with the projector. In the end it wasn’t such a disaster though, as I’d already uploaded it to this web server and could access it from someone else’s laptop which had a working converter (thanks Deri!). It just looked a bit wonky in the different screen resolution and didn’t scroll so smoothly on Deri’s Compaq.

Anyway, valuable lessons learned about giving presentations:
- Adaptors are cheap, just buy tons of them and bring them all. Why not.
- If you’re giving a presentation about software testing or development and you’ve coded your presentation, make sure it works properly.
- Actually just don’t code your presentation, it’s probably not worth it. With the time you save, you could be rehearsing! Or learning a second language. Or watching TV. Or doing just about anything else that’s better than coding your presentation.
- Stock photos be damned – all that Instagramming has finally paid off!
- Have a backup plan. Make this backup plan involve a USB thumb drive. Do not make this backup plan involve the internet. Especially do not make this backup plan involve Github.

Who am I kidding – I had fun with this, and I got to tell a funny story too. Plus I learned a little bit more about HTML5 and CSS in the process. Overall the presentation was well-received, and that’s the important part.

If you’re interested in what the talk was actually about, Dan Ashby has written a great blog post about it.

For my next act, I’ll be speaking at the Selenium Conference in Boston in June. With sensible slides. Probably. :)

Comment

How I ended up working for Google this year

Filed Under: News by Trish Khoo — 8 Comments
December 20, 2012

So, to recap my year briefly:

- Decided to move to London
- Decided to move to San Francisco instead
- Went to SF Agile and met some great people
- Left Campaign Monitor
- Decided to move to London again
- Did a short stint as a Programme Test Manager at NBN Co which was rather eye-opening
- Joined Wildfire Interactive
- Wildfire Interactive was acquired by Google
- Moved to London, started work at Wildfire Interactive as a Test Engineer which so far has been awesome
- Moved into amazing Google offices

So it’s been a pretty incredible year! I’ve learned so much and met some really amazing people. I’m really happy to be working with the Wildfire team in Google here in London and I look forward to what 2013 brings.

Comment

A lesson in exploratory testing

Filed Under: Manual testing, Random musings by Trish Khoo — 13 Comments
October 9, 2012

Okay, I think I finally get it. I’ve been looking at exploratory testing all wrong.

I was trying to break down exploratory testing into a set of learnable techniques that can be followed by anyone to get them to be better testers. But exploratory testing isn’t like that. I was getting really frustrated because there were significant elements to it which were hard to define. But that’s because exploratory testing isn’t a series of set definable techniques.

It’s like learning the guitar. You can learn the chords if you want, but that’s not how you become a good guitarist. At some point you have to start figuring out how to make the guitar work for you, how to get it to do what you want. And learning the chords is not the only way to learn the guitar. If somebody gave you a guitar and you had no idea what it was, given enough time you could still learn how to make it play beautiful music. And if someone asked you how you did that, and asked you to teach them, maybe you would struggle to find an effective way. They could mimic exactly what you do, but it wouldn’t help them understand how to create music for themselves.

I used to learn piano. I’d show up to the lessons and practice the same songs over and over again, but I never enjoyed playing the piano, or even making music in general, until I started experimenting with my own methods of learning. In order to become better at something, you do need practice, and practice helps, but you also need to take an active part in the learning process. Practice will only take you so far.

Testing is the same. Like James Bach says, testing is a craft. It’s not something that you can teach as a series of steps. It’s something that you need to practice, and then take an active participation in your own learning and growth towards becoming better at it. That’s why it’s hard to measure if someone is a good tester. To go back to the piano analogy, I did AMEB (Australian Music Examination Board) exams for piano and I achieved grade 5. I put absolutely no emotion into my piano playing, I just followed the notes and directions on the sheet music. I learned a few scales. My knowledge and technique was probably grade 5 standard. But I was not a great pianist. Music is not a mindless, step-following activity. Nor is programming. Nor is testing.

I think what I wanted was an effective way of teaching testers how to be better testers. But showing them techniques will only take them so far. They need that interest and drive to learn more, otherwise they’re just doing what I did when I was learning the piano – following a series of steps. They’ll never become great testers, or great at anything, if they don’t take an active interest in their learning experience. I can’t wrap a process around a people problem.

Ultimately, in order to teach something, it’s more valuable to inspire a desire for learning than to transfer knowledge.

To that end, it’s important to acknowledge exploratory testing as a skill that requires practice. Some people say they are naturally “bug magnets”, but natural talent without dedication to self-improvement will only get you so far.

Comment

Using Ruby and Automator to partially automate your exploratory testing on OSX

Filed Under: Automation, Manual testing, Tools by Trish Khoo — Leave a comment
September 19, 2012

Some time ago I talked about linking automated scripts to keyboard shortcuts to generate test data on the fly. Since then, I’ve updated the code, stored it in Github and figured out how to make it work in OSX using Automator, rather than installing some additional app. You can find the Github repository here.

For some reason, setting this up with Automator was a right pain, so I’m recording the steps here so that I never have to go through it again. But once it’s set up, it’s very handy.

Prerequisites:
- You must have Ruby set up on your machine, ready to run. OSX does have this set up by default so maybe you’ll be alright.
- It probably wouldn’t hurt to know a bit of Ruby so that you know what the code is doing, then you can use this knowledge to do all kinds of awesome things instead of just doing what I did!

1. Create a new Automator Service
This screenshot should explain what you need. The important things to note are:
- Choose type “Run Shell Script” under “Utilities”
- Use Shell “/usr/bin/ruby”
- Select “Service receives no input” in “any application”

Then just copy and paste the source code into the action area.

2. Modify the source
You will need to change the filepath of the web2 file to wherever it resides on your computer. Important – the shell script will ignore references to your home directory and relative filepaths so you must use an absolute filepath (e.g. ‘/Users/trish/Documents/web2′)

For the email script, you will need to modify the email address to be your own email address (where it says “YOUR_EMAIL”).

3. Add a Growl notification (Optional)
If you have Growl installed you can add a notification for these actions which looks kind of cool and also lets you know that the script is working.

4. Save it!
You might also want to run it to make sure it’s working properly.

4. Link it to a keyboard shorcut
Open System Preferences > Keyboard and you should see your new services, ready to have keyboard shortcuts associated with them.

Important! Pick a keyboard shortcut that’s not already in use by your other apps, otherwise your new keyboard shortcut will be overridden. I ended up going for something like AppleKey+Shift+2 which seemed to work okay for most of my apps.

5. Try it out!
Press your keyboard combination while you’re filling in online forms and watch your test data magically appear! Isn’t that easier than trying to think up unique data all the time?

Comment

Interview questions to ask your future employer

Filed Under: Interviewing, Office survival by Trish Khoo — 6 Comments
September 1, 2012

Anyone who’s ever been in a crappy job knows emerges a little more wary and much better prepared for the next one. As interviewing is a two-way street, it’s always a good idea to ask your potential employer a few basic questions before accepting the gig. After all, you need to make sure that the company is as much a fit for you, as you are for them. You are the best judge of how to get the best productivity from yourself, so you need to know that your new environment will be conducive to that.

Obviously there are some pretty basic questions that you should be asking, like what the role is, what it pays and what the culture is like. But here are a few more questions that you may not have considered.

Here are some good questions to ask a future employer (in no particular order).

1. What are the expected working hours? What are the typical working hours?
The minimum hours in your contract and the expected hours may not be the same. If you’ve got a family to get home to, you don’t want to be the only guy walking out the door at 5pm while your boss glares at you for not working back until 7pm like the rest of the team.

2. Do I get admin access to my own machine?
I once had a job heavily involving software development where I didn’t have administrator access to my own machine. This proved challenging. When I asked my next employer this question, he said “yes, of course” and looked at me like I was mad. A good sign.

3. Are there any downloading, workstation installation and internet browsing restrictions? Is online activity monitored?
Being able to download and install things on my workstation, as a software tester/developer, should be a no brainer. I’m more than happy to respect a company’s security policies, but when they prevent me from doing my job and don’t actually contribute towards company security, then I tend to object. I’m not doing anything naughty online that I’m worried about being monitored, but I’d be interested to know if my future employer deems such monitoring to be necessary.

4. Which source control software do you use?
If the answer is “email”, run.

5. What’s your take on personal devices in the workplace? Supported? Unsupported? Outlawed?
In some places it’s useful to have your laptop or iPad around for things like meetings, presentations and testing if one isn’t provided for you.

6. What kind of workstation will I be using? What is the refresh rate?
Nothing’s more depressing than being handed a dinosaur of a machine and falling asleep while waiting for your applications to load. Sometimes fixed by bringing in personal devices (see above).

7. What software development methodology do you use?
Don’t accept any one-word answers without further explanation.

8. Are employees allowed to listen to music while working?
It’s a personal preference, but in my experience most software engineers like to block out the world with headphones in order to focus, myself included. It’s a great trick.

9. How are performance reviews conducted?
I haven’t been burned by this in the past but it’s sure interesting to know.

10. Does your team use continuous integration?
It’s a good indication of how the team works, and a good lead-in to a discussion about how the team operates on a day to day basis.

11. Do I get a permanent desk?
“Hot-desking” seems to be getting pretty popular these days. It might not be your cup of tea. It’s probably also best to check if you get a desk at all. Who knows, maybe your next employer will have you sitting on the floor due to desk restrictions. Nothing would surprise me.

12. What’s the biggest challenge you have faced while working here?
Might as well interview the interviewer. You might get some interesting answers.

13. What test case management system / bug tracking tool / etc do you use?
If the answer to any of this is “Excel” it’s probably a red flag.

14. How often do you deploy to production?
There’s no right answer here, but you’ll probably get an interesting one.

15. What are you hoping to achieve by hiring me? How do you see the candidate contributing to your team?
An excellent question, courtesy of my friend Adam Yuret.

This might seem like a lot of questions, but if you’re considering taking a full-time job that’s a long-term commitment and it’s worth being sure about it. Asking more questions sparks more discussion and that’s a good thing.

Comment

I’m moving to London

Filed Under: News by Trish Khoo — 6 Comments
July 14, 2012

So, a couple of months ago I wrote a post about how determined I was to move to San Francisco.

Yeah, about that… :)

Well in the meantime I changed my mind. I had initially set my sights upon London, changed my mind briefly to San Francisco, wrote a blog post about it, and then changed my mind back to London. It was a tough choice, but I think it’s what I really wanted all along. And here we are – I’ll be moving to London in late August and I’m rather excited. :) It’s a one way ticket and I have no idea how long I’ll stay there. I just plan to enjoy myself and see where life takes me.

As for work, I’ll be joining the lovely folk over at Wildfire Interactive as a Senior QA Engineer.

I’ll miss Sydney and the rest of Australia, but I think the world’s not such a big place once you can tough out a 24hr plane trip. If you’re in Sydney, don’t miss my presentation next week at the Sydney Testers Meetup. It’ll be my last meetup, but the meetup group is being left in capable hands.

See you in some country, somewhere!

Comment

The evolution of the QA newsletter

Filed Under: Reporting, Test Management by Trish Khoo — 1 Comment
June 12, 2012

Last night I had a dream that I was leaving Campaign Monitor. Then I woke up and realised that in less than a week, this sad dream was going to come true. I’ve had such a great time working here and I’ll miss it immensely. It’s a sad truth that often, doing what’s best for you means saying goodbye to people you love.

So before I go, here’s a little look back at one of the more interesting things I implemented at Campaign Monitor – the QA newsletter.

When I first joined Campaign Monitor, there was no formal testing, let alone test reports. I needed a way to communicate test results to the team, and I also needed a way to introduce testing concepts to a company that discouraged formal meetings. So I decided to use the company’s own product to deliver the test report – I made the weekly test report into an email campaign.

Using the application under test in order to send email campaigns had several advantages. First, there was the “eat-your-own-dogfood” advantage of experiencing the product as a user, and experiencing all of the associated bugs and user experience issues associated with that. It helped me put myself in the end-user’s shoes. Also, I could take advantage of Campaign Monitor’s tracking features to tell who was actually reading the newsletters and clicking the links. So I could track the kind of impact the newsletter was having on different teams.

Reporting can often be like a marketing exercise, and the fun newsletter format of the test report made it eye-catching and easy to read. As the company culture was fun and laidback, I included some funny pictures and interesting links to keep people reading.

The first newsletter I made was using a free template from Campaign Monitor’s free templates page. As there had been no formal testing done prior to my arrival, the newsletter was a way to educate the team about testing terms, test practices and changes that I was making to the test infrastructure. This report tries to answer the questions “What is testing?” and “What are you doing?”

I had a progress update about the current project, an update about improvements made to the automation suite, an introduction to a testing terms and a link to some testing-related blog posts. I sent out an email like this once a week in the same format. Each one introduced the team to some new testing terms, and kept them updated on the changes that were being made to the development process with regards to testing. I sent the email to everybody in the company (which at the time was less than 20 people).

I received some very positive feedback about the newsletter. Using Campaign Monitor meant that I could track who opened the newsletter and clicked links. It also meant that I could test the main function of the application (sending campaigns) end to end as a byproduct of creating and sending the report.

After about a month of sending these reports, I decided it was time to introduce the team to some metrics. In particular, I wanted to draw attention to the burndown chart and the automated test results. The burndown chart was visible to the team in JIRA, but hardly anybody was looking at it and most people didn’t know what it meant. So I included a screenshot of the burndown chart with an explanation of what the trend was telling us. This report tries to answer the questions “What do I need to know?” and “Why should I care?”

The team had access to the daily automation results, but I wanted to show the results trend over time as an indication of declining or increasing build quality.

The previous report tried to answer the question “What do I need to know?” But the managers and the support team just wanted to know when we could release the product. The developers just wanted to know how much work they had left to do. So the real questions I needed to answer were “Why aren’t we done yet?” and “Why is it taking so long?”

Project progress is still the star of the show in this report. Freshview values quality over deadlines, and this shows in the language of the progress report. When the development culture is “It’s done when it’s good enough to release”, the question “Is it good enough yet?” becomes the deciding factor for the release date.

The detailed results are an attempt to show progress towards that “good enough” goal, and to answer the questions “Why aren’t we done yet?” and “Why is it taking so long?”

Then we moved the above to a dashboard, so this part of the newsletters became redundant . The dashboards evolved too, and the test case and bug counts were removed completely. You can read more about this in this article written for The Testing Planet: Dash Dash Evolution.

So the newsletter became more of a general project update, as well as a way to try out new email templates and show the team what they looked like. You have to remember that this is a team that has one meeting a week, private offices and rare, if any, standups. This was filling a communication gap.

This newsletter started answering the question “what are the testers doing?”. We started photographing the dashboard. With smaller teams, communication was better so formal reporting was required even less. This was more of a way to let support know what was in the works.

It was pretty obvious by this point that the dashboard was making the newsletter kind of redundant, so all that was required was a brief progress update.

We had a brief attempt to revive the format, and make a project-specific test report. This one has a checklist, a highlighted section for blocking issues and a little paragraph about progress. We found that reporting this kind of progress on a weekly basis was a bit pointless. At the start of the project, nothing would be checked off the checklist. Towards the end of the project, the checklist would be complete within a matter of days. So while the checklist may have been useful, the newsletter was the wrong medium for this kind of information.

In the end I realised that the reports had outlived their usefulness. The initial reports had achieved their goal of creating awareness of testing practices when testing was still new. The next few reports had kept the support team in the loop about releases, and later this was replaced by having the support lead attend weekly developer meetings, and having developers make more of an effort to notify support about upcoming releases. Moving the testers’ offices physically closer to the developers’ offices also helped a lot – most discussions were happening in the hallway outside developers’ offices, so we could actually be a part of those meetings.

So we don’t send the newsletters anymore. But they are a great example of an idea that worked for the right contexts at the right times. Sometimes throwing something away is progress too.

Comment