Interview horror stories
From InfoCamp
Session leader: Kimra McPherson Time: 2:30 p.m.
Kimra's a former reporter before heading to the iSchool. Worked at San Jose Mercury News, where she covered small town issues, interviewing mayors and the like. Later, she became a crime reporter for them, covering breaking news both online and in print. In any gaps, she wrote features on Roller Derby, typewriters, and texting while driving. She left the Merc to became a TV critic and reporter for Buzzsugar.com
Last year, she went back to school to become a user researcher, and has found that interviewing from journalism and user research aren't that different. In fact, they have a lot of commonalities.
The talk is five principles for avoiding bad times in interviews.
1. Find the right time to ask the hard questions When an intern, Kimra heard a story from her editor. As a young reporter, he'd scored an interview with Eppie Ledbetter, also known as Ann Landers. She had a long-running feud with her sister, known as Dear Abby. So he decided he had to ask the tough questions, and his first query to Landers was "So when was the last time you talked to your sister." She hung up. He was devastated and thought he had blown the opportunity. Five minutes later, she called him back and told him to always ask a nice question first, so if your source hangs up at the tough question, you have something any way. She offered to let him call again, and try the interview over.
In user research, you're not going to offend feuding advice columnist sisters. But there's still tough stuff to cover. There's a way to develop a rapport, ask more neutral questions, and when you feel you're making a connection, ask the one that's a little bit more hard-hitting.
2. Have back-up If camcorder, get tape recorder, tape recorder, get a pen, a pen, get a pencil, because pens don't write on wet paper when reporting or observing outdoors. Also, have more questions than you need. You sometimes need more questions for short interviews than for long interviews. 10 questions for 10 minutes, 15 questions for 1 hour. The pace of the questions mean you need to be ready for one-word answers.
3. Make sure the interview is comfortable Doing contextual interviews for a company testing part of its website, met participants at coffee shops. Unfortunately, only brought own laptop, which was atypical and led to users who didn't even know how to use that laptop. No one knew how to make a mouse click, let alone use the website. So find ways to put people in their natural setting.
4. Your participant is always right Did a user research session where a fellow researcher said that the source was wrong. "I dispute that!" It set the wrong tone, and it means you're not getting the person's actual story, which is what you're there for. People are telling you their truth, which is more important than objective truth.
5. Know everything, but be naive Expertise is good to help you understand what you're seeing, but don't correct your participants, ask dumb questions, ask people to explain what they're doing in terms children can understand. Can actually lead you to greater clarity.
Horror Stories
Dan Turner: One interview I did, an assistant to a fiction editor. We used Wordstar to edit the text. I had the chance to interview a band I had heard on KCRW. I was really psyched to do something new. Went down to the record company, met the band, started asking questions, they felt really bored answering questions. Finally, one of them asked, didn't you read the press kit? I said, "What's a press kit." They give you the given information so they can get onto the new stuff. Know what they expect you to know and be aware of context. Don't know what you don't know.
Dan Turner: A project from this term, doing an ethnography survey for a technology company. A lot of people on the team hadn't done interviews before, so an hour getting consensus on the wording of the questions and their order. As it turned out, the second question never got asked, because it just wound and wound to new stuff. Have more talking points than work in any order. a script is a bad idea. More friendly to the interviewee. You may learn more. Don't try to stop people and take what they want to know.
Dan Turner: Interviewing someone to be the center of this ten top tips article. Went to a coffee house with big tables. Going back and forth with these questions. Someone at the table. A porn star! Context is important.
Pete Mortensen: Asking leading questions is dangerous -- you won't know what the answer means.
Kimra: Project looking at user collaboration and information sharing on campus. Writing up a great interview guide, really making sure we had it right. Just had a refresher that long pauses are good, let people think, don't jump in and let them tell you. Asked a question, awkward pause. Here comes a good answer. Finally: "I have absolutely no idea what your question means. What is that about?" And I froze, maybe these are the answers. I don't get where you're going, I had the long awkward pause. Eventually it was a long enough answer and he was a tolerant participant that we got back on track. One moment on the verge of derailing the topic. Looking back, I could have done more research about his research going in, and more research about the tpoic to have other words at my disposal if I needed to re-word those questions. What about someone else.
Dan: Even more so if not a face-to-face method. On the phone or e-mail when not personal charm to tide it over. Reporting on a novelist, got an author who said, did you even read my work? One time, interviewing Steve Wozniak on the phone and then John McCarthy, the creator of LISP. Woz could only talk about what he was excited about at the moment, which wasn't what I had asked to interview him about. He just wanted to talk about school he was working on, nothing to do with article. McCarthy, a computer genius. Can you explain it like a five year old? "No. You need a doctorate to understand it." Tying back to user testing. Brought up in UI class, being able to talk their language or at least knowing what that language is. If testing a website, or have someone interested in testing it out. You don't want to have only command-line stuff.
Kimra: User research that isn't in journalism. In journalism, you mainly interview experts who want you to know how much their know. Had a lot of interview subjects who say, "I don't know a lot about that. I'm probably the wrong person to talk to." Actually, that's exactly the person to talk to. I just want your thoughts on this. I don't think I know enough to be helpful to you. To know exactly enough.
Dan: Journalists looking for gotcha moments, the opposite of what you want to do. Testing something with nine year olds, ten year olds. How to use a process we hadn't thought of to teach them something else.
Abi: I've been interviewed more than doing the interviewing. Most journalists are not technical. Really hard to walk them through. Do you know what the thing you're talking about is? I got my moment of fame for 15 seconds on MacNeil-Lehrer the day after Thanksgiving. Make sure to record everything, because the data will mean later on. Not just record but transcribe it as well.
Kimra: The timeline for user research is so much longer than journalism, that stuff isn't fresh, and jokes and emotion need to be added in. Journalism is immediate, longer stuff requires a different level of memory.
Pete: What do you do if the interview subject is good at doing interviews? We've ended up in situations where a new research project has the same participants, to the point that we generally try to avoid doing research in Chicago anymore. That's because the point in problem framing phases of research is to get a broad diversity of opinion in depth.
Kimra: It does help in both journalism or research to flat out say "Is there something you wish we had asked about instead of what we thought was important? Widely applicable.
Kimra: I take a lot of market research tests, and I find myself adopting different personas depending on which industry it is. What's the first gym you think of when you think of gyms instead of banks? "It's my bank, and it's in my neighborhood." Why did I actually sign up with those places. I never said I joined Washington Mutual because the ATMs were funny -- it said, "What's up, Kimra?" But I never put that in a survey, because they never ask that?
Dan: Do you ever change your diction when talking to a non-native speaker? Finding different ways to talk around a concept. If it doesn't translate directly from culture to culture.
Kimra: There was a concept in German that came up in English as having "horse sense."
Mike: Does anyone in the entire world have "horse sense." The reverse of that, exists across cultures as horse shit.
Dan: In person, one-on-one, do you want to remain more authoritative, do you want to mirror their feelings and validate.
Kimra: A quick listen in the different kinds of "mm-hmms." One is dismissive, one is "that's so interesting, tell me more."
