This is a guest post by Dan Davis, a product manager who works with us and recently conducted remote user interviews for a client project. He was gracious enough to write up what the experience was like and some of our lessons learned. Thanks, Dan!
Design sprints changed the way we work. The benefits are worthy of a post themselves, but one inherent challenge is that preferably they are an in-person venture. The concept largely depends on a group of people hunkering down in a room together for a week or so to roll out a marble slab full of big ideas and slowly whittle those down into a statuesque prototype by week’s end. One of the key pillars comes from user interviews. It enables us to stress test ideas outside of the bubble of our project team and actually answer the question of “how would real users react to this?” During COVID, we’ve transitioned to an entirely remote workforce, and thus entirely remote design sprints. We liked the results, despite some of the challenges. As we continue to refine our practice, we tried a new method of conducting user interviews recently.
Much of the value in a good user interview comes in making the subject feel comfortable and empowered. Quality insights typically come from people who feel like their input is valued and their expertise is recognized. Easier said than done. It’s a lot of soft skills work, and that’s simply easier to convey sitting 1:1 in a room together. So we set out to determine if we could conduct quality interviews at distance. The answer is yes. Here’s how we did it.
We gathered our ideas, storyboarded, and whipped together a prototype in InVision, all pretty similarly to how we’d normally conduct these practices. Once we had our materials ready, we went out and recruited someone to actually conduct the interview. Ideally, this is a curious person. Product experience helps, but isn’t essential. Again, this piece is primarily about empowering your subjects. With that person in place, we set up an introductory call to drive through our materials and make sure the interviewer understood the scope of the project, the audience, and some key areas we held open questions on.
While recruiting an interviewer, we were also recruiting subjects, looking for professionals with expertise in this subject area that were open to give 30 minutes of their time for an Amazon gift card. We did this by posting a basic Google survey on our social media channels and reaching out to folks in our network to find people who may be interested.
During the introductory call with the interviewer, we reviewed the group that signed up for interviews and selected a group of five subjects with diverse backgrounds: differing genders, races, levels of experience, and areas of expertise. We e-mailed the subjects and scheduled Zoom video calls via Google Calendar within the same week. We asked the participants to arrive 5 minutes early and to be on a computer capable of conducting a video conference.
We conducted interviews via Zoom. The interviewer opened with some small talk and a few introductory questions about the subject’s history and experience. After a few minutes, we’d ask if they would like to take a look at our prototype, and we’d share the InVision link with them and ask them to screenshare.
Largely, this went off without a hitch from a technical perspective. A few folks weren’t as fluent in Zoom, but in a year where much of the world transitioned to some form of video conferencing, most were at least somewhat familiar with some platform. Zoom’s interface is simple enough so it wasn’t difficult to direct if they weren’t familiar. Often, more difficulties came in them opening the link and then losing the Zoom window. Our Zoom is set to automatically record everything to the cloud. Rather than lead introductions with the awkward “ask permission to record” verbal exchange, we made sure it was abundantly clear in all of our disclosures ahead of the interview.
Once the prototype was open and they shared their screens, we drove them through the scenario and reminded them that this was still a work-in-progress prototype, so not everything would be working as you would expect. Depending on comfort with technology, some users would immediately begin clicking around and exploring without much prodding. Others required a little more prompting and leading questions. InVision mostly worked fine, though we ran into a few hitches with using the tour points and having a user click into a dead end. This could probably be resolved in how we set it up in the future. One limitation from InVision’s platform is that even their public view enables commenting. One of our interviewees incidentally clicked the commenting icon and became confused.
In either scenario, we found this successful in extracting the type of good feedback we’ve come to expect from past in-person interactions. Ideally these sessions last around 30 minutes, but sometimes can vary depending on how talkative the subject may be. In a remote setting, a little extra time may help, just to help compensate for the setting and allowing people to grow comfortable.
Since the sessions were recorded, it was easy to review and take notes asynchronously. The next week, we regrouped as a project team and shared notes and identified themes we saw across the interviews. The recordings are also a nice artifact to have if we need to refresh on a particular piece of feedback.
Did it work?
Overall, yes. We’d experimented with some different user feedback mechanisms in past projects and felt this delivered much more valuable insights. There’s a few challenges, though.
When we conducted them in person, all interviews would be held in a single day, whereas we opted to conduct these remote interviews over the course of a full week. So there's a bit of a time/efficiency loss.
There’s an extra layer of technology hurdles, since the user now is responsible for navigating their video conferencing software and opening the prototype link. Not a major issue, but a bit more complicated than simply opening the project and putting it in front of someone for them to immediately interact with.
Finally, the distance factor is a little bit to overcome without in-person interactions. It’s much easier to convey warmth and comfort in person than over a video conference. Most folks warm after a short period, but definitely something to consider.
As a 100% remote team, this practice will continue at a distance, so we’re glad to learn we still found great value in the process and gained a ton of actionable insights.
Interested in running a design sprint? We'd be happy to facilitate! Get in touch.