Recruiting users for Interviews & Usability Tests : Step-by-Step Guide

Hello, wary builder! 👷

One of the most frequently asked questions I get at Looppanel is — how do I set up user interviews with my own customers? Response rates are low, no show rates are astronomical — and really we’re all struggling to get those calls set up before the end of a sprint.

After much iteration on my own methods, co-solutioning with teams like InVideo and advice from industry experts like Nathan Lippi, I’ve come up with the ultimate guide to help you find and recruit existing users of your product for user interviews & usability tests.

Don’t worry —this isn’t a high-level, 30,000 ft view. This is a deep-in-the-weeds, play-by-play of all the things you need to know to get those calls set up & get user insights to your team in days (not weeks!).

Ready? Let’s jump in 🚀
‍
👫 Step 1: Align on the Audience
📝 Step 2: Make an Outreach List (and check it twice!)
✉️ Step 3: Decide on Your Outreach Channel
🎛️ Step 4: Set Up the Campaign
⏰ Step 5: Reminders
🧹 Step 6: After the Call

👫 Step 1: Align on the Audience

The first step in any project is defining your goals. In this case, the key question to answer is - who are you trying to interview?
Broadly, there are 2 types of audiences:

• New users: If you’re building a product or feature for a completely new user base that isn’t served by your product at all today, you fall into this bucket. You have to go out and recruit users from ecosystems like LinkedIn / Slack / Discord / Reddit, or go through a paid panel or recruiting agency that can source candidates for you.
  This guide is not focused on this use case! We’ll be writing another one up for how to recruit new users shortly

• Existing users: If you already have a product, 80% of the time you’re looking to speak with folks already using it. In fact, you often need to speak to a very specific segment of existing users, such as:
  1. Users from the real estate industry [a demographic or industry based segment]
  2. Users who have actively used the analytics dashboard [product usage-based segment]
  3. Users who requested a Slack integration [feedback-based segment]

In this guide, we’ll teach you how to find, reach & schedule calls with existing users of your product.

💡 Quick note: To successfully recruit your own users, you may need to rely on support from other teams in your company. For example, you may need access to CRM / support / product analytics data to find the right list of users to contact. You may also need an email automation tool that you sales or marketing team uses, or a in-product survey / support tool that the support or product team uses to reach users.

📝 Step 2: Make an Outreach List (and check it twice!)

Based on the type of user segment you’re trying to reach, you’ll need to access company data and extract the list of user emails that correspond to a specific trait. Let’s go case-by-case:

• General users: Sometimes you don’t need any segments at all! You just need a “typical” user to give you feedback on a part of the product everyone uses.
  1. In these cases, try your best to get a good distribution of users that is representative of your true audience (e.g., if 70% of your users are 20-30 year olds, 7 of your 10 calls should be scheduled with this cohort)
  2. To get started, you’ll need access to a general database of all your users (typically can be found through your product analytics tool or database)

• Demographic / Industry segment (e.g., real estate agents): If you’re defining users you need to speak to by industry, company size, demographics, etc., there are a couple of ways you can identify them:
  1. The easiest route is if you have a CRM or a database where the user has identified their industry or demographics (e.g., Hubspot). To find out what data is captured here, talk to your sales team
  2. Product usage can also be used to define a demographic or industry segment. For example, if your product has different experiences for students and teachers, you can use analytics data to define each segment with a high level of accuracy (e.g., anyone setting up a class is a teacher, and anyone answering questions on a test is a student)
  3. Based on how you define these segments, you’ll need to get access to one of the following datasets:
  CRM data
  Product analytics data

• Feedback-based segment: If you’re looking for users who were unsatisfied with a part of your product or requested a new feature, you’ll need to do one of the following...
  1. Access customer support data (e.g., Zendesk)
  2. Access your feedback collection tool (e.g., Canny)
  3. Talk to your CS / sales team 🤷♀️

• Product-usage based segments: If you want to speak to users who have experience with a specific part of your product so you can test your new design iterations against their expectations, you’ll need to...
  1. Access your product analytics tool (typically Mixpanel / Amplitude) or database and run a search on users who have used a particular product feature. Frequency and recency of use are key - the more recently and more often someone has used the feature, the more likely they’ll be able to provide useful feedback
  ‍2. Once you’ve filtered in your product analytics tool, you can usually extract the email list of these users directly from there (reach out to your data team or product lead if you don’t have permission!)

If you’re emailing users to reach out to them (decide based on the information in Step 3), you may need to export their email addresses & names as a list

• If you’re reaching users by messaging them inside your product itself or using your CRM for automated outreach, you may be able to use a single tool for creating the user list & reaching out to them (e.g., Intercom lets you create user segments based on which you can reach out to them). In this case, you don’t need to export this list of emails from another tool or database.
  
• Either way, make sure your list of users is active on the product (they've used the product more than once, and relatively recently - say in the last 2 months)
  
• Your audience filter might look something like this...

💡 Quick note: Being mindful of user data & treating it securely is really important! If you have direct access to this data, try to anonymize it where possible and delete it from your system as soon as your outreach is done. Depending on the geography of your customers, you may also need to conform to GDPR or other privacy laws. In these cases you may only be allowed to reach out to users if you have their consent to do so.

✉️ Step 3: Decide on Your Outreach Channel

There are 2 ways to set up your research recruitment campaign:

• Inside Your Product (in-product recruiting): This involves setting up an in-app widget that can pop-up for a specific, targeted set of users and ask them to schedule a call with you from within the product itself
  
• By Email: The traditional method - reach out to users via email, ideally using a bulk-emailing product (your sales / marketing team should already have one set up!)

Ask yourself these questions before choosing a method of outreach:

• How big is your user base?
• How responsive are they?
• Do you need a few calls on an ongoing basis or a larger number of calls clustered together?

Here’s our experience with both channels:

• In-product recruiting has a higher response rate from users, but the total volume of users on your product in a single day may be small. For example, 5% of the users who see a pop-up in the product may schedule a call with you (vs 1% via email), but maybe only 50 people from your target group visit the website everyday (⇒ that’s only ~2-3 user calls scheduled). Use in-product recruiting if you:
  1. Want a small number (2-3) ongoing calls every week
  2. and / or have a large unique set of users from your target group visiting the product daily (e.g., > 3,000 users from the target group)
  3. and / or have a very responsive and engaged user base (typically true if you have a strong community or the product is a 'home' for some set of users - e.g., a CRM for sales folks)

• Email enables you to reach a large cohort of users all at once. While most users will not open your email, because you’ll be emailing hundreds or thousands of people at once, you may still be able to set up a reasonable number of calls. Use email outreach if you:
  1. Want to get a larger number of calls in a small time period
  2. Have a small user base visiting the product on a daily basis
  3. Your user base is not very responsive

💡 Quick note: The response rates you see will vary highly based on your product, who your users are (industry, geography), and the type of incentive you offer. Test out both methods carefully for each new segment / product you're doing research for!

🎛️ Step 4: Set Up the Campaign

💬 In-Product Recruiting

If you’re reaching your users inside your product, you can use your existing in-app survey tool (e.g., a standard in-app widget like AppCues or Intercom works!)

Depending on the in-app widget you’re using, you’ll want to set up a way to reach the right users & let them schedule a call with you.

• For example, in Intercom you can directly create user segments based on their actions on the platform and reach out to them using a Custom Bot or Calendly to set up a call
  
  
• On MoEngage you can create an audience segment or upload a list of emails of users you would like to reach out to.
  1. Create a new in-app campaign > inbound > on-site
  2. Select a template (ideally a small one on the side of the page so it doesn’t obstruct the user flow)
  Upload a custom image with the outreach content you need [you can make a copy of **this template** to start!]
  Link your calendly to the button in the widget
  3. Choose a trigger action or upload your list of users to reach only a specific list

• Not sure how to make this work? Reach out to me at kritika@looppanel.com and we can walk through the steps together! 💯
  
  

💡 Quick note: If you choose an in-product messaging campaign, remember to turn it off when your recruiting is done!

📮 Email Recruiting

If you’re emailing your users, get set up with the email automation tool available within your company (e.g., Salesloft, MoEngage, ActiveCampaign, Hubspot) - ask your marketing team if you don't know which one it is.

Upload the list of emails of the segment you want to reach out to OR create a list of users inside the product itself (Hubspot / MoEngage and others let you create user lists based on CRM data or product usage!).

• For example, for Hubspot you’ll want to...
  1. Go to ‘Contacts’ > ‘Lists’
  ‘Create list’ if you’re using CRM data to define your segment (e.g., based on industry information already in Hubspot)
  OR ‘Import’ > ‘Start an import’ if you’re uploading an excel file of data from another source
  
  2. Once your list has been set up, go to ‘Automation’ > ‘Sequences’
  Add your email template as the first step in this Sequence
  Send your Sequence out to your ‘List’ of users (open rates, response rates will all be captured for you by Hubspot!)

But what should that email outreach even contain?

We tested a bunch of different email subject lines, and surprise, surprise - $ is the biggest motivator for users to open emails & show up for calls! After much exploration of templates, testing and iteration of our own, here’s the optimized email template we use for outreach...

✉️ Outreach Email Template

Sender: [Kritika from Looppanel]

Subject: $25 for your feedback

Body:

Hey [First name],

We’re developing [new feature] at [company], and I noticed that you’ve used existing [related feature] extensively in the past.

I’d love to learn more about what you’re looking for with [feature/product name] and let you influence our upcoming features. Basically, we want to help you make better [feature/product name] for your audience!

If you have 15 minutes to chat, feel free to block a slot at this link: [BUTTON: Schedule a call]

As a thank you, we’re happy to provide a $25 Amazon or equivalent gift card for your time [hyperlink the gift card text]!

Cheers, Name Role @Company

P.S. If none of these times work for you and you want to participate, just reply with whatever time works better for you =)

💰 On Incentives...

I know offering incentives to users for their time can be controversial. Some argue that you have to compensate people for their time, while others claim that money changes the motivation with which users show up for your calls.

In my opinion, if your users are already engaged and really want to provide feedback of their own accord, it’s okay to set up calls with them - after all, you’re trying to make their lives better (hopefully)! If your audience is not responsive to outreach, a little extra incentive in the form of gift cards, free subscriptions to your product, or swag doesn’t hurt. I personally haven’t seen the quality of user calls suffer as a result.

💸 On the Value of Incentives

If you do offer an incentive, think about what $ amount is meaningful for your users. 💸💸💸

For students, $25 is usually a good amount of money, whereas for ultra high net worth individuals or extremely busy professionals, it may not cut it. In the latter cases, see if you can offer more money or provide something else of value (exclusive content or resources that only you can provide, expertise that your team has, or swag).

📆 Optimizing Your Calendly

Calendly is a great way to set up user calls without all that back and forth! But even then, you need to optimize your calendly for research recruiting. There are 3 specific changes you should make:

1. Open up as many time slots as possible on your calendar so users have multiple options to choose from
   
   
2. Reduce the number of days available to book slots in (e.g., just the next 3 days). This is so that calls are booked while your study is active & running (and not 2 weeks out). It also increases the likelihood that your users will actually show up after booking a slot
   
   
3. Ask your users for their phone number when they book a slot in case you don’t already have this data (it can be an optional field)
   
   

⏰ Step 5: Reminders

You're not done yet! You still have to make sure people show up to your calls. Even if you get people to sign up for a call, No Shows are a real issue.

To maximize the odds that users will show up, schedule reminder emails to go out:

• 24 hours before the call
• 1 hour before the call

In case No Show rates are very high, you can also text your users a reminder to show up for the call (this is why it’s helpful to get their numbers while scheduling!). The honest truth is, our email inboxes are already overflowing and it’s all too easy to overlook reminder emails.

🕛 Reminder Email Template [24 hours before]

Sender: [Kritika from Looppanel]

Subject: Reminder: $25 for your [Company] feedback session at [XX time]

Body:

Hi [First name],

Thanks for signing up for a call with me!

I’m really excited to hear your feedback on [Company’s X feature] & learn what we can do to improve your experience in the future. Your feedback will directly impact new features we release!

Looking forward to speaking with you [today/tomorrow] at [9am PST] at this link [ZoomLink]

We’ll share your $25 gift card after the 15 min call is complete!

Cheers,

[Sender First Name]

🕐 Reminder Email Template [1 hour before]

Subject: Reminder: $25 for your [Company] feedback session at [XX time]

Hi [First name],

Just wanted to check that you have everything you need to join me for a quick call at [9am PST]!

Here's the Zoom link for our call[addhyperlink]

We’ll share your $25 gift card after the 15 min call is complete!

Speak soon

Cheers,

[Sender First Name]

🧹 Step 6: After the Call

So you’ve just had your user calls and you’re armed with amazing insights - hurrah! 🎉

Now it’s time to make sure you have closed the looppanel with recruitment:

• Did you offer incentives? Here’s how you can pay them out

1. Most US-based customers are happy with Amazon.com gift cards 📦
2. If you’re planning to do a lot of international studies, it may make sense to invest in a solution like Tango Card, Rybbon or Tremendous
3. If you’re doing a one-off international study, here’s an airtable list of gift card options for countries across the world (we’re growing this list so feel free to contribute by emailing other countries / gift card options to kritika@looppanel.com!)

• Make sure you turn your recruiting cycle off!

1. If you’re using an in-app tool, discontinue the campaign
2. Turn your Calendly link off and make sure to put a message saying your research campaign has ended for any users who open the link at a later date

Congratulations! You just ran a successful User Research campaign 🎉

You’re one step closer to building a product your users truly love!

Have any tips, tricks, or feedback to add to this guide? Email me at kritika@looppanel.com to share your learnings with teams all across the globe 🌎