Testing

Complete the Testing Your Assumptions worksheet if you haven’t already. See steps in the first part of this module.

• Invite a diverse group of community members to the town hall. Consider asking influential people in the community to help with outreach. Remember, you are looking for feedback that will strengthen your concept, so now’s the time to extend the invitation to as broad of a group as possible. You should plan to have between 8 and 30 confirmed participants for this type of session.
• Give yourself at least one hour to host this session.
• Choose a location that is easily accessible and familiar to your target attendees. 
• This session can be run either virtually or in-person.
• If done in person, make sure the space is conducive of good interaction. People should be able to move around easily, have good lighting and HVAC, access to A/V resources, etc.
• Prepare materials. If done in person, ensure you have enough materials on each table and for every participant (e.g. at least one marker and one stack of sticky notes in each color per person). Your materials should include:
• Pink sticky notes
• Yellow sticky notes
• Green sticky notes
• Markers
• Concept briefing
• Materials for concept briefing (see “introduce the concept” below)
• Prototypes: During the feedback portion, you will need to provide an artifact that represents the concept. It could be a summary poster that shows key wireframes, key storyboard images of your role play or service journey, communication materials, or the mock-up itself.

Allow participants to sit where they choose. By sitting with friends and colleagues the town hall participants form natural teams with like-minded people. This natural grouping creates opportunity for each member to participate within their team, and each team encourages other members to participate. (Zuckerman-Parker)

• Each team should be no more than 4-6 people.
• If conducting virtually, prepare the breakout feature and assign groups to conduct the reflection exercise together.

• Introduce the agenda and the purpose of the session. The agenda should include introductions, background on the project and/or the concept, format of the activities, introduction of the prototype(s), breakout activities, debrief, and wrap-up.
• Do a round of introductions by conducting a roll call. Ask each participant to introduce themselves and say a bit about their role within the learning community. If you are concerned about time, stick to their name and their title.
• Introduce the project. You may want to include the primary objective, brief background information, and an explanation of how the prototype got to its current iteration.

This is your chance to give a short five-minute briefing of the concept. Each type of prototype can be introduced differently. Consider this an opportunity to introduce the core problem you are trying to solve and how the concept will provide a solution. (Refer to your Testing Your Assumptions worksheet and your Concept Poster from Prototyping.) Some ideas for introducing the concept based on the type of prototype include:

• Concept poster. Introduce the poster. Start by clarifying the problem you are trying to solve. Share the concept you generated, focusing on the key frames and features, and allow time for clarifying questions.
• Role play. Prepare a scenario that demonstrates your concept. Act out the scenario, including key interactions and behaviors you want to assess.
• Wireframes. Share the sequence of key wireframes, describing the key features and the benefits they are meant to create.
• Communication materials. Print out key materials and distribute to the group. Ask the group to review and annotate them. For example, you could pass out highlighters and ask them to highlight the pieces of content they found more relevant or valuable. You could also pass out red pens and ask them to cross out what they find irrelevant. You could also ask them to underline and write questions marks next to content they find confusing or hard to understand.
• Mock-ups. Gather participants around the mock up and describe its key features and intended benefits. If there are important interactions occurring between the different aspects of the mock-up, make sure to demonstrate them. You may also consider telling a story about how someone would use this mock-up, helping those in the session to understand how it would work.
• Service journey. Review the service journey narrative, highlighting key aspects for the individual’s experience.

• Distribute the concept artifacts to each table. These should represent  your concept and provide specific visual cues to the core elements of your prototype.
• Distribute pink sticky notes, green sticky notes, yellow sticky notes, and markers. Make sure there are enough for each person in each group to have a stack of each color and an individual marker. 
• Introduce the activity. “Today we will use the Rose, Thorn, Bud method for gathering your input. First, we will think about the ‘Roses.’ These are things that you like about the concept. Use the pink stickies to capture your roses. ‘Thorns’ are things you don’t think will work, or may cause issues in the community. Use the yellow stickies to capture your thorns. ‘Buds’ are your ideas for improvements. Use the green stickies to capture your buds. We will move through each of these together, so wait to begin until we give you the go ahead.”
• “Everyone, grab a stack of pink stickies and a marker. Take the next two minutes to list, by yourself, aspects of the concept you liked. Use one sticky per piece of feedback.”
• After two minutes, add “Now, as a group, put your stickies on the prototype. Share with each other as you add to the poster. If more topics emerge as Roses as you discuss, feel free to add more pink stickies as you discuss.” Allow about five minutes for this discussion.
• Now move on to the Thorns. Give instruction, “Grab a stack of yellow stickies. Take the next two minutes to capture, by yourself, any aspects of the concept you think will pose a challenge or that you didn't like. Remember, one observation per sticky.”
• After two minutes, invite them to discuss as a group the aspects they wrote on their stickies, and tell them to place the stickies on the concept artifact. This time, add the following instruction, “As you share your thorns, add sticky notes to each cluster that answers, “Why is this a thorn?”
• Now, ask them to reflect on emerging ideas based on the concept. “Grab the green stickies. Are we missing anything? Take the next five minutes to think, individually, about any gaps in the concept or ideas to make it better. Using the green stickies, write or draw a picture to record these ideas.” Once they have had some time individually, ask them to share and attach the stickies onto the relevant areas of the prototype. 
• Instruct each team to pick a spokesperson as they wrap up, and this person will share back details from their group dialogue during the debrief. 
• As you facilitate this exercise, be sure to check in on each breakout group. Check that there are no questions and clarify, if needed. If you see stickies that don’t make sense, or seem vague, ask them to add detail or rephrase.

• Reconvene as a large group. 
• Choose a group to begin sharing their Roses. Probe for details as needed so that you understand why each of their identified features is truly important. Some follow-up questions could be:
• What about that was valuable to the group?
• Who does this feature benefit the most?
• If you had to identify a rose as the most critical, what would you select? Why?
• Invite the next group to add roses. Ask them to share any that are different from the first group, or reiterate elements that the first group shared. Ask similar follow up questions, building on the information shared.
• Do this for up to four groups, and if there are additional groups, open the floor to have a volunteer share any other Roses they want to be heard.
• Repeat this exercise for the Thorns. Start with a group who didn’t speak yet. Follow-up questions for Thorns can be:
• Tell me more, why is this a Thorn for the group?
• Why was this feature so problematic? 
• Who is most negatively impacted by this Thorn?
• What associated challenges might come up as a result of this?
• Be sure to take detailed notes on the feedback here. You want to make sure you really understand why the feature is a problem 
• Now move to the Buds. Invite volunteers to share gaps and opportunities they see. If no one volunteers, choose a group that has many green stickies on their prototypes, and invite them to kick things off. Follow up questions for Buds include:
  • Does anyone have a reaction to that idea?
  • Does this change the problem we would be solving? What does it change it to?
  • Who benefits the most from that Bud?
  • What needs to be true for that Bud to work?
  • How do we make this concept more equitable?
  This conversation will help you challenge your assumptions about what matters most, what the community deems critical, and provide insight into the values of the community, helping you iterate and improve the concept.
• While conducting the debrief, pay attention to each team’s unique identity. There will be perspectives that are unique to each group, but it is most critical to uncover the common themes and findings that cut across all teams.

• Conduct a second roll call (if you have time). Give participants two minutes to write down, individually, something to share with the group as a reflection of the activities. Their reflections could be something relevant that has not been brought up yet, or something that surprised them. This is a good way to have the group reflect on the dialogue and share thoughtful input that might have been missed. 
• If someone does not have anything new to share, they can share something that stood out to them about the conversation. 
• If you are pressed for time, invite volunteers rather than the whole group. If no one volunteers, call on three to four people to share final thoughts before closing.
• Thank the group for their contributions, and share next steps in your process.
• Include information on ways in which the community members can stay involved.

Reflect and record using the Iterating Worksheet. See steps in the final section of the module for completing this worksheet