Social Benefits Tribunal Appeals Process
The Social Benefits Tribunal (SBT) is an independent body that hears appeals from people who disagree with a decision on social assistance from the Ontario Works or Ontario Disability Support Program.
While I was interning at Experience Design Lab , an in-house research team in Government Ontario, our team was involved to conduct user research to review the appeals process from the perspective of different parties invovled.
PROJECT CONTEXT
- Explore the experiences of different user groups involved in the SBT appeal process, to identify successes, pain points and areas for improvement.
- Duration: May 2024 - Dec 2024
- Team: SBT Team (Client), Design Lead, Senior Designer, Experience Design Co-op x 3 (incl. me)
- Research objectives and Interview script development; Participant recruitment, communication & scheduling; Interview facilitation; Data analysis and reporting.
Highlights of my experience
I conducted 15 interviews across 7 user groups, including SBT internal staff, appellants and respondents.
I created the interview script for appellants, their legal represents, and appellants.
I handled the logistics side of the research to make sure it aligns with the coordinating the internal team's time
I analyzed data through journey mapping mapping and facilitated the report-building process.
RESEARCH PREPARTION
Understanding the context: Desktop research and co-working session
We began by conducting desk research, reviewing the website of the SBT and the process flow chart to gain a thorough understanding of their process.
Unlike our usual approach, we adapted to the fact that SBT did not frequently use Miro for collaboration. Therefore, we primarily utilized Miro for internal note-taking and organizing our thoughts, rather than for co-working sessions.
Revise and re-revise the user research plan
Given the sensitivity of the research topic, we carefully crafted the wording of the user research plan. After the review of client team, we also dedicated multiple meetings to address each comment. This involved a balance of accepting suggestions, making slight adjustments where needed, and seeking clarification when necessary.
Despite the back-and-forth, our collaborative efforts led to a research plan that everyone supported.
Developing the moderator's scripts
I worked on scripts for appellants, their legal representatives, and respondents. I developed these scripts by reviewing relevant questions and keeping our broader research objectives and user group-specific goals in mind.
Since the scripts were created iteratively, I ensured to revisit session notes from previous user groups, incorporating any follow-up questions or areas we noted for deeper exploration.
RECRUITMENT
Targeted Outreach & Follow-Up
I led most of recruitment for this project (5/7 user groups). While most of them went smoothly, I encountered some challenges with legal representatives. Initially, the screener received limited responses. After a week, I took the initiative to follow up with a message encouraging recipients to forward the sign-up sheet to colleagues who might also be interested, after consulting with the team.
As a result of this targeted follow-up, we received responses from several additional participants, including some who weren’t on our initial contact list, and we ended up talking to 8 legal reps!
Streamlined Team Scheduling for Sessions
With five members in the team, coordinating schedules can be tricky, especially with other projects' meetings and rescheduling requests.
Initially, I managed this by checking team members’ Outlook calendars to find availability. Later, I created a shared calendar where each team member could block their unavailable times, allowing for more streamlined and efficient communication.
INTERVIEWS
Demonstrating dry-run with stakeholders
To make partner team felt more comfortable with our interview process, we scheduled dry-run with them. I took the lead in facilitating this demonstration, with one client acting as an internal staff participant. I began with small talk to put the participant at ease, adjusted the question and script sequence in real-time, and asked relevant follow-up questions.
The dry run was a success, and the partner team felt reassured enough to inform their staff that the sessions would be manageable, smoothing our collaboration.
Trauma-informed research approach
Given sensitive nature of the topics discussed, I dedicated considerable time to reviewing trauma-informed approach materials, drawing from both internal resources and online materials.
Unsurprisingly, emotional moments did arise frequently during sessions. Through this experience, I’ve gained confidence in balancing empathy with neutrality.
It's ok to have silent moments
During the process of our interview sessions, I received feedback suggesting I allow for more moments of silence and not worrying about going through all the questions.
To address this feedback, I reviewed my recordings and noted areas for improvement. Over the following weeks, I focused on integrating more pauses, writing sticky notes as reminders. I also read articles on the value of silence in user research.
Analysis
Leading the note cleaning
To ensure a smooth analysis experience, throughout the process of the project, I tried to clean up notes no later than one week after the session. When two other Co-op students joined the team, I shared them with my approach by documenting my process and reviewing with them.
I’m proud of this effort-This organization not only supports deep analysis later on but also enables us to quickly pull high-level patterns when updating the client team.
From affinity diagrams to journey map
We divided up the analysis by user group, which each member did two tracks to ensure a comprehensive analysis. For both of my responsible user groups, I approached the analysis iteratively. I first conducted a rough affinity mapping to capture initial insights. In the second step, I used journey mapping to identify themes and map out pain points along the way.
Working on the Report
Organizing reports by user journey
With 7 user groups and 35 interviews, we aimed to balance overarching insights with group-specific findings. For the report, we decided to present general insights first, followed by details specific to each user group. Therefore, I collaborated with a colleague to map out the process and roles of each user group, which guided us in structuring the report effectively.
Coordinating Report Rewriting Under Tight Deadlines
As the deadline approached before Christmas, I took the initiative to organize a meeting and facilitate discussions. During the meeting, we collaboratively built the skeleton of the report slides and divided tasks by user groups to ensure timely completion.
When it comes to actually writing the report, my approach involved quickly writing down all initial findings based on the user journey and then fixing the grammar or formatting later, and maintained timely communications with the team in the comments.
Reflection
This project spanned nearly my entire time working at the research lab, and I’m truly grateful for the opportunity to be involved in such a meaningful endeavor. Throughout the process, I took notes on my reflections and lessons learned. I included the two most important learnings below:
Keeping stakeholders in the loop makes the research process much smoother. Inviting them to sessions and dry-runs, as well as providing updates, helps ensure they stay aligned with the project.
With 7 user groups, we wrote out all our facilitation scripts at once. Instead, we developed each script sequentially, allowing us to incorporate insights from previous user groups into the new sessions.
