Ready, Test, Go. brought to you by Applause // Episode 36

Opening Minds with Accessibility Research

Transcript

(This transcript has been edited for brevity.)

DAVID CARTY: Have you ever enjoyed a cruise so much that you totally forgot where you stopped along the way? And I don’t mean because that Margaritaville stop gave you a week long hangover. Jamila Evilsizor had a cruise experience like that, a Star Trek cruise, in fact. And she enjoyed every minute of it.

JAMILA EVILSIZOR: Star Trek has just been part of my life as far as I’ve been able to watch TV. I’d say a question often asked is, what’s been your favorite vacation? And I would say my most memorable vacation by far was a Star Trek-themed cruise. People often ask me, where did you go? And I honestly can say I don’t remember.

There was one day that I did throw on a red shirt of all the possibilities of a uniform that particular day. And then there was some these moments where you actually got to sit in the captain’s chair and just imagine what that would be like for a moment. It was fun.

There was a soundtrack that was playing on loop for seven straight days. And I cannot get it out of my head. It’s not based out of any of the Star Trek series. They made it custom for the cruise ship. We’ll probably just go back just for the sake of getting that earworm done because sometimes it just gets stuck in your head. And you’re like, you know what, I just need to hear it again. Play it out. We’re good to go.

CARTY: Whether it was meeting cast members throughout the ship or breaking down the science of the show itself, Jamila found all kinds of activities to keep her engaged. I know they had a pool deck. I wonder if they had a holodeck.

EVILSIZOR: What I do remember is passing LeVar Burton in the hall, talking to Q in the elevator, playing trivia with the Lower Decks cast, watching just so much Star Trek that it felt like I was really on a Star Trek for seven days. So it was pretty unforgettable.

Yeah, anything and everything that you wanted to get involved with, you could. There was marathons from all the different series just playing the entire time. There was different versions of trivia that you can jump into. If you decided to wear costumes the entire time, there was multiple costume award ceremonies happening.

There were panels where members of the cast would get together and just give us, this is what the scoop was like, this is what it was like behind the scenes, or giving us helpful hints about what was coming up in upcoming seasons. So it was just a little bit of everything. You could pick and choose what you liked.

I would say my favorite parts were they actually brought scientists together to talk about the real science of Star Trek. And so that was really nice to be able to geek out with everyone else, just to see how rigorous they were on trying to make sure that as much science was built in as possible.

CARTY: Jamila is still a Star Trek fan and has rewatched some of the series several times. What resonates with her is the hopeful, unifying message that the movies and the shows put forward.

EVILSIZOR: 100% Next Generation [is my favorite]. I would say, though, formative years was Voyager, Deep Space Nine. And then as an adult, went back and watched Deep Space Nine because I never was able to watch it like at the time that it came out. So it was always a rerun as an adult coming back and really appreciating the arc that was Deep Space Nine for what it is.

So I’ve gone back and rewatched all of those series over and over and over again. I think what’s interesting about Star Trek is it actually left me with a very interesting perception of the way the world could work and the way that we could treat one another. And it’s influenced my value system, for lack of any better phrase, and realizing that we have the intelligence and means to build a better society if we really put our minds to it.

Is it always going to be perfect? By no stretch of the imagination. But we can try and attempt that. And that’s what I really appreciate about the Star Trek series, is what would that life look like. I think that was the power of that series, as it really gave us a chance to realize, reimagine what that future could look like. I would do it again in a heartbeat, definitely.

CARTY: This is the Ready, Test, Go. podcast brought to you by Applause. I’m David Carty. Today’s guest is Trekkie trekker and inclusive design expert Jamila Evilsizor.

Jamila has a versatile skill set in user experience research and software project leadership, contributing to projects at BNSF Railway and NASA’s Jet Propulsion Laboratory. She brings expertise from enterprises, nonprofits, and startups.

She joins us today to discuss her work with accessibility research, a topic reshaping organizational behavior and executive mindsets. Product quality is one thing. And inclusivity is another. It’s a challenging outcome to strive for. Yet, it makes all the difference for those individuals who need a different approach to interact with our digital world.

Let’s hear why user voices are more persuasive than guidelines, basically that whole carrot and stick metaphor, and why, for many, the ultimate goal of a great digital experience isn’t delight. It’s simply having the product work. Let’s boldly go with Jamila.

Jamila, user voices can be very persuasive, especially when it comes to incorporating the perspectives of people with disabilities in a product or experience. So can you tell us a little bit more about how important this perspective is and maybe a moment where this type of real feedback fundamentally shifted how a team saw their product or experience.

EVILSIZOR: UX professionals, they pride themselves on elevating the voices of the people who use their products. But we often forget to include certain voices in that conversation.

According to the CDC, 25% of the population has a disability, which means they’re navigating life differently. They’re using different tools. They have a different set of adaptive strategies that we’re just not considering when we design.

When we forget to bring them into the conversation, their voice is not being heard. So when I started researching with our users with disabilities, we started noticing that stakeholders quickly realize that the assumptions that they were making about these users was incorrect. And they were placing them in boxes that they never should have been placed in.

CARTY: So that perspective is really helpful. And I think real feedback goes a level deeper than maybe what guidelines or audit reports might be able to offer. So how does real feedback really help to fill those gaps and provide some more actionable advice?

EVILSIZOR: We have to think of guidelines as the bare minimum. They’re by no means the definition of what a better experience really looks like.

Let’s take just movies as an example. Movie theaters showed us the latest and greatest films from Hollywood. But then streaming services came into the picture. And they’re bringing those films to our homes on terms that we can appreciate. I can watch it when I want to watch it. I can pause it when I need to get a refill on my popcorn.

Is watching it at the theater a great experience? Yes, it is. But it’s only a good experience if I have the energy to leave the house, if I’m not in immunocompromised, if I don’t have someone that I have to take care of at home. So streaming sort of gave us that chance to partake in the movie watching experience but on my own terms.

So when we think about web content accessibility guidelines, it’s simply just giving us a functional experience in the digital world. It’s not giving us the digital world on our terms. It’s not giving you the nuances of a person’s life.

If someone needs to rely on screen magnification today, but because they have a migraine tomorrow, they need to rely on a screen reader, it doesn’t take into those considerations. We’re never going to know that until we take the time of understanding the perspectives of the people who are using our experiences and understanding how they’re really going to use them in real life.

So that means understanding their patterns of usage. What are all the different technologies that are being used when our experience is in play? And then we can define what an optimal solution looks like.

Even if we do comply — we’re accessibly compliant now. But I have an accessibly compliant app. I’m asking someone to maybe use that app with one hand. But this individual is blind. So that means their other hand is using a cane. But I’m asking them to do something in this physical experience with both hands being used, are they going to be able to do the experience I’m expecting them to do at their best ability with no other hands?

So these are the types of things we don’t realize until we start bringing voices into the conversation and observing and asking questions of how they would interact.

CARTY: So if we’re gathering those perspectives, which is all too helpful, what strategies have you found most effective for enabling that sort of continuous, quick feedback from participants? And maybe how do some other methods fall short?

EVILSIZOR: The most effective way that we have been able to incorporate that on a quick basis is by creating essentially a panel of people that we have available that use a variety of assistive technologies from a variety of disability disabilities who might interact with their experience in person versus virtually in different ways, making sure that they’re available to talk to us in the moment when we might need them.

So that a product manager, a designer, an engineer can engage tomorrow after their stand up session. And they all of a sudden have a question. They can ask something that’s foundational. They can ask something that is about a prototype, whether that’s already been coded or it could just be a hypothetical Figma file. It allows them to get the questions quickly while the product lifecycle is going through.

CARTY: When we spoke before, you shared a sentiment from a user who had conveyed their frustration with an experience. Instead of looking to be delighted and wowed and just blown away by a product, they just wanted it to work. A pretty simple concept. Can you explain some of that context? And does that more broadly reflect the perspectives of people with disabilities when it comes to their digital and analog experiences?

EVILSIZOR: I think it comes back to those guidelines again. Those guidelines are the bare minimum of what is needed for someone who’s not using a mouse or a keyboard the way we would expect them to engage with this digital tool.

And by bare minimum, what I mean is it’s coding 101. It’s design 101. So when you hear stats from WebAIM that says 90% of homepages are inaccessible, that term, inaccessible, means they don’t work. They don’t work for people.

So I was asking a user naively, how do we make this experience more delightful? And their response was, I am just looking for it to work. I asked, is there a competitor out there that really makes this experience shine? Once again, the answer was, nobody really has this figured out yet. No one has it working.

So it almost reminds me of Maslow’s hierarchy of needs. We like to think you have to get the basics down before we can start talking about things like delight. So that’s where we are right now, unfortunately, in the digital landscape, is just trying to make sure that we have a system that works for everyone so that they can just accomplish the one simple task they were doing, which just means that we need to be doing better across the board.

CARTY: Yeah, there’s no shortcut to the top. And along those lines, making sure it works, we want to make sure that this feedback doesn’t just sit in some kind of report somewhere. So how do we make this actionable? How do we turn these insights into decisions that actually drive what design and engineering are doing?

EVILSIZOR: You have to bring stakeholders along for the ride. You have to have them understand what the lived experience is like as much as possible.

So for the next usability session, the next listening session that you set up, don’t craft the user questions by yourself. Have the stakeholders, the engineers, the product managers provide what they want to know. That gives them a vested interest to show up to the session so that they know that there are questions going to get answered.

And when they sit in that session and they see the struggle of a person just trying to accomplish what should be a very simple task, it makes it so much more real. And it makes it so much less of an edge case.

CARTY: Yeah, you’re turning accessibility from something that’s an on paper objective to something that’s real and I would imagine tapping into your empathy. So you’re coming to understand it in a much more human level, which is a great place to get to.

And to that point, accessibility, as you mentioned earlier, seems to mean different things to different people. And even legally, there’s a lot of disparity all around the world between countries and administrations. So it’s not a surprise that there is some differentiation at the team level as well.

So what misconceptions do you see teams having about accessibility that this type of user research can help clarify?

EVILSIZOR: I think the most common misconception I have is that accessibility is only about making apps usable by a blind individual who’s using a screen reader and that those users typically have an assistant with them that they easily can ask for help from.

It is true. That is a portion of that population. It is not the only portion of that population.

Accessibility is about making it usable for a data analyst with a migraine, a court reporter with carpal tunnel, an engineer with dyslexia, an executive with color blindness, a customer service rep with a speech disorder, a deaf graphic designer.

For you, as you age, we’re building things for our future.

Another misconception is that a person with a disability always has an assistant or a family member to help them out if things get rough. And some might but not always.

And even if they did, I want you to think about what you’re asking of a person. Well, if our app doesn’t quite work out, they can always ask for help. Think about your day to day. Do you want to have someone sitting behind you, helping you every single moment of the day click a button, read what this graph says, make sure that your focus isn’t trapped with your mouse?

Everyone wants to navigate this world independently, with autonomy, with dignity, with privacy. And a medical diagnosis does not make that any less true.

So we’re creating these digital and physical experiences. Our safety net shouldn’t be assuming someone is going to ask for help. Our safety net is building an accessible way of working that makes sure the product that we’re creating is born accessible.

CARTY: Autonomy, dignity, and privacy — that’s a great way to put that. I love that.

User voices can be a powerful catalyst. But technology and executive leaders still want evidence to back those stories up. So let’s talk about how research translates into influence inside the organization.

You’ve worked across enterprises, startups, nonprofits. What have you learned about how evidence persuades leaders in different contexts?

EVILSIZOR: I’ve learned it’s very important to become very good at reading a room. I’ve been in rooms where leadership and I are completely aligned on the values of the organization, the mission, and the steps it’s going to take to get there. So evidence in that circumstance was just me doing my job. They trusted the hiring process enough that when I said, this is what needs to happen, we were able to act and move forward.

And then I’ve been in rooms where the leader wants to understand and care about what you’re saying. But they need very clear, concise ways of breaking it down on why it matters and what they’re trying to accomplish right this minute.

So when I say read the room, I don’t mean the room that you’re in that moment in time. I mean the team that you’re on in the quarter of the fiscal year of the organization that is working in the current economy of that time. All of that context needs to be factored in into how you tell the story and provide your evidence because that leader is thinking about that and assessing that every minute of their day.

CARTY: That’s a great point, grounding it in reality, essentially.

When you’re making the case for accessibility, how do you balance these great qualitative stories that build empathy with quantitative data that helps prove the business case?

EVILSIZOR: So qualitative data paired with quantitative data — you have to do them both.

We all know the numbers — a quarter of the population. But we have to extrapolate that a little bit further. Look at how much quicker we’re going to be able to achieve this target by increasing buy-in from y percent of the population by making this investment.

I also like to lead when I just tell the qualitative story with examples of people’s experience. Instead of a 30-second highlight reel of negative feedback from a variety of users, sometimes it’s much more powerful to have someone sit with a three-minute reel of one person doing something that should take 30 seconds to accomplish.

When you make someone sit with that discomfort, they realize the severity of what they’re asking people to go through and the power that we can provide when we invest in a better, more accessible solution.

CARTY: I believe that empathy goes a long way. I’m sure it’s a very persuasive case. You’re an expert at putting together a very persuasive story. But you’re still going to face some skepticism because the realities of the day to day, as we talked about before– there are budgets, there are tight timelines. There’s a lot of work to do.

So in your experience, what kind of evidence has the most power to change minds? Is it numbers, stories, or maybe something else entirely?

EVILSIZOR: There isn’t a magic bullet. It’s all about what matters to the person who’s in front of you in that moment.

If you’re in the middle of a high-speed project where meetings are happening every 30 minutes just to get something pushed out the door — a number, an NPS score, a usability score — and showing that it’s dropping is very powerful. It helps make decisions quickly. It gives you that quick thumbs up, thumbs down signal. That way, people’s brains aren’t overloaded. They just know the direction they need to head.

When we have the time to be visionary, that’s where the videos of users who are expressing that frustration or those moments of delight can be very powerful because you can hear their smile and their frown. And it’s not something that’s easy to forget and will stay with that team as they’re designing, as they’re coding, as they’re testing.

CARTY: The organizations that have the most success with their accessibility programs are the ones who tend to successfully build a culture around the practice. What role does user research play in not only shaping the product or the experience but actually shifting the company’s culture around accessibility?

EVILSIZOR: You have to make it part of your research ways of working.

I’ve seen longer term success with studies that include people with disabilities as part of the sample of the standard user research study than those that are independent one-off usability studies explicitly of an accessible population.

One-off studies solve one-off problems. When they are part of a study that you were going to run anyway, it’s part of the solution that’s going to stand for a long time because it was part of the product roadmap. It was meant to be built in. And it’s part of the product at the end of the day.

It reminds people that are also part of that conversation that these aren’t edge cases. These are people that are not just using our work but relying on these products every single day. And we have a responsibility to make sure that product works for them.

CARTY: So if we fast forward five years — I don’t remember how they did that in Star Trek. Maybe there was a wormhole involved or some kind of device, some MacGuffin, I don’t remember. But let’s just go ahead and fast forward five years. How do you hope in that time accessibility research will be positioned within organizations? And what sorts of initiatives do you plan to push for?

EVILSIZOR: Well, I hope to start seeing the definition of quality rigorous research include people with disability at all times.

I plan to be pushing for education of those who conduct research, whether that’s on a full-time basis or it has been democratized to their role, and making sure they understand the principles and ways to conduct that research in a responsible, ethical, and meaningful way.

CARTY: Jamila, lightning round questions for you quickly here.

First, what is the most important characteristic of a high-quality application?

EVILSIZOR: A WCAG compliant one. So Web Content Accessibility Guidelines — you have gone to the latest version, 2.1, 2.2. And you’ve made sure that you pass at least A and AA.

CARTY: So crucially important. What should software development organizations be doing more of?

EVILSIZOR: Talking to users who use assistive technologies.

CARTY: I love that. And what should software development organizations be doing less of?

EVILSIZOR: Launching a product before ever talking to someone, particularly a screen reader user.

CARTY: They just know we’ll love it, though. They know better.

And finally, Jamila, what is something that you are hopeful for?

EVILSIZOR: I hope in this age of emerging technologies left, right, and center, we start seeing our ability to scale accessible experiences on a quicker, faster level and to more and more people.

CARTY: I love that. That’s a great vision to end on. And Jamila, thank you so much for joining us. We appreciate it.

EVILSIZOR: Thank you for having me.