Building Sam’s Club's Inclusivity Program
The vision of a world-class inclusivity and accessibility program should be to design and create experiences that everyone can use. However, many organizations struggle to implement this vision. In this webinar, the Sam’s Club team discusses how iterative changes helped provide the momentum, the evidence and the empathy to make broader changes across the organization and to start an inclusive program.
This blog emphasizes the following key points in developing an inclusive program:
Organizations must make a core commitment to serving all customers and employees
Involving persons with disabilities is integral in the planning and design phases of the software development lifecycle
It’s critical to create regular education opportunities for all teams on accessibility and inclusive design
Learning and resources must be organized and accessible to all teams for ongoing reference
Movement and improvement at any level is success; don’t let perfection get in the way of moving toward goals
Set up designated time to review lessons learned so they may be incorporated into future efforts
Access the on-demand webinar, Progress Over Perfection: A Conversation With Sam's Club on the Tenets of Their Inclusive Program, read the blog below, which covers many key points from the webinar, or watch a few short videos, excerpted from the full webinar featuring our three main speakers:
Jamila Evilsizor, Senior Design Researcher at Sam's Club
April Karnes, Senior Director, Head of Research at Sam's Club
Jason Munksi, Associate Director of Strategic Accounts at Applause
The transcript below has been edited for clarity and length.
Introductions
Jason Munski: Hello, and welcome to our webinar, Progress Over Perfection: A Conversation With Sam's Club on the Tenets of Their Inclusive Program.
My name is Jason Munski. I'm an Associate Director of Strategic Accounts at Applause. I am an older, white male, blond haired, blue eyes, and I might have a big nose, but that's up for debate. I have the big privilege of co-presenting this with Jamila Evilsizor, Senior Design Researcher at Sam's Club, along with April Karnes, Senior Director, Head of Research and Experience Strategy at Sam's Club. I welcome you both. And Jamila, I'm so glad that you're co-presenting with me today.
Jamila Evilsizor: Yes, definitely. Thank you so much for the opportunity, Jason. As you mentioned, I'm Jamila Evilsizor, a Black woman with glasses and a black sweater. I am a senior design researcher, have been for almost three years at Sam's Club and I lead our inclusive experiences program here.
Munski: Excellent. Thank you, Jamila. April, thanks for taking time out of your busy schedule to join us today.
April Karnes: I'm so happy to be here. As Jason mentioned, I am April Karnes. I am the Senior Director, Head of Research at Sam's Club where I've been for about six years. I am a middle-aged white woman with blonde hair and glasses, wearing a black dress and a sweater.
Munski: Fantastic. We will have another three guest speakers as well. We're going to do a little bit of one-on-ones with Jamila leading. Discussing different impact experiences across the organization from products to research, as well as user experience. And that will be Sandra, Bridget, and Jeffrey.
About Sam’s Club and Applause
Munski: Let's do a little background on both Applause and Sam's Club. So real quick, Applause is a world leader in testing digital quality. We carry out testing across all stages of the software development lifecycle, and we help our clients release faster and with confidence. With more than 1 million digital experts registered in our community, Applause delivers authentic real-world feedback on the quality of your digital assets and experience. And we want to ensure that they work every single time. April, over to you for a little bit of background on Sam's Club.
Karnes: Sam's Club is a wholly-owned subsidiary of Walmart. We're a membership-based warehouse shopping club, which started in 1983 by Sam Walton, the founder of Walmart, who really wanted to figure out a way to serve small business owners like himself, who had a different need when they were shopping than just how a regular family shops. We have grown since 1983 and now we are about 600 clubs, mostly in the US and Puerto Rico, but also in some international markets like Brazil, China and Mexico.
Putting all members at the center of their business
Munski: Thank you, April. So, let's talk about why we’re here. Why was this program stood up? If we look to the analysts, we get a sense of the purchasing power of persons with disabilities. It's interesting.
Forrester is saying that people with disabilities spend close to $2 trillion annually. In The Return on Disability report, they say people with disabilities and their friends represent 73% of critical purchasing decisions. And the World Health Organization says about 15% of the world's population lives with some form of disability. So, with Sam's Club obviously being in the retail space, what is your vision to be servicing the people with disabilities?
Karnes: Yeah, we talk a lot at Sam's Club about putting the member at the center of our business. But we can't say those words without thinking about our members with disabilities, whether those are permanent disabilities or temporary. As you're showing here it’s a huge market opportunity, and we would not be serving our members in the right way if we didn't think about how we build experiences to include this group as well.
Sam Walton really believed in leadership through service. And that is a principle that guides our business today in everything that we do. We also talk about keeping the member at the center of everything that we do. For me and for Jamila in the research and experience strategy practice, that's not only keeping the member at the center, but keeping the associates at the center of everything that we do and every conversation that we have.
It's really easy to think about inclusion and really only think about accessibility. And there are legal reasons that you think about accessibility as a business. And there are really defined guidelines to be compliant in accessibility. And so it's a pretty easy, kind of a low hanging fruit area to think about inclusion. But inclusion is really a basic human right. And we knew that we wanted to consider not only accessibility, but also usability, also inclusivity in things like socioeconomic practice, in things like natural and preferred language when our associates and our members want to do business.
Getting the inclusive program started
Evilsizor: So the way we had to approach that was to look at what our landscape looked like. We noticed we did want to make sure that we had products that were accessible. And our digital product portfolio was broken up. There's member-facing products and then there are associate-facing products for our employees. But our accessibility efforts were really focused on member-facing products.
And while that is a really good way to make sure that our customers are getting what they need from an accessible point of view, if we're thinking inclusively and we're wanting to make sure that our associates are at the center of that experience, and that they have the tools that they need in order to do their job, regardless if they have a disability or not, there was an opportunity there for us to focus in on accessibility.
We wanted to make sure that everyone across the organization was thinking about not only accessibility, but those inclusive principles that April mentioned before. So we really thought about how to tackle this from an organizational perspective. We had to think about how to bring in all of the domains within our portfolio and make sure that they're actively involved in this inclusive journey.
We looked at accessibility reviews for the member-facing web experiences and native app experiences. We were performing inclusive usability design studies on them and were remediating any accessibility bugs that we saw. But those actions were very reactive in nature. What if we were proactive? What would it take for us to be proactive and shift our considerations of inclusivity a little bit further to the left in the software development lifecycle (SDLC), rather than doing it at the end? Who can we start involving?
So, that involved not only assessing the existing accessibility gaps that we had, but also how to change the ways that we work so that we stay on top of the changes that are coming out. How do we do it in a fluid and efficient way, building inclusivity along our journey.
We looked at the teams that are focused horizontally, that are supporting all of our products at all times who have an eye on every single aspect of the products. We made sure that we're incorporating inclusive practices, so that we have sustainable components that have been tested for accessibility, that have been coded with accessibility in mind, and are inherited by all of the products across the entire portfolio.
We made sure that we recruited members with disabilities, associates with disabilities, and brought them into the conversation at the beginning of our SDLC. And getting their feedback from the start so that as we're creating our user stories and we're designing our experiences, and we start testing what we've created, their voice is constantly being heard. And making sure that voice that we're using when we create these experiences is inclusive in nature and not ableist, or discriminatory, or biased in any way.
But what we were seeing was we would come in, we’d do those audits and then maybe a few months later, we'd see maybe a couple of bugs that we talked about before creep back in. And so we didn't see as much engagement with the concept. So, what we realized was we need to start developing empathy. We need to start helping the people that are creating these digital experiences understand what our members are going through. Seeing exactly what they're experiencing, understanding what those pain points are, and then developing a system to change the ways that we work.
Disabilities workshops, knowledge base and creating accessibility advocates
Evilsizor: We started members with disabilities workshops that were based on these snapshots of who we saw, different members interacting with our digital experiences in different ways, and we brought them in. And we had one-on-one workshops with them, where our experience teams heard and saw them interact with our digital experiences on the web, our mobile app experiences, and heard from their own mouths what our members were struggling with, and what they liked about our design, and what we should continue to doing to continue making it an accessible experience.
We took those learnings and started developing an inclusive knowledge base supporting our associates through weekly accessibility office hours, having a channel that documented the resources, a catalog of all of these videos that we had with our members with disabilities, so people can go back and continuously reference them. We started building a video library on this resource site, so that people can go see, oh, these are the ways that we need to be incorporating accessibility from day-to-day.
Here's how an assistive technology might interact with our experience, and how do we fix that? How do we address those needs? We built this into knowledge and education from the beginning. And when we started doing that, we started noticing something happening among our associates that were creating these experiences. They started becoming more engaged. They started asking more questions. They’d say “I've never actually used a screen reader. I would like to be able to check my experience before it goes live.”
We created an Accessibility Champion series in order to help train advocates that wanted to take accessibility a step further and be that voice on the ground in their projects to make sure that the message is being spread. They also help everyone on the team keep track of the resources that we had and take advantage of them.
We started doing testing on our associate-facing applications, and bringing our associates feedback from our research and our user interviews, and letting the teams know this is what they're saying that we need to make more accessible experiences. We started exploring, how do we scale up accessibility testing throughout the entire life cycle, throughout the entire year and not just as a once, or twice, or four times a year audit? How do we continuously do this testing so we're always on top of things?
We hope that these activities take us into the future, where we are influencing policy changes that we're influencing other departments within our organization as well to consider accessibility long-term. But one thing I just love about this journey is just seeing the engagement increase and increase among our team members. Because at the end of the day, we are seeing that engagement, because empathy is driving that clarity of purpose. It's helping people see that we are all equal: Our members, our associates regardless of if they have a disability. If they have to rely on an assistive technology, they deserve equitable service. They deserve equitable tools in order to shop or work here at Sam's Club and in the way that they need to be able to do.
And so through these activities based in empathy, we're seeing our culture being influenced, we're seeing improvements over time, and we're seeing different areas within the organization break some habits that they continuously have and start getting on to this inclusive journey along with us.
Munski: That's great, Jamila. My thoughts here are we talk about the tenets of the program and I think that this articulates it very well. The clarity through purpose, trying to influence culture, and then using the evidence of empathy to drive both. I think that's especially powerful.
One-on-ones with members of the Sam’s Club team
Munski: Let's check in with our guest speakers. We have Jamila doing some one-on-ones with Sandra, with Bridget, and Jeff to talk about how the associating activities have impacted their day-to-day. So Jamila, take it away.
Evilsizor: Let's go ahead and start our conversation off with our group product manager, Sandra. Which sessions have you attended as part of the inclusive experience program at Sam's?
Sandra Tyran: I attended the accessibility training sessions earlier this year. And then I also sat in on one of the Voice of the Members with Disability sessions, the one about blindness. And then while I haven't gone to the office hours for accessibility, my designers have.
Evilsizor: What drove you to attend?
Tyran: I just wanted to figure out how to get started implementing accessibility in our products. What are those first things that we need to tackle? And then the voice of the member session in particular was really useful for me because I wanted to know what the user experience looked like. Because, without understanding their journey and how they're using screen readers, I don't know what pain points they're going to face, and how best to incorporate accessibility into my products.
Evilsizor: What would you consider your biggest takeaways from the sessions?
Tyran: Oh, just seeing how they use the screen reader. So using the H key to navigate through the header. So, then I know, OK, if that's how they're navigating, then how do we put headers in, as well as the workarounds? So the session on blindness when they tried to click on the cart button, when they hit “enter,” it wasn't actually opening up the cart. So they had a workaround to include a virtual mouse so they could click on it to open. We just need to make sure that our products are working according to how screen readers work. It was also really interesting to see how fast a screen reader will read the content. For those of us not used to it, it’s amazingly fast.
Evilsizor: How do you feel your ways of working have changed since you've attended the sessions?
Tyran: We've incorporated accessibility within our workflows. So whenever we're creating epics for functionality and new features, we always make sure there is an epic for accessibility. So we can build that in as we're developing our features. And then for us, it's taking it one step at a time because building out full accessibility, it's a lot to build in. So we're starting with the things that you've recommended like headers, and buttons, and links.
And then also what I found really helpful is attending the accessibility office hours. So whenever our designers have questions around how a particular flow works, like a tab order on a table, we can always go to you for office hours, and then get some help and guidance on that.
Evilsizor: Thank you so much for your insight, Sandra.
Let's hear from a fellow colleague of mine —a design researcher who's involved in our user experience teams. Bridget, thank you for joining us here today. Which sessions did you have a chance to partake in?
Bridget Hart: I attended a few of the Voices of Members with Disability sessions. ADHD, quadriplegia and low vision. And then I also have been participating in our Accessibility Champions project group.
Evilsizor: What were you hoping to gain out of those sessions?
Hart: I was really interested to hear a direct perspective from a person with a disability. I also wanted to learn more about how we can improve our work here at Sam's.
Evilsizor: Gotcha. And I heard you throw in the Accessibility Champions program on top of going to the Voices of Members With Disability sessions. What was that like for you?
Hart: It was a great opportunity to learn and improve my skill set-related accessibility. But also, I wasn't going to be doing it alone. That was very appealing to know that there was a group, a cohort and we'd be going through it together. And that's been one of the great things about it, learning as a group. We get to ask questions, we're encouraged to try out the technology and make mistakes, and then practice.
I'm a kinesthetic learner, so as someone who really appreciates a hands-on approach, getting to try out the tools, but then to also have follow-up with our group has been phenomenal for learning and really getting a better understanding of tools that our members with disabilities might be using.
Evilsizor: Was there anything that surprised in all of this?
Hart: I think with Accessibility Champions, specifically, I was surprised at just how much the technology could do. I knew there were some of the things that we looked for, and I have a friend who's blind and I've seen her use the technology. But to actually use it myself to see how specifically she could drill in and navigate was pretty eye-opening. There's a lot to learn.
Evilsizor: Gotcha. Do you feel that your approach to research has changed at all after attending?
Hart: Definitely. Even for projects where there isn’t a specific accessibility ask, I'm making sure to just keep it in the back of my mind. I might be checking for things or call it out that we want to check for accessibility considerations. And also, I see it as my role now to just bring awareness around what our resources are, where people can reach out if they have questions and the mindset of accessibility not just being an add-on, this is something we need to be doing.
Evilsizor: Well, thank you so much for being such a strong accessibility advocate.
So let's transition the conversation a little bit and talk to a people leader within our organization. We have Jeffrey, who is a director of user experience here with us today.
So tell us, which sessions did you have a chance to attend out of the inclusive experience program?
Jeffrey Lui: I’ve attended the initial accessibility training sessions. And I've also attended two of the Voices sessions. Session one with a member with visual impairment and the second one with a member with motor skills or motor impairments.
Evilsizor: So what made you decide to sign up for those sessions and sit in?
Lui: For the initial accessibility training, I wanted to get a deeper and higher understanding of what accessibility means when we're thinking about design. I think it's common for many of us to think of accessibility as not being able to see, not being able to hear. But it's so much more than that. There's so much to learn, even the way we communicate. Had I applied my learnings appropriately, I would have done the proper introduction. I would have said, Hi, my name is Jeffrey Lui. I'm an Asian male with white and black, salt and pepper hair. Well, maybe more white these days. But I have spiky hair. I'm wearing a white button-up shirt with a black vest. So even the way we communicate can make someone with impaired vision feel included.
Regarding the Voices program, I attended the sessions because there's so much value and learning to be gained from observing and really listening to our members with disabilities. For example, when designing on mobile with the Talkback or voice assistant feature for members with visual impairment, we can pretend, we can close our eyes, we can put a blindfold on to use, but it's not the same as someone who depends on it and relies on this feature to be functional.
Until we listen and truly observe our members firsthand, we're lucky if we get 30% on how we think they actually use it. So I was hoping these sessions will help elevate Sam's Club designers to the next level. Because when we're designing with accessibility in mind, we end up making products useful for everyone, regardless of their abilities.
Evilsizor: So what did you learn when you attended?
Lui: A lot. There are so many types of disabilities, not just speech, hearing, visual, mobility, but cognitive as well, difficulty concentrating, language barriers and being directionally challenged can affect someone's ability to have access. I think we often think of disabilities as permanent, but these lessons have expanded my awareness to temporary and situational disabilities.
For example, if we go to the optometrist to get our eyes dilated, we'll be disabled temporarily with low to blurred vision. As experience designers, one of the core value principles we apply or hope to gain is having empathy. I think one of the biggest takeaways or learnings from these sessions is heightening one's level of empathy. Observe your members with disabilities and see how they interact with your product.
Connect with your members better, get a better feel of the emotional frustrations or challenges of the pain points they might experience. Going back to the example of getting your eyes dilated, if we're not used to it and we get frustrated because we can't see or have low vision for two to four hours, that time will pass. But I think greater empathy is needed to help those where that time may not pass, or is indefinite.
Evilsizor: From your perspective as a manager of other designers, how do you feel that your team's way of working might have changed after attending these sessions?
Lui: The team and I are definitely a lot more mindful regarding designing with accessibility and inclusive design in our review sessions. I think prior to the lessons, the questions were more basic. Can you double-check that font size? Can you check that color against contrast ratio?
But I think after the lessons, we go deeper into the interactive discussions. For example, what will talk back, say if the image was selected. If it says something generic like “image one,” then we need to update the description or add metadata to support the image. And designing with accessibility mindfulness is part of the process of inclusive design. So, to use a non-digital example, if we're designing a ramp for someone in a wheelchair, it will also benefit a new parent pushing a stroller, or a worker that is pushing the carts or bins around.
Lastly, accessibility mindfulness is not just solving an issue we see now, it's thinking ahead. Accessibility can occur unexpectedly, it can happen to anyone, it can gradually creep up on us. It's natural that we're all advancing in years and some of us might develop muscle slowness, loss of muscle control, or even develop a tremor. And knowing that we have a lot of members at Sam's Club shopping in their 50s, 60s and much older, we have to consider if that touch target is large enough on the mobile phone.
As our result of our education, we've even enhanced some of our recent testing patterns and experiments, so when we test Scan & Go at the clubs, we replicate a tremor just to ensure that the product will be successfully scanned, so members of all ages and abilities can shop the same way with the same experience.
Evilsizor: Thanks Jeffrey, this is really great to hear.
Lessons learned
Munski: Fantastic insight from your program team. As we look ahead and start planning what's next, we should reflect on the progress that we've made to date. So Jamila, what are some of the things that we wish we would have known sooner?
Evilsizor: I think engaging leadership is key. When leadership gets involved, it creates a surge of momentum. Also, when you’re doing something the first time, you often assume you’ll get it done very quickly, but in reality, things take a lot longer than you think. So having a realistic expectation is important. And, then celebrate success along the way. Appreciate the small wins. You don’t have to finish something to realize you’re making progress.
Also, I think people starting programs like this should focus on quality over quantity. Plan the right amount of time and ensure you have the right resources. That’s what will have people leave the sessions and go off and sell it for you. Again, at the end of the day, it’s about progress. If we do it better than yesterday, it’s a win. Progress over perfection.
Munski: So, conversely, we talked about things that we wish we would have known sooner. But the other side of that is things that we're glad we actually did to continue that momentum. What are some of those things?
Evilsizor: I think that making sure that wherever you're trying to drive these initiatives from are going to support the organization as a whole. Our decision to make sure that these initiatives were supporting the entire organization in a horizontal manner that can scale was a good decision. It's really helped everyone in the organization have a chance to be part of this inclusive journey and conversation. We can't do this by ourselves. We have to have the voice of our members, the voice of our associates being what really drives this conversation. These aren't lecture series, these are sessions where we really hear how our empowered members feel about things. How our associates feel about things. And letting them drive the conversation to help us build that empathy and make sure that we have that appropriate mindset when we're creating experiences.
Munski: What’s interesting about the Advocates program is you do it by role, something that we didn't really discuss earlier in the program. You’re dicing it up between product management, QA and engineering as well.
Evilsizor: Definitely. Because we all have a part to play in this. And so we want to make sure that we have everyone as part of that conversation, everyone sitting at the roundtable and everyone learning about accessibility. Understanding their role to play, and understanding what their counterparts are trying to do as well so that we can all work together cohesively.
Plans for the future
Munski: What's the long-term plan? And I'll aim this question at April.
Karnes: Yeah. So our grassroots effort with our leadership support has gotten us a long way toward where we want to be with inclusive experiences. But we still have a long way to go. And what we feel is that empathy, building that empathy and continuing that empathy conversation is really at the center of everything that we do in inclusive experiences.
There are a few things that we are putting into place for a longer-term view. First, we are building a design studio. It was announced a couple of weeks ago. It will be built in Bentonville, Arkansas. And that is really meant to be a physical manifestation of the design thinking process. That helps us keep the member and the associate at the center of all of our conversations and allows us to do things like have an experience lab, where we can use assistive devices of all different kinds, where we can really integrate hardware and software in a much smarter way to ensure that those experiences are being built to be inclusive.
Second, we are just beginning conversations about an inclusivity maturity model with a third party. That will help us understand where we are as a Sam's Club organization when it comes to how we think about inclusion across many groups.
Third, and as an extension of both of those things, one of the things that is going to be critical for us is continuing that organization-wide influence in conversations around inclusive experiences. So it's really easy for the product development teams to think about inclusive experiences and product engineering designs. We live those daily, right? But our finance team should be thinking about how they are building inclusive experiences. Our HR group, our accounting team, our procurement team, they should all be thinking about inclusive experiences. And so broadening that out, again, using empathy as our linchpin is how we think we're going to get there.
Q & A
Munski: So, as we near the end, let's just do a little bit of a Q&A. We've been collecting questions from some attendees here. Let me just pause real fast to read through a couple of them. Ok, so the first question is about the People With Disabilities workshops. How many workshops did you do and how did you select the individuals?
Evilsizor: We had five sessions. And we went through some of our customer feedback to get an understanding of the different experiences that our members were having and using their assistive technology of means. And so based on that, we created snapshots of our members from differing perspectives, from not only having visual disabilities, but even motor disabilities, those who might not be able to use a keyboard or a mouse.
We partnered with Applause in order to recruit Sam's Club members along those different member snapshots that we created. We brought them into the conversation so that we could actually have a one-on-one conversation, see them actually use our digital experience using their assistive technology of choice, and hear directly from them what was good and bad about the experience and how we might improve it.
Munski: Great. And it seems that Bridget spurred lots of questions in relation to the Champion training series. Jamila, can you give a little bit of background as to the structure of the Champions series? How did you recruit? And how is it going today?
Evilsizor: We decided to kick that off when we realized the inclusive experience initiatives work when being led by a small and nimble team. However, we wanted to make sure that conversation of inclusivity and accessibility still happened when we might not be in the room. And that means making sure that we have members across the organization supporting various digital experiences on the lookout for advocating on behalf of accessibility.
And so we created this program for anybody, any product manager, any designer, any engineer, any researcher, anyone who's involved with creating a digital experience so that they can learn about accessibility, trained on how to use a screen reader, so that they can explore experiences themselves and get expertise on a critical piece of assistive technology. We’ve worked hard to build this into our ways of working and the responsibilities of every single role.
In addition, we meet at least monthly to have a roundtable to discuss the latest accessibility topics coming up in everybody's respective teams and provide resources that they can take back to their teams, as well as continuous learning opportunities for these groups to share. We want to maintain a supportive area for us to have these conversations grow our accessibility and inclusivity knowledge and expertise for the organization.
Learn more about how to build inclusive design into your organization. Read our ebook, Shift Left and Build Empathy Through Inclusive Design.
