Design of Training Portal for Customer-Service trainees
How I designed a customized training portal for service personnel (customer service and engineers) for best customer experience of Fire TV Hotspot
Note: Because of the non-disclosure agreement, some of the information on this page has been replaced with fictitious ones or has been blurred out. All non-system icons are from https://fontawesome.com.
Project at a glance
The big idea
Make teaching and learning, fun and engaging for customer-service trainees
Biggest benefit
Happy customers!
Engaged customer-service trainees provide better service.
Biggest challenge
1. Finding ways to engage
2. Implementation
Target device
Fire tablet
Project background
Fire TV Hotspot
Fire TV Hotspot allows customers to enjoy Amazon Prime membership while watching shows on Amazon Fire TV stick using Amazon WiFi!
Three levels of trainees
There were 3 types of service providers
Roles involved
Trainee
is being trained as Customer Service agent.
Age: 18 – 25 years
Economics: Lower middle class
Experience: 0 – 2 years
Trainer
is training and updating the lessons
Age: 20 – 30 years
Economics: Lower middle class
Experience: 2 – 5 years
Problem faced:
The product was new, very technical and constantly evolving.
The Customer Service agents not able to ~one-forth of the problems.
This made customers unhappy and they left!
Customer problem resolution chain: Customer Service did not perform well
When customer calls, Customer Service agent picks. If it’s network issue Network Engineer looks into it. If it needs repairs on field, Field Engineer goes to the site.
Types of customer service agents:
They are the first-line of contact for customers.
They will try to resolve most common issues faced by customers.
If it’s a network issue, they pass it on to Network Engineer.
Challenges in existing training system
Instead of blaming Customer Service agents, I interviewed them
Amazon was launching a new product in market for first time.
Initially the existing customer-service agents (from eCommerce team) handled customer calls.
We were losing customers because of poor response to their problems.
Poor training was the issue of poor performance of agents
I interviewed 8 to 15 customer-service agents, of each category, to understand the problem.
Spoke with
In order to
1. Poor performers
Understand what problems they faced
2. New trainees
Get unbiased opinion on what they felt
3. Experienced employees
Understand what they've seen in other industries
And where we can improve
4. Top performers
Understand what's working well and their techniques
5. Trainers
Understand what problems they face
Second round of interviews highlighted that the trainees were not sufficiently motivated to complete the training tasks.
Some of the most common comments on existing training system
This shows that the training system was not sufficiently engaging.
Top reasons for poor learning:
1. Poor engagement
Existing system has usability issues so was less engaging.
2. Poor hands-on learning
Our trainees needed more real-world learning than reading-only material.
3. Dependency on trainer
Existing system was trainer dependent. If the trainer was poor, the learning was poor.
There was lack of attention to an individual hence quality of training was poor.
4. Outdated trainings
The trainers are too busy to update the training frequently because of the clunky interface and processes.
Me and my PM agreed that we needed a new training system
Tablet v/s Desktop
Which system to use?
This is a SaaS product that runs on Fire tablet and on desktop.
I built consensus in our team using the arguments:
1. Fire Tablet is justified for cost, portability and we can easily scale on it.
2. No personal devices needed. Hence it's secure to use Fire tablets.
3. Using Fire tablet helps turn the flywheel for our business
| Fire Tablet | Desktop | |
|---|---|---|
| Price | 5 ✅✅✅✅✅ | 2 ✅✅ |
| Portablity | 5 ✅✅✅✅✅ | 2 ✅✅ |
| Ease of access | 3 ✅✅✅ | 5 ✅✅✅✅✅ |
| Scalability | 5 ✅✅✅✅✅ | 5 ✅✅✅✅✅ |
Using Fire Tablet for the business made a lot of sense
Image courtesy: Verge.
Concept
What would solve this problem?
Simplifying the workflow: Focusing on what really mattered
What will solve the problem?
1. Trainees learn on their own using real world examples.
2. Reduce dependency on trainers
Mindset shift:
Instead of classroom training, trainees will learn on their own, at their pace.
Training would be on-demand and in real time.
Concept of the system:
Training in real-time, using actual call recordings and getting rewards and feedback
Using Job-to-be-done framework
Capturing key requirements using JTBD framework
Users
Trainee
is being trained as Customer Service agent.
Trainer
is training and updating the lessons
Each role had a specific objective "job" to complete.
Jobs-to-be-done framework worked the best for this to communicate the key requirements of experience.
Examples of JTBD statements for requirements
Getting Funded
How I convinced leadership for building a new training system
The leadership was not ready to invest in the new system.
However looking at the better customer retention and lower long-term costs, they were convinced.
Better training > Better issue resolution > Happier customer > Retention
Existing system would lose customers. Hence had high long-term costs.
New training system would train agents to solve customer problems. Hence retain customers longer.
UX Success
Defining what “successful” experience means
Learner’s reaction > 4.5
Trainees rating: Is the training training favourable, engaging, and relevant to solving customer’s problems?
Measured by: (1) Interview with trainees (2) Post training survey
Errors per session < 2
How many times error message is displayed: Number of times trainees faced some usability issue.
Measured by: (1) System log (2) Usability testing
Stars earned > 80%
How many stars did trainee try to earn: More the star trainees try to earn, more engaged they are
Measured by: (1) System log (2) Post training survey
Time to build new lesson < 30 min
How long does it take a trainee to build new lesson? Shorter the time, easier it is and less efforts were needed.
Measured by: (1) System log (2) Post training survey
Ideation
Everything began as hand-drawn sketches
To work efficiently, I presented a number of ideas to my PM and developers to check feasibility.
I finalized the concepts on paper itself.
Time spent on designing in Sketch was reduced because of this.
Sample of sketches I ideated upon
I used sketches to evolve concepts and present to development team.
Killers: Power and competition
Features:
(1) Leader-boards (2) Fastest speeds
Achievers: Win points
Features:
(1) Provide "stars" (2) Show achievements
Socializers: Interacting with others
Features:
(1) Learn together (2) Help others
Explorers: Explore and find hidden items
Features:
(1) "Easter eggs" (2) X-ray features
Engagement: Something for Everyone
How I designed the features for better engagement following Bartle’s taxonomy
The final product was a combination of different methods.
After several iterations and working with Product Manager and Tech, I was able to implement the learning system.
I used the newer AWS design system which improved the appeal of the product.
How different personalities are motivated
Designed using AWS, this SaaS product combines different mechanisms to engage different personalities.
Reducing dependency on trainer
Self-serve learning experience using AI
We recorded a lots of real calls.
I reached out to AWS Transcribe team to build the intelligence.
An AI would "teach" where the trainees can improve their response.
Advantages of this system:
1. Improved engagement
2. Reduced dependency on the trainer
3. Allowed self-paced learning
AI based, real-world call simulator to train trainees
I designed this system to simulate real-world call and how trainee would handle it in real-time.
The AI suggests improvements and a chance to earn more points.
Improving Usability testing
Usability testing revealed some improvements
The patterns were new and I needed to make sure they were usable enough.
For the response screen (shown below) I asked, as an example,
Example 1. “Looking at the screen, what was the outcome of the test?”
Asked to: Test how fast the users can find the “Pass” chip.
Expected response: “Result was pass with 33 out of 37 Stars earned”
Result: Most users had to look all the way past the title to know this. This took time.
Update to design: The users already opened this page, so knew the title. I placed the upper right corner.
Next round UT: Users spent less time finding the results.
Example 2. “Speak out what do you see in the area below the title card?”
Asked to: Test how users understand the playback section.
Expected response: (users describe the pause, 10s forward and back buttons, etc.)
Result: Most users were confused by the sound wave pattern. Users were not able to correlate the playback with the transcript.
Update to design: (The follow up Q “what would you expect?”) I replaced wave-pattern with a familiar seeker with tappable markers. Users can know exactly where they need improvement. They can listen to what they said.
Next round UT: Users tapped on the markers and played out the response. They understood where the improvement was needed.
Running usability tests helped improve the design
I updated the seeker interaction since trainees could not locate where exactly they were given feedback.
Help trainers train better
Allowing trainers to build their lesson plans virtually
Why? Training should scale up with business.
As a fast evolving business, the trainers should be able to update the training with new material.
I designed an intuitive UI to build and edit training plans.
This helps to train Customer Service agents with latest developments.
Simple training builder for trainer
This allows trainers to create and edit training. And cater them to specific trainees.
Bonus: Managing the trainers
Giving management visibility over training program
Helping managers manage the trainers
This was not originally planned for. However, I saw how management struggled to maintain the Excel sheets. Hence I integrated all the data to help see how training program is working.
Management can see performance of training program
Using this they can take a decisions for improving performance and program
Getting the typography right
For a service which consumes low RAM, the typography contributes to the aesthetics.
Sample of typography style-sheet
This style-sheet was made a part of REACT toolkit as well. What was designed was developed.
Design updated for low RAM
The app was to run on low-end Fire tablet, so the design needed to be light on graphic and redrawable elements.
I worked with Android Studio in initial periods to check if my designs meet the RAM consumption standards.
Sample of Android Studio showing RAM consumed
I optimized the design for lowering RAM consumption without affecting usability
Accessibility Blue-lines
Design under the hood
I delivered the experience for folks using assistive technologies. It took a bit of learning on how HTML is read by these systems.
Sample of accessibility annotations I created to make the product usable to those use assistive technologies
Measuring effectiveness
Using Kirkpatrick’s Four-level Training Evaluation Model to find how successful we were
Business success: How our system measured compared to older learning system
Measuring UX success
Target
Actual
Learner’s reaction > 4.5
4.8 rating on self-reported surveys and interviews
Errors per session < 2
1.3 average errors per session measured on logs
Stars earned > 80%
94% Stars earned on average from customers
Time to build new lesson < 30 min
24 min needed to build a lesson on median for a trainer
What could've been done better?
- Improved aesthetics: Using AWS-UI had technical limitations. It could have improved using some other wrapper.
- Better support for regional languages: AWS Transcribe supports only limited languages. While these cover most of the conversations it could have improved.
- Better support for pronunciation: The trainees using this may not have English as their front language. AWS Transcribe fails to identifying incorrect pronunciation.
- Better animations: To reduce the tech debt many animations were simplified.