Evaluation Methods

To evaluate the user experience of Joka Jobs, we employed a multi-method approach including user survey to gather quantitative data on user demographics, attitudes, and perceptions about job search in general and task-based usability testing where participants completed representative tasks while thinking aloud, revealing pain points and areas for improvement.

User Personas and Journey Mapping

• To gather more targeted and relevant data by identifying key user groups.

• To design usability test tasks that reflect real-world user behavior.

• To gain deeper insights into user needs and frustrations.

User Survey

Tools:

• Google form

https://forms.gle/6vZ5JyM7UtNChm2fAforms.gle

Link to survey google form

Goals:

• Understand the general audience's attitude towards job search.

• Identify their preferences and usage patterns about looking for jobs.

• Identify pain points as well as useful features in their current job search methods.

• Explore gaps and propose new additional features.

Sample survey questions:

The survey used demographics, Likert scales, multiple choice, ranking and free-text questions to gather both quantitative and qualitative data.

1

2

3

4

5

Moderated Usability Testing

Tools:

• User testing: iPhone, Joka Jobs app, iPhone screen recorder, printed consent form

• Observations: Notepad, stopwatch

Information Architecture:

We started with creating a information architecture diagram to understand the different user flows within the app and to help us design the tasks for the usability testing.

Information Architecture Diagram

Test Plan:

1. Location: In person

2. Moderation: Moderated by Aki Matsunaga, Srijan Sarkar (UX Researchers)

3. Participant recruitment: Professional and personal networks

4. Metrics measures: Clicks/Steps and Time taken to complete each tasks

5. Study Objectives - Which are the outcomes expected for this study:

   • General UX evaluation including navigation and layout to identify stability and shortcomings

   • Identify whether the users can accomplish the task

   • Discover what the users found easy and struggled with during the tasks

   • Get feedback on how to improve the product

Test Steps

We followed the below protocol for each test, in order to reduce inconsistencies and make the results reliable

Test Protocol

Tasks

The tasks assessed in this test are as below:

Usability test tasks