Talking Eyes Media
I was part of a team of 4 designers tasked with designing and conducting a usability test on Talking Eyes Media, a multimedia non-profit. The purpose of this test was to discover the most common usability problems for new users of the website, and to use the test data to make recommendations for possible changes to the layout of the website for improving usability.
The test was composed of a pre-experiment questionnaire (seventeen questions), five minutes of free observation on the site, a list of ten basic usability tasks, a set of post-experiment multiple-choice questions (six questions), and a set of post-experiment discussion questions (nine questions).
Most of the participants of this test indicated that they had a generally high proficiency with online tools and websites. The average age of participants was 25, and all participants had at least some college education. The majority of participants were women with a gender ratio of 5:3.
The following if the report we submitted that details the user testing, results and suggestions for improvement.
Our group had a structured set of instructions that each member followed while conducting the participant observations. After explaining the goals of the test and explaining how it would be conducted, we had the participants sign a basic consent form saying he/she understood and consented to being recorded during the test. Throughout the test, the participant’s screen and voice audio was recorded and the participant was encouraged to talk aloud through both the “free view” (a period of five minutes for each participant to explore the website at their leisure) and the usability tasks.
We then had the participant complete a “pre-experiment” questionnaire to get demographic information, as well as a basic understanding of his/her familiarity with the Internet and digital media.
The participant was then given 5 minutes to view the Talking Eyes Media website at his/her pace to get familiar with the site and its content. During the “free view,” the researcher took detailed notes about the pages the participant clicked on and approximately how long he/she remained on each page.
Next, the participant was asked to complete a series of 5-7 tasks testing the usability of the site. The user was given 2 minutes to complete each task. Tasks ranged from finding specific information about the site to “showing” the researcher how he/she would make a donation to Talking Eyes Media.
After the completion of the tasks, the participant completed a “post-experiment” questionnaire and discussion questions about the experience to gauge additional feedback on his/her views about the website.
Our three main goals for the test were as follows:
1. Derive clarity about where users are getting lost while browsing talkingeyesmedia.org.
2. Figure out crucial information users want to know about the company.
3. Provide helpful recommendations for the future website redesign.
The users tested for the Talking Eyes Media study were comprised of males and females between the ages of 19-30. All participants were either currently enrolled in a university, or had completed, at minimum, a bachelor’s degree.
7 out of 8 participants spent at least four hours on the Internet per day. 6 out of 8 participants said they used online media daily to explore topics of interests. The remaining 2 participants said they used media 2-3 times per week for the same purpose.
The results of the sliding scale questions showed that most of our participants used the Internet very often to get news and had fairly advanced computer skills. They also used the Internet fairly often for online shopping, watching videos online, and blogging. In general, the population we interviewed could be considered very “Internet savvy.”
Only one participant knew about Talking Eyes Media and had visited the website previously. Some of participants’ favorite places to visit on the Internet included: Twitter, Facebook, Tumblr, New York Times, YouTube, Netflix, Huffington Post, and Deadspin.
In reviewing our test results, we found limitation issues in three areas: (1) the homogenous demographics of our test subjects (2) the inconsistency in the testing environment and (3) the limitations in the sample size.
1.The demographics of our test subjects were similar across the board particularly in educational background and skill level. The majority of our test subjects were female. There was little diversity among our subjects and the range of ages tested was fairly narrow.
2. In terms of the test environment, the majority of our subjects performed the usability test on a laptop, primarily a Mac. We are unsure of how the test experience may have been different if the subject were to take it on a desktop computer, or an alternative computer system, but this trend should be noted.
3. Our sample size consisted of eight test subjects, which we found limited the scope of results. In addition, although we were all working from the same script, we are certain there was some variation in the way the test was given among the four different testers.
• Our tests subjects also noted many positive aspects of the site.
• The majority of our users were enthusiastic about the site’s ‘About’ section with the clear display of clients, as well as an eloquent and succinct mission. Users found the content on the page easy to locate and easy to read.
• The social media was well placed and located where users expected to find it.
• Users found the the images on the site visually striking. The videos were perceived as very high quality and addressed interesting topics.
• One user described the site as being ‘fun’ saying that it wasn’t the usual place he would spend time, but that he loved how much the site “[focus] its product over its branding.”
• Minor problems is defined as problems that 2 or more participants referenced but that did not affect the usability of the site. This also includes problems that individual users found without being prompted in the tasks.
• Three of eight participants said that the “list of services” that Talking Eyes Media provides should be in a more prominent place, such as the homepage. As it currently stands, users had to go to the “clients” and look on a sidebar.
• Five of eight participants said that the visual aesthetics of the site needed improvement. “For a website that features visual content, it leaves a lot to be desired,” said one participant. These users described the color scheme as “drab,” “gross,” and “old school” and said that there could be better font and design choices.
• Participants said that there was too much textual information overall, but that there should be “more foregrounding of textual descriptions of the videos… [to] quickly orientate me in what the video is trying to accomplish.”
• Although none participant said that that there should be more “moving elements” because the site is about video and multimedia production.
• No participants had trouble accessing the blog, two of eight said that the blog should be integrated into the site for continuity. The researchers also agree that integrating the blog into the site would heighten the cleanliness and professionalism of the company’s online presence, as opposed to being sent to a new tab of Blogspot.
• Participants eventually found the “upcoming events,” but it was buried on the homepage and not intuitive. One participant said that “a calendar of events would be helpful” and another described the overall information as being “sometimes hidden.”
• The most common problem participants ran into during the usability test when not using quickly specific pieces of content on the Films and Store pages. While most participants said that the thumbnails on these pages were attractive and interesting, they also complained that the thumbnails did not provide enough information about the content they linked to. This meant users had to unnecessarily click through multiple links to find a piece of content about a specific subject without using the search function. “I felt at times as though I were moving from unexplained still image to unexplained still image,” said one participant. Several participants were completely unable to find a specific video or book during a usability task.
• Participants also complained that the categories did not always clearly explain the subjects of the films or books. One participant didn’t even realize that the separate categories on the side-navigation contained more videos than were displayed on the main Films page. “Yeah I totally missed the whole categories for the ‘Films’ page… maybe it was the size of the images that grabbed my attention and kept it to the right side of the page, ignoring [totally] the categories,” they said.
• The two catastrophic problems that we identified during our usability study were (1) the search function was not working properly on the website and (2) one of the videos said that the Adobe Flash Player was out of date.
• Though the search function worked part of the time, it did not offer results for a variety of terms to identify the videos. The user who encountered this problem became frustrated and had difficulty finding the media that interested her.
• The video that could not be played was the “Aging in America” video. The user could not open the video due to an out-of-date Flash player.
• Need to overhaul the site with a more modern look
• Reorganize site layout to be more intuitive
• Rework the categories for the films page to be clearer, add category descriptions, split larger categories up into more specific smaller groups of videos.
• Rework the featured Films page (the one you get to through the top navigation) to make it clearer that there are more than just the one film displayed for each category of films. Think about adding a row of smaller thumbnails under the featured film to represent other similar films.
• Change the font-size and coloring of the links to categories to make them stand out more and draw people as much towards those pages as the featured videos.
• Add more information under the thumbnails on the Films page. Even a small amount of descriptive text for each individual video would make finding specific content much simpler.
• Integrate blog into the main site so you don’t have to leave the site
• More prominent placement of mission and services- placing this in the “about” page would resolve this problem.
SUGGESTED SITE ARCHITECTURE
NEW TALKING EYES MEDIA SITE