Tutorial Usability Report

Word 2001 for Mac is the Macintosh version of Microsoft Word, a widely used word processing program which is used to create basic documents such as letters and memos, and also more complex documents such as newsletters and brochures for the more advanced user. Word allows the addition and basic manipulation of elements such as: graphics, tables, and charts to documents.

The document tested was the first draft of Tutorial: Shading a Table with the Formatting Palette (Mac). The document provided to the test participants is four pages long and included also included a fifth page with basic a basic Formatting Palette overview for reference on the palette’s Borders and Shading and Table Cells sections. After testing the first participant it was noted one line of text was the wrong font, and the step for changing the border color had been left out . Corrections were made to the document before the second participant. Changes have been highlighted and noted on the second version after testing was complete to show what is different between the two versions.

Testing took place March 7th and March 9th of 2003. Test participants were asked to come in and test the Tutorial, go through the tasks requested, and give feedback relating to the clarity of the document and any part of the document that could be improved. Test participants were given verbal instructions to complete each task requested and were told that they would be timed. Test participants were also told that testing was being performed on the tutorial, not the test participant and that they would remain anonymous. Test participants used the author’s Macintosh computer to insure the same version of Word was used to test as was used to write the tutorial to ensure greater reliability of test results. A blank document was open for the participants and the values under Document in the margins settings were changed. The Formatting Palette was open on the right side of the screen.

Both participants had at least some experience with a Macintosh computer though their primary computer use is with a PC. The first participant has participated in a usability testing scenario once before this test on other projects and understands the purposes of a usability test. The second participant has ___ participated in a usability test before.

The objectives of the test were to have the user follow the steps in the tutorial completing the tasks requested and to determine the clarity of the steps, explanations, and graphics provided for each task. The four tasks requested include: Margins, Basic Table Layout and Entering Table Information, Shading Your Table, and Coloring the Table Border.

As stated above, test participants used the author’s Macintosh computer to ensure the same version of Word was used to test as was used to write the tutorial to ensure greater reliability of test results. A blank document was open for the participants and the values under Document in the margins settings were changed. The Formatting Palette was open on the right side of the screen.

Test participants were provided with a copy of the first draft of the tutorial. The document consists of four pages and a one page overview of the Borders and Shading and Table Cells sections of the Formatting Palette. The test evaluator used the Procedure Test Form to record information about each task and the time it took to complete each task. The test participants progress was rated upon their stating completion of the task and participants were then asked to rate the following: steps, graphics/screens, explanations, and terms. Participants were also asked to provide any other feedback they felt would be useful.

The test evaluator observed the test participants as they proceeded through each task. Any notes on participants’ activities were taken on the Procedure Test Form for each task. The test evaluator timed each task, recorded each participants’ ratings and feedback, saved the final table for each test participant, and set up the document for the next participant.

Participants had various difficulties in completing the tasks assigned. Both participants said they tried to read ahead and the it wasn’t the tutorial. Results are provided below with the times and notes for each task below the table.

Participant 2: 7:30:30 to 7:31:45

Participant 2: Used the Format > Document not the Formatting Palette to setup table.

Participant 2: 7:33:20 to 7:40:45

Participant 1 & 2: Chose AutoFit before entering information. Participant 2 became frustrated when the table looked different than the example under Basic Table Layout. Research by the author after testing to recreate the error indicated this came from choosing AutoFit to contents rather than Auto in step 3 under Basic Table Layout.
Participant 1: Steps comment: Doesn’t say how to enter text.
Participant 1: Graphics/Screens comment: Show how to find AutoFit. All of the icons look alike.
Participant 1: Clicked from cell to cell and used keypad for numbers.
Participant 1: Suggested adding an explanation indicating to hold down the shift key to select both columns at once.
Participant 2: Steps comment: Didn’t realize the Formatting Palette was open on the right side of the screen.
Participant 2: Went to font and scrolled down to choose Times New Roman (toolbar not palette)
Participant 2: Clicked center in the toolbar across the top of the screen to center text again not using the Formatting Palette to perform a task.

Both participants were told to hit enter to bring Advanced down to the second line of the cell when they questioned why it looked different. Both participants were also told to not worry about typos in the first column when they became frustrated. This helped relieve frustration and encouraged the participants to continue the tutorial. Entering the numbers was not a problem for the participants.

Participant 2: 7:42:35 to 7:46:00

Participant 1: Clicked the fourth row, Student/Retired first then chose the third row, Nonmember
Participant 2: Chose Pattern color not Fill color, never figured what they did wrong.
Participant 2: Somehow has gray bar under table probably had the cursor underneath the table when selecting the Pattern or Fill color under shading.
Participant 2: Went back after timing was stopped and changed font weight to bold within 10-15 seconds.

Participant 2: 7:48:50 to 7:50:15

Participant 1: Chose the wrong border, but went through again and succeeded in choosing the correct border.
Participant 1: Steps: When evaluator asked what color the border was after test completion participant said it was black. The realization was made by the evaluator that the step indicating to change the Border color to Dark Blue had been left out of the document. Participant was reassured that this was the author’s error not the participants. This step was added before test participant 2.
Participant 2: Graphics/Screens comment: The icon to the side helps to clarify things. Suggested noting the location of the outside border in the palette because they all look the same.

Information from test results will be taken into consideration and the appropriate changes will be made. Changes noted by the first participant, changing the font to the correct font on one line, and adding the step on changing the border have been made as noted above. Other suggestions and minor changes such as condensing the header text to close spaces between words and adding a table of contents will be made as well.

To:	Dr. Barker
From:	Robin
Subject:	Tutorial: Shading a Table with the Formatting Palette (Mac) Usability Testing
Date:	March 11, 2003