Usability test results - Revision history

Jjh at 18:21, 13 January 2008

2008-01-13T18:21:00Z

Jjh: /* Test Results */

2008-01-13T18:12:44Z

‎Test Results

Jjh at 18:12, 13 January 2008

2008-01-13T18:12:27Z

Jjh at 18:09, 13 January 2008

2008-01-13T18:09:10Z

Jjh at 18:05, 13 January 2008

2008-01-13T18:05:27Z

Jjh at 18:05, 13 January 2008

2008-01-13T18:05:10Z

Jjh at 17:59, 13 January 2008

2008-01-13T17:59:30Z

Jjh: first stab

2007-09-21T17:16:57Z

first stab

New page

Using this test protocol, it was possible to
compare the innovative map-based econsultation
technique with the conventional
paper questionnaire used in the Probation
Board consultation. The chart shows the
difference between the post-test scores given
to the web map and the paper consultation questionnaire. The scores were 5 for strongly agree down to 1 for strongly disagree. The difference can range from +4 to -4. Where the question
was a negative one, the difference was taken away from 0, so that in all cases +4 is better for the
web map, and -4 is worse (i.e. the paper questionnaire is better).

Now for two of the testers (R and S), the web map was better than the paper questionnaire on all
but one criterion. The exception was the organization of information on the page. For those with
less Internet experience, the questionnaire was better. But on every other criterion of ease of
use, the map-based interface was better. The third tester (P) was over 60, and had had less
experience of computers and the Internet. On some criteria he found the paper questionnaire
easier (based on his experience of filling in forms in the past). Nevertheless, he was more
satisfied overall with the new web map than with the traditional questionnaire.
During the tests, there were frequent complaints about the paper questionnaire. Q5 was
particularly hard to grasp. It reads like a university or A-level examination question.48 But even
when answering the simpler questions, the testers discussed what the questions mean before
attempting to answer. In contrast, there were few problems when using the on-line map. They
found the probation office locations, recognised local features, and managed to manipulate and
move around the map display. Writing in several words as a comment on a particular site was
not a problem for two of them: the third got the research assistant to help him. In the discussion
afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.
Appendix 7 shows some sample comments added to locations on the map. They relate directly
to the issues of the consultation, focused on specific sites. By starting from the particular, rather
than requiring consultees to give general comments, it is both easier for consultees to express
their views, and consulters to understand and analyse them.

48 The question started: “5. PBNI Equality Scheme has given a commitment that in carrying out its functions it
will have due regard to the need to promote equality of opportunity between people who fall under the following
groups as stated in s75 of the Northern Ireland Act 1998. To help us consider potential inequalities please
identify the people in the following groups who you believe will be most affected by the proposed changes (for
staff and service users):- Religious belief: Positive [ ] Negative [ ] ...”

@@ Line 10: / Line 10: @@
 ==Test Results==
-Now for two of the testers (R and S) (See Figure 1), the web map was better than the paper questionnaire on all
+Now for two of the testers (R and S) (See Figure 1), the web map was better than the paper questionnaire on all but one criterion. The exception was the organization of information on the page.
 For those with less Internet experience, the questionnaire was better. But on every other criterion of ease of use, the map-based interface was better.

@@ Line 31: / Line 31: @@
 *''Belfast court (Headquarter). They would be affected by bus strick. They would try to find another way to visit a location—P''
 *''lt is just convienet to people living in belfast to go to both courts beside center.—P
-Ormeau Road There is no disadvantage to this office, staff are very helpful, safe room to speack to probation office.—P''
+*''Ormeau Road There is no disadvantage to this office, staff are very helpful, safe room to speack to probation office.—P''
 *''It is parking for people, easy for bues and shops.—P''
 *''Shankill Road. The marker wasnt near the location. i live on this road and i got a bit confused finding it on the map.The marker should be on the location.—S''

←Older revision		Revision as of 18:12, 13 January 2008
Line 27:		Line 27:
	In contrast, there were few problems when using the on-line map. They found the probation office locations, recognised local features, and managed to manipulate and move around the map display. Writing in several words as a comment on a particular site was not a problem for two of them: the third got the research assistant to help him. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.		In contrast, there were few problems when using the on-line map. They found the probation office locations, recognised local features, and managed to manipulate and move around the map display. Writing in several words as a comment on a particular site was not a problem for two of them: the third got the research assistant to help him. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.

−	~~Appendix 7 shows~~ some sample comments added to locations on the map. They relate directly to the issues of the consultation, focused on specific sites. By starting from the particular, rather than requiring consultees to give general comments, it is both easier for consultees to express their views, and consulters to understand and analyse them.	+	The following are some sample comments added to locations on the map.
		+	*''Alderwood House (ISU). There is car parking, and bus and shopping. I feel safe to go to office. There is good services offer in office. Some community perception and people might not go over there. —P''
		+	*''Belfast court (Headquarter). They would be affected by bus strick. They would try to find another way to visit a location—P''
		+	*''lt is just convienet to people living in belfast to go to both courts beside center.—P
		+	Ormeau Road There is no disadvantage to this office, staff are very helpful, safe room to speack to probation office.—P''
		+	*''It is parking for people, easy for bues and shops.—P''
		+	*''Shankill Road. The marker wasnt near the location. i live on this road and i got a bit confused finding it on the map.The marker should be on the location.—S''
		+	*''Lisburn Office. found the location i needed. needs no improvements.—S''
		+	*''Newtownards there needs to be more improvement on the map details. if i was really looking for the location i wouldnt get any where because its confusing.—S''
		+
		+	These comments relate directly to the issues of the consultation, focused on specific sites. By starting from the particular, rather than requiring consultees to give general comments, it is both easier for consultees to express their views, and consulters to understand and analyse them.

@@ Line 1: / Line 1: @@
 ==Test Method==
 Using the test protocol, it was possible to compare the innovative map-based e-consultation
-technique with the conventional paper questionnaire used in the [http://www.pbni.org.uk/ Probation Board] consultation. The comparison is shown in a chart (See Figure 1)
+technique with the conventional paper questionnaire used in the [http://www.pbni.org.uk/ Probation Board] consultation. The comparison is shown in another chart (See Figure 2)
 [[Image:PBNI_Consult_Chart.png|thumb|Figure 1: Chart]]
 The chart shows the difference between the post-test scores given to the web map and the paper consultation questionnaire. The scores were 5 for strongly agree down to 1 for strongly disagree. The difference can range from +4 to -4. Where the question was a negative one, the difference was taken away from 0, so that in all cases +4 is better for the web map, and -4 is worse.
@@ Line 10: / Line 12: @@
 Scores suggest that the the paper questionnaire was considered as better.
-Now for two of the testers (R and S), the web map was better than the paper questionnaire on all
+Now for two of the testers (R and S) (See Figure 1), the web map was better than the paper questionnaire on all
 but one criterion. The exception was the organization of information on the page.
@@ Line 21: / Line 23: @@
 ''5. PBNI Equality Scheme has given a commitment that in carrying out its functions it will have due regard to the need to promote equality of opportunity between people who fall under the following groups as stated in s75 of the Northern Ireland Act 1998. To help us consider potential inequalities please identify the people in the following groups who you believe will be most affected by the proposed changes (for staff and service users):- Religious belief: Positive [ ] Negative [ ] ..''
-But even when answering the simpler questions, the testers discussed what the questions mean before
+However, even when answering the simpler questions, the testers discussed what the questions mean before attempting to answer.
 attempting to answer. In contrast, there were few problems when using the on-line map. They found the probation office locations, recognised local features, and managed to manipulate and
-move around the map display. Writing in several words as a comment on a particular site was not a problem for two of them: the third got the research assistant to help him. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.
+In contrast, there were few problems when using the on-line map. They found the probation office locations, recognised local features, and managed to manipulate and move around the map display. Writing in several words as a comment on a particular site was not a problem for two of them: the third got the research assistant to help him. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.
 Appendix 7 shows some sample comments added to locations on the map. They relate directly to the issues of the consultation, focused on specific sites. By starting from the particular, rather than requiring consultees to give general comments, it is both easier for consultees to express their views, and consulters to understand and analyse them.

@@ Line 1: / Line 1: @@
-Using this test protocol, it was possible to compare the innovative map-based e-consultation
+==Test Method==
-technique with the conventional paper questionnaire used in the [http://www.pbni.org.uk/ Probation Board] consultation.
+Using the test protocol, it was possible to compare the innovative map-based e-consultation
-[[Image:|thumb|Chart]]
+[[Image:PBNI_Consult_Chart.png|thumb|Figure 1: Chart]]
-The chart (See Figure 1) shows the difference between the post-test scores given to the web map and the paper consultation questionnaire. The scores were 5 for strongly agree down to 1 for strongly disagree. The difference can range from +4 to -4. Where the question was a negative one, the difference was taken away from 0, so that in all cases +4 is better for the web map, and -4 is worse (i.e. the paper questionnaire is better).
+The chart shows the difference between the post-test scores given to the web map and the paper consultation questionnaire. The scores were 5 for strongly agree down to 1 for strongly disagree. The difference can range from +4 to -4. Where the question was a negative one, the difference was taken away from 0, so that in all cases +4 is better for the web map, and -4 is worse.
 Now for two of the testers (R and S), the web map was better than the paper questionnaire on all
-but one criterion.
+but one criterion. The exception was the organization of information on the page.
 The exception was the organization of information on the page. For those with less Internet experience, the questionnaire was better. But on every other criterion of ease of use, the map-based interface was better. The third tester (P) was over 60, and had had less experience of computers and the Internet. On some criteria he found the paper questionnaire easier (based on his experience of filling in forms in the past). Nevertheless, he was more satisfied overall with the new web map than with the traditional questionnaire.
-During the tests, there were frequent complaints about the paper questionnaire. Q5 was particularly hard to grasp. It reads like a university or A-level examination question.48 But even when answering the simpler questions, the testers discussed what the questions mean before
+During the tests, there were frequent complaints about the paper questionnaire.  Q5 was particularly hard to grasp. It reads like a university or A-level examination question:
 attempting to answer. In contrast, there were few problems when using the on-line map. They found the probation office locations, recognised local features, and managed to manipulate and
 move around the map display. Writing in several words as a comment on a particular site was not a problem for two of them: the third got the research assistant to help him. In the discussion afterwards, they expressed their satisfaction with the map interface, and how easy it was to use.
 Appendix 7 shows some sample comments added to locations on the map. They relate directly to the issues of the consultation, focused on specific sites. By starting from the particular, rather than requiring consultees to give general comments, it is both easier for consultees to express their views, and consulters to understand and analyse them.