Pilot testing

On going work:

In parallel with finalising the tool, the prototype is tested on school students. The following evaluation templated is used:

Anna is an online tool, developed by a consortium of Europe’s leading universities, as part of a project called INGDIVS (INcreasing Gender DIVersity in STEM): http://www.ingdivsproject.eu

Specifically, Anna tries to provide information from current engineering students and working engineers in an easily accessible manner to potential students – allowing them to better relate to the skills, attributes, attitudes and experiences of people who have chosen that route. It is our hope, based on previous research, that by making it easier to relate to people who have previously made that choice (to study engineering), that potential engineers are able to see themselves in these roles and make a more informed choice of course study.

How you can help

We have built a first prototype of the tool, and we are looking for feedback on how easy it is to use, and how useful it might be. We hope that you will be able to supervise some user testing. We will not ask you for, nor will we record, any identifying information from your students. We hope the information below is clear and informative, and your local university partner should be able to assist you with any queries.

Resources Required

You need two people for this process. One person takes the lead in communicating with the students, while the other has the role of rapporteur – taking notes of students behaviours, reactions and responses. The lead person may wish to refer to these or other notes, but doesn’t require any other resources. The rapporteur may choose to use a paper version of the enclosed form, or to complete it electronically. The students (3-5) will each need access to a computer, with the web accessible Annawebsite tool loaded and ready to be used. The total time for an individual group will be approximately 30-35 minutes, and we hope that you can test 3 groups. These groups, if applicable, should comprise an all male group, an all female group and a mixed gender group.

How the Test Process Works

There are three principal phases. Firstly you give a short introduction to the group of students (we suggest 3-5 students) – we’ve prepared a short script which you can use if you like (about 5 minutes). Secondly the students take the test, individually, at the computer. This will take about 10 minutes. Finally, you bring the students together (the focus group) and prompt them to discuss their experience. There are 5 topics we would like the students to discuss. We have provided some sample prompts for each topic, and then a ‘wrap-up’ question to summarise in a quantitative way the collective opinion of the group. The 5 topic areas are:

  1. Understanding of the tool’s purpose
  2. Usefulness
  3. Ease of use
  4. Suggestions
  5. Any other comments or feedback

What sort of students should we seek?

Any students from the pool of students who could potentially progress to engineering programmes at university. They do not need to be only those who have strongly decided already – in fact it is probably better if they aren’t. Equally, it is probably better if they have not strongly rejected Engineering. The ideal students are ones who either have decided, or who have another preferred option but are  not negative towards engineering studies.