Using technology for evaluation purposes:
A Frontier College cost-effective approach to evaluation for volunteer-based organisations

Across Canada Frontier College has over 3,000 volunteers working at over 250 sites. Volunteers work with children, youth and adults in a variety of settings ranging from federal prisons to local communities. In 2001 Frontier College undertook an evaluation that would survey all of these volunteers to get feedback on their tutoring and overall volunteer experience. As daunting as this task may seem the use of technology allowed us to undertake our first full evaluation in the spring of 2003.

The use of information and communications technology (ICT) in volunteer organizations is a burgeoning field. A recent study indicated three factors that may be related to ICT use in voluntary sector organizations:

1. Individual characteristics of managers of volunteer resources
2. Organizational characteristics, such as size and mission
3. Volunteer program characteristics

As 90% of Frontier College volunteers are university students technology seemed like a logical and sensible way to collect information from our volunteers. In many respects Frontier College was leveraging the technology that is commonplace for the majority of our volunteers. There were three distinct phases in the implementation of our evaluation strategy:

Phase 1 – Development of an online database

In this phase we created an online volunteer database. The database is accessible to all of our staff across Canada and contains basic volunteer information. The most critical piece of information is a volunteer’s e-mail address. In the first two years we have captured over 4,500 e-mail addresses. The volunteer database is integrated into our INTRANET.

Phase 2 – Development of an online communication strategy

Sending e-mail to over 3,000 recipients can put a tremendous strain on computer resources. To address this we subscribed to a bulk e-mail service that allows us to send over 10,000 e-mail messages per month. This service allowed us to establish a series of electronic communiqués to our volunteers. This series included a message from the President of Frontier College, program updates and links to updated sections of our website. In the future we will be sending electronic communiqués to our volunteers every two months. From a volunteer management perspective this has been a tremendously successful exercise.

Phase 3 – Design and launch of an online survey After many months of research and survey design our online survey was launched in February 2003. After evaluating numerous electronic survey tools we opted to use Zoomerang (http://www.zoomerang.com) as our survey tool. Electronic survey tools also offer a huge cost advantage over traditional surveys and the ensuing data entry costs. Overall we were able to launch our survey and collect data for less than $1,000.

Once launched, we were extremely happy with our survey results. Over 35% of our volunteers responded to the survey and the combination of both quantitative and qualitative data has been spectacular. We have been able to glean a great deal of information about volunteer recruitment, training and ongoing support. For example, we were able to survey all of our one-to-one adult tutors about the kind of follow-up training that would have been useful to them. The following chart reflects some of the feedback that we received:

This technology has also allowed us to gather over 3,200 anecdotal comments from volunteers; a volunteer reflects on the impact of their tutoring:

’Working with Frontier College brought me out of my comfort zone and into a new neighbourhood, a new culture and new people. I did meet several friends along the way, co-tutors and day-staff, as well as learners. Watching some of these people (who are being tutored) causes me to realize how fortunate I am to have a life and resources that have allowed me to pursue academics. (…)

I do believe that community service (in whatever capacity) is beneficial and essential. Those with abundant resources should do their part. Although our society is far from an utopia, we only stand to progress from the dedication and caring of numerous individuals who volunteer their time and skills for the betterment of others.’’

Conclusions

The benefits of an electronic evaluation are summarized below:

• Allowed us to reach a spread out and transient volunteer population
• Cost effectiveness
• Leveraging the technology that our volunteers use on a daily basis
• Ease of data analysis
• Real-time monitoring of responses

Some of the challenges of electronic evaluation include:

• Personal communication works much better for many smaller volunteer organizations
• Accessibility
• Technology is great when it works!!

Lessons Learned

There are a number of key lessons that we have learned from our initial evaluation:

• In our initial pass we collected far too much information. In the future we will simplify the evaluation sent to all volunteers.
• The importance of survey design

Overall we were extremely pleased with our initial attempt at electronic evaluation. We will be refining the survey and re-launching it early in 2004. Stay tuned for more learnings and details:

If you would like more information about technology and evaluation please contact:
Dan Khimasia – National Program Manager:
Frontier College, dkhimasia@frontiercollege.ca