Surveys for information can take a variety of different forms. Surveys work particularly well for learning about engagement in programs or the effectiveness of programs, and can be used to measure student learning outcomes particularly when learning is spread over a variety of learning experiences across campus or classes or happens over the span of multiple years.
Counts
Counts are useful when you need to know is how often something is happening or how much engagement you have with one of your initiatives.
For example, you may want to consider counting:
- Attendance at an event or workshop
- Products generated from a program or workshop
- ‘Melt’ (participants dropping out) during a longer program
- Courses with vocation in syllabi or in learning outcomes
Open Ended Questions
Your survey can include open-ended questions, which require the respondent to construct a response. You can get a lot of information from these responses, but they also take much more time to review and gather into a report. Open ended questions are particularly useful early on in programs when you are still looking for themes or want to understand more about how people are engaging with a program.
Example questions directed towards students:
- In what way has [insert program/class/curriculum] supported your development of a sense of purpose?
- What activity or class has most supported your development of a sense of purpose?
- What is understanding of vocation? Or what is your definition of vocation? This question can work well before and after a programmatic intervention if your goal is to create a shared definition understanding of vocation across campus.
Example questions for a faculty survey to understand what is already happening on campus
- Do you have student learning outcomes associated with the topic of vocation in your courses?
- How do you define vocation?
- Where do you think the topic of vocation fits in with the courses you teach?
- What is the one effective thing you do to engage with your students on the topic of vocation?
Scenarios
If you do not have common language around vocation or you want a more indirect way to engage student thinking relating to vocation, you can provide a variety of pre-determined scenarios and ask students which one resonates with their thinking the most. This approach works best if the scenarios are constructed in a way that it is not obvious that one particular scenario is the ‘best’ or ‘right’ answer.
The scenarios survey gives an example where students are provided three different ways to think about the work they do and its relationship to a sense of calling. Students then respond to each scenario indicating how much it feels like them and the way they think about their own work.
Multiple Choice Questions
A survey with multiple choice questions is much easier to process and analyze, but you must know enough about your program and its goals to know what you want to ask.
One example of using multiple choice questions to understand engagement with a program is the survey used to evaluate the Program for the Theological Exploration of Vocation (PTEV). Note that this survey does include a mixture of multiple choice questions as well as free response questions.
You can consider using standardized measures related to vocation and calling, which can be found here.
Any survey question can be given at two different time periods to look for changes in student thinking or disposition. Note that sometimes (especially over shorter periods of time) you may not see much difference because people tend to overrate their abilities or skills when they are less aware of what they don’t know. One way around this is to survey students at the end of a program or experience and ask them at that time to reflect back on their ability at the beginning of the experience as well. For an example of this, go to the bottom of Classroom Assessment Tools.