Project Methodology

This project was delivered across three phases of research: (1) Review, (2) Exploration, (3) Benchmarking.  These phases, and the project timeline, is presented in the diagram below. The following sections provide further details of the methods used in each phase to support the research.

Project phases

Phase One – Review

The first phase of the project aimed to undertake a review of existing practice and understandings of quality in work integrated learning.  

Systematic review

The first part of this phase undertook a systematic review of relevant literature and documentation. Robinson and Lowe (2015) explain that a systematic review, rather than a broad literature review, employs a precise question and a defined ‘data extraction’ method.  Normally a systematic review will engage with only a relatively small number of papers (e.g., less than 50). To guide the review of literature, the following questions were used as a focus:

  1. How is quality in higher education and work integrated learning currently defined and understood?
  2. How are constructs of quality used in shaping current measures and frameworks of practice?

Relevant documents were sourced from a desktop analysis and review of publicly available quality assurance approaches and work integrated learning frameworks. Documentation included key literature from relevant journals and publications, university policy documents and available practice materials, and available government reports.

Practitioner Interviews 

During Phase One of the project, interviews were undertaken with a wide range of WIL practitioners across the Australian higher education sector.  Interview participants included, for example, administrative and professional staff, academic teaching staff and university leaders, and were identified across a wide range of Australian universities.  A matrix of possible interview participants was developed by the project team to ensure that the interviews were somewhat representative of a diversity of voices across the whole sector.  

These interviews used a semi-structured approach (Fontana & Frey, 2005) with the following key questions guiding the interviews:

  1. How does your university define ‘work integrated learning’?
  2. How does your university currently assure the quality of curriculum design, content and student outcomes?
  3. How does your university currently support, or manage, curriculum change and transformation?
  4. If you were to design an institutional framework of quality for WIL experiences, what elements should be considered?
  5. How does your university currently assure the quality of a WIL experience?
  6. What barriers do you think exist in your university to the adoption of an institutional-wide framework for the quality assurance of WIL?

All interviews were transcribed and analysed for common emergent themes about good practices in quality assurance of curriculum and WIL and to identify key elements for a guiding framework. 

To support the interviews, and ensure a breadth of voice in shaping understandings of quality, a short questionnaire was also distributed to approximately 90 identified leading WIL practitioners across the sector.  The survey reflected the thematic interest of the interviews with four short response questions:

  1. In a few sentences, how does your institution define work integrated learning (WIL)?
  2. How do you (or your university) currently assure the quality of a WIL experience?
  3. In your experience, what are the elements that should inform a quality WIL experience?
  4. What barriers do you think exist in your university to the adoption of an institutional wide framework for the quality assurance of WIL?

An additional 11 written responses were received with the responses incorporated into the data corpus for this phase of the project.

The project team met face-to-face to review the transcripts of the interviews and responses to the questionnaire, and undertook a thematic analysis of emerging ideas of quality.  Emerging from this phase were four thematic domains of practice which were used to inform the shape of the framework: (1) student experience, (2) curriculum design, (3) institutional requirements, and (4) stakeholder engagement. Within each domain, standards and principles of practice were developed based on the literature and interviews.  This draft framework, and associated interim report, was distributed for feedback and input to an external reference group, comprised of 26 well-regarded WIL researchers and practitioners across the Australian higher education sector.

Phase Two – Exploration

Phase Two of the project utilised workshops and questionnaires to explore understandings of quality of WIL across different disciplines and institutions and verify findings from Phase One. The use of workshops as a methodology was informed by the work of Ørngreen & Levinsen (2017) who argued for the utility of this approach as a way to clarify and test emergent ideas. This phase allowed for the testing of analysis and conclusions emerging from Phase One, in conjunction with the development of deeper understandings of underlying assumptions through the use of a series of workshops and feedback processes. Across this phase three workshops were held, with one each in Brisbane, Melbourne, and Sydney to explore and establish a foundation of understanding around quality. Approximately 30 – 40 people from a wide range of universities and practice attended each of these workshops, with the first two workshops held in conjunction with the local ACEN chapter, whilst the third workshop contained only staff from the University of Sydney, but drawn from a wide range of areas across the university.  

The workshops were structured to provide opportunities for participants to work in small groups to interrogate the proposed framework and consider how the suggested standards and principles may be realised in practice. The structure broadly followed an arrangement of introducing the project and framework followed by a series of small and large group discussions employing the following questions, arranged under three focussed themes, to frame discussions:

  1.     Validating the draft framework
    1. What are the strengths of the framework as presented?
    2. What are the gaps, or additions, with the framework that need consideration?
  2.     Mapping quality indicators
    1.     Considering each component / standard, what would successful practice  / activity look like for this component?
    2.     How would you measure / evidence success in this practice?
  3.     Barriers, enablers and benchmarking
    1. Where across the university would the various points of evidence be located?  How accessible are these points of evidence?
    2. What time would be required to map against the framework, and how much is this part of common practice?
    3. How mature are the evidence and data processes at your university to support benchmarking against such a framework?

After each workshop the framework was adjusted by the project team based on feedback and comments from participants. Following completion of the third workshop the project team met to review all feedback and shape a final draft framework. This final draft framework was distributed to the external reference group and workshop participants with a survey which focussed on the success measures of clarity, thoroughness, usability and quality.  

The emerging framework modelled a structure which identified key goals, principles and elements of WIL, quality standards, success measures, and provided a range of examples of evidence which demonstrated success. Such a model reflects similar quality frameworks, such as the National Teaching Standards as developed by AITSL (see: https://www.aitsl.edu.au/teach/standards).  

Phase Three – Benchmarking

Phase Three employed a peer-review process of benchmarking to consider the consistency of understanding the quality framework and to further explore usability, utility and implementability of the framework. Within this phase, two benchmarking case studies were used: (1) institutional benchmarking involving QUT, RMIT and USyd, and (2) faculty/discipline benchmarking involving the Faculty of Health, QUT, and Faculty of Health Sciences, USyd.  

The benchmarking was supported by a spreadsheet which was developed to provide guidance to participants around the collection and noting of evidence.  The benchmarking spreadsheet allowed for the gathering of the following information against each standard:

  • Appropriate examples of evidence
  • Identification of the responsible person for such evidence
  • Explanation of the level of quality for the evidence
  • Accessibility of and ease to locate evidence
  • Opportunities for improvement to the standards / framework and challenges to be addressed

The benchmarking activity also facilitated the ongoing collection of illustrations of practice for each of the standards, alongside general calls to the WIL community for contribution of relevant examples of good practice.

It was not the aim of this phase for the benchmarking reports to be collected as part of this project; instead the purpose was to test the usability and utility of the framework.  Therefore, data for this phase was collected through recorded reflections captured as part of regular project team meetings, with key ideas and emergent themes identified within these reflections.