## Introduction to Mixed Method Designs

Introduction to Mixed Method Designs

Mixed Methods Designs
Certain questions cannot be answered by using qualitative methods, while others cannot be answered using quantitative methods BUT certain questions can be answered by

using both! (Walker, 1985, p. 16)
Qualitative and quantitative methods can be combined in three distinct ways:
Explanatory: An approach in which quantitative data are first collected, then the results are explained and elaborated with qualitative data. Example: Use of

questionnaire to provided an overview of students’ attitudes towards drug testing followed by several in-depth interviews of specific students with positive and

negative attitudes to understand how those attitudes were shaped.
Exploratory: An approach in which qualitative data are collected first, to explore the factors that are important to investigate, which is followed by quantitative

data collection. Example: A few interviews with students and a content analysis of several surveys allowed a researcher to determine the important factors around which

an attitudinal scale on drug testing programs was developed. Administration of this survey gave an overall view of student’s attitudes to a specific program being used

at a local school.
Triangulation: An approach in which quantitative and qualitative data are collected at the same time (one set of data does not inform collection of the other), to

provide a more comprehensive and complete set of data. Example: The use of a scale addressing attitudes toward drug testing programs could be administered to the

students in a school. Information from focus groups and interviews could be used to confirm the conclusions drawn from the survey.
Irrespective of design…what is important is that both approaches are done well.

Introduction to Mixed Method Designs

Explanatory mixed methods approach
Example 1: EXPLANATORY mixed methods approach
An approach in which quantitative data are first collected, then the results are explained and elaborated with qualitative data.
In this example quantitative data supported the qualitative data. This research was conducted in a private boy’s school in Sydney. The teachers noticed that the number

of boys in the school with poor fitness was increasing (i.e. the problem statement). This resulted in the teachers and researchers asking the following research

question:
(1) How can the levels of fitness be increased?
A researcher, who was trained in physical education and had worked in schools for many years, developed a program to try and answer this question. The researcher

developed a specific physical activity program for boy of low fitness. The program was called the Fitness Improvement, Lifestyle Awareness Program (FILA). It involved

26 x 1hr sessions held on Friday afternoons and additional lunchtimes sessions (held on Tuesday and Thursday). The Friday afternoon sessions comprised theory and

practical components and the program focused on three main areas (physical activity, small screen recreation (time spent watching TV, DVDs, playing computer games

etc), and consumption of sweetened beverages and fruit).
The aims of the study were to test the feasibility, acceptability and potential efficacy of the FILA program. An experimental design was used (i.e. measures were taken

before the program started, then the FILA program was implemented and then measures were taken after the program had finished and participants were randomised into a

intervention group or a control group). Qualitative data were also collected throughout and at the end of the program.
We will now look at the first two aims and see how qualitative and quantitative data were used to complement each other.
Aim 1: To determine the feasibility of the program
Several sub questions were asked: Could participants be recruited? Could they successfully be randomised into either the intervention group or the control group? Could

they all be retained from the beginning of the program to the end of the program and could all data be collected?
The quantitative data showed that 33 participants were successfully recruited and randomised into either the intervention group or the control group. All but one

participant was retained (i.e. completed the program and the measurements). This participant left the school; therefore measures after the program had finished could

not be collected. All outcome measures were collected as planned.
Retaining all participants is was a positive outcome but the teachers and researchers wanted to know why participants remained in the program. What was it about the

program that encouraged them to stay? In order to find out this information they collected qualitative data through interviews with the students. Read the quote below

from a student. Why do you think he remained in the program?
•    Yeah, I enjoyed all the games we played. To be honest I think it was better than what the normal fitness groups were doing. It was more fun. I was in the

normal fitness groups in Term 1, but then in Terms 2 and 3, I was in FILA and fitness was much better and much more fun (Participant C).
The qualitative data confirmed the quantitative data. Without the qualitative data the teachers and researchers would not have known why the students chose to remain

in the program from the beginning to the end.
Aim 2: To determine the acceptability of the program.
For acceptability aims, the sub questions were – Could all the planned lessons be implemented? What was the average attendance? Did participants enjoy the sessions?

What did the teachers and parents think of the program?
The quantitative data showed that 14 of the 16 planned Friday afternoon sessions were implemented and all of the lunch time sessions were implemented as planned. Bad

weather and an unplanned public holiday were the reasons for not implementing two of the Friday sessions.
In relation to attendance, the quantitative data showed the following:
•    Average attendance – 84%
•    One session with 100% attendance
•    6/16 sessions with >90% attendance
•    4/16 sessions with < 80% attendance (beginning and end of school term)
•    Average attendance to lunchtime sessions <40%
What do you notice about this quantitative data? Perhaps the most striking aspect of this data is the fact that the lunchtime sessions were poorly attended. The

teachers and research wanted to know why this was the case. Interviews (qualitative data) with the students revealed some reasons why this may have been the case. Read

the quotes below to see the themes identified from the interviews. What were the main reasons why students did not want to attend the lunchtime sessions?
•    Yes, I attended lunchtime sessions in Term 3 only, as in Term 2, I had both music lessons and homework recovery sessions to attend during Tuesday and Thursday

lunchtimes (Participant B).
•    Lunchtimes are supposed to be our time where we can spend it with our friends, and not be forced into doing a lunchtime session (Participant To).
At the end of each session participants completed an enjoyment questionnaire. The enjoyment ratings are shown in the table below.
Week    Enjoyment (%)    Session
1    79    Physical activity
2    60    Physical activity
3    97    Healthy eating
4    75    Screen time
5    64    Physical activity
6    86    Screen time

What do you notice about these results? Why were the physical activity lessons enjoyed less than the other sessions? Read the quotes below from the students to

discover why the physical activity sessions were not enjoyed as much as the other sessions. Again without this qualitative data, an explanation for the quantitative

data would have remained unknown.
•    My favourite activity was the fruit and sweetened beverage displays and eating/drinking these after we played a game of deciding which items were the healthier

(Participant M).
•    I liked the sweetened beverages session, because we had the chance to taste most of the beverages, which was kind of cool. I liked that session the most

(Participant Q).
•    I didn’t enjoy the physical activity component, because I hated the sports. All the other groups got to play the original games, like rugby, but we had to play

these hybrid sports, which were a waste of time and not as enjoyable (Participant N).

Introduction to Mixed Method Designs

Exploratory mixed methods approach
Example 2: EXPLORATORY mixed methods approach
An approach in which qualitative data are collected first, to explore the factors that are important to investigate, which is followed by quantitative data collection.
This example used qualitative research as a starting point for subsequent quantitative research. This research focused on improving the physical activity levels of

girls in a school that had a high percentage of students with non-English speaking background students. Physical Education teachers noticed that the level of physical

activity that adolescent girls were participating in was very low (i.e. the problem statement). The teachers asked three research questions:
1.    Why did the adolescent girls not want to participate in physical activity?
2.    How could the level of physical activity be increased?
3.    If an alternate sports program was introduced would girl’s level of physical activity increase?
We will look at each of these questions separately to discover how qualitative data informed subsequent quantitative data.
Question 1: Why did the adolescent girls not want to participate in physical activity?

To answer this question, the teachers and the primary researcher conducted a series of semi-structured interviews. If you remember from previous weeks, semi-structured

interviews involve a small number of participants and use open-ended questions. Approximately 10 interviews were conducted with the adolescent girls. Girls were asked

questions about what they thought about physical activity lessons at school, whether or not they enjoyed physical activity lessons and what barriers stopped them from

participating in physical activity lessons.
Read through the following quotes. Why do you think the girls did not want to participate in physical activity? What themes can you identify from these quotes?
•    I don’t like sports where you are put on the sport like cricket or the other bat games where everyone is looking at you (Student A).
•    I don’t like softball – it’s really boring half the time you’re standing there… In the sun or you’re standing against the wall not doing anything (Student

B).
•    I think that we should just wear sport uniform for the whole day ‘cause I hate getting changed (Student A).
•    The sports uniform is ugly and uncomfortable. It really is and when it’s really hot it’s a really heavy fabric, it sticks to you… (Student B).
Question 2: How could the level of physical activity be increased?

The physical education teachers and the researcher discussed the qualitative data then met and suggested ideas how they could change the lessons to accommodate the

students but also work with the constraints of the school and the requirements of the Department of Education. Three suggestions were presented to the School Executive

for approval. The first idea was to involve only trained enthusiastic teachers in physical activity lessons. In secondary schools, teachers are often required to teach

in subjects not related to their specialisation and at this particular school many different teachers were required to regularly teach physical education. A decision

was made that only trained teachers that were enthusiastic about physical activity would teach the girls. What effect do you think this had on the participation of the

girls? Why? The school approved this suggestion.

The second idea placed before the School Executive was to let the students wear their sports uniforms to school so they did not have to carry their uniforms to school

and then get changed. A great idea! However, this was not approved by the School Executive.

The final suggestion was to give the girls a choice of activities that they would like to participate in physical activity classes. So instead of being instructed in

activities that they did not enjoy or felt uncomfortable about, they could choose activities that were more enjoyable and all students could participate in. The School

Executive approved this suggestion and various meetings were then held with the girls to discuss different activities that they would like to participate in.

Activities like Pilates, walking and aerobics were suggested.
In summary from the qualitative data, teachers understood why the girls did not want to participate in physical activity and what activities could be offered to

increase physical activity participation within school lessons. As a result, an alternate trial program (i.e. a program incorporating Pilates and aerobics) was

developed.
Question 3: If an alternate sports program was introduced would levels of physical activity increase?

To test the effect of the trial program quantitative methods were used. All girls in year 11 completed an enjoyment physical activity survey. These were then analysed

(using quantitative methods) and the girls with the lowest enjoyment levels were invited to participate in the new activity program. Forty girls returned their consent

forms. An experimental research design was then used. Revisit Quantitative Research Part 2 to refresh your memory about experimental research designs. Experimental

research involves collecting measures before and after implementation of a program, and participants are randomised to either an intervention group or a control group.
So in this example, before the program was implemented a number of measures were collected. These included physical activity, body image and self worth measurements.

Physical activity was measured using an activity monitor and body image and self worth were measured using a questionnaire. After these initial measures were taken the

girls were randomised into either the intervention group or the control group. The intervention group participated in the new trial program that was developed based on

the qualitative data and the control group participated in normal school physical education lessons. The trial was 12 weeks in length and girls participated once every

fortnight for 1.5 hrs.
At the end of the intervention, all the measurements were taken again and then the results were compared with those taken at the beginning of the program. In the

intervention group, improvements were seen in all aspects compared to the control group. So the answer to question 3 is yes – the level of activity in Physical

Education lessons of adolescent girls will increase if a trained teacher facilitates the lesson and if the students are offered a variety of activities, like Pilates

and aerobics. The time spent collecting qualitative data was certainly valuable and informed the quantitative data. Perhaps if the qualitative data had not been

collected, teachers would have just continued in their normal lessons which may have further decreased the physical activity participation rate of the girls.

Introduction to Mixed Method Designs

Like all research methods there are a number of significant advantages associated with a mixed method approach and a number of limitations that should be considered.
Some of the advantages of a mixed method approach are:
•    Incorporates the strengths of both qualitative and quantitative approaches
•    Provides a more comprehensive view of the phenomena being studies
•    Able to answer broader and more complete range of research questions
•    Does not limit the data being collected
•    Can provide stronger evidence for conclusion
Conversely, some of the limitations of a mixed method approach are:
•    Requires expertise in both methods
•    Single researcher may not be able to collect both types of data
•    Requires extensive data collection and resources
•    More expensive
•    More time consuming

Introduction to Mixed Method Designs

A mixed method approach can certainly be beneficial; however researchers may also need to consider the following points. If these points are not considered, it is

likely that the mixed method approach will not provide that required data and may potentially provide the wrong type of data.
•    Is a mixed methods approach feasible?
•    Is a mixed methods approach warranted?
•    Have the research questions been written for both quantitative and qualitative methods in the study?
•    Have quantitative and qualitative data collection procedures been clearly identified?
•    Are the procedures for data analysis consistent with the type of mixed methods study being presented?
•    Is the written structure of the study consistent with the type of mixed methods study being presented?

Introduction to Mixed Method Designs

Introduction to Mixed Methods: Capstone
Weekly ‘Capstone’ Activity and Discussion:
Once you have read through the week’s Moodle content, locate a published mixed methods study in your area of specialisation. Reading through the article, consider:
1.    What mixed methods design was adopted;
2.    Whether a mixed methods approach was warranted (i.e., is the study weakened by removal of the qualitative or quantitative data);
3.    The qualitative and quantitative data sources, and whether they both contribute to an understanding of the underlying research problem and question; and
4.    Whether the research question is clearly communicated, and is truly mixed methods in nature.