Original Research

Are we there yet? Mixed methods research in the South African Journal of Economic and Management Sciences

Patrick Ngulube, Beatrice Ngulube

Received: 16 Feb. 2022; Accepted: 02 June 2022; Published: 29 July 2022

Copyright: © 2022. The Author(s). Licensee: AOSIS.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Orientation: In 2015, a study was conducted to explore the prevalence of mixed methods research (MMR) in the South African Journal of Economic and Management Sciences (SAJEMS) between 2003 and 2011. This study builds upon that study to establish the extent to which the use of MMR has developed in SAJEMS.

Research purpose: The purpose of this research is to explore and compare the levels of adoption of MMR in the present study to the one reported in the study of 2015.

Motivation for the study: This study goes beyond the one of 2015 by exploring methodological transparency in the use of MMR by contributors to the Journal and answers the question: are we there yet?

Research design, approach and method: The study analysed the methodology used by articles published in SAJEMS from 2012 to 2019. Content analysis was conducted on a total of 362 articles published in SAJEMS. Each of the nine articles identified as utilising MMR was analysed based on the indicators of use of MMR in the extant literature.

Main findings: The findings show that studies neither stated the appropriateness of mixed methods nor specified the designs. Integration was done by a few articles in a limited way. The passing of 8 years has not brought about any difference in the results of the use of MMR in SAJEMS. The use of MMR remains underrepresented. It is evident that SAJEMS is not there yet.

Practical/managerial implications: This article raises the need for methodological pluralism as an alternative to quantitative and qualitative methodologies.

Contribution/value-add: Researchers are informed of the advantages of using MMR and incorporating the third methodological movement to obtain superior results.

Keywords: mixed methods research; methodological transparency; pragmatism; integration; research designs.

Introduction

The advantages of mixed methods research (MMR), with its focus on methodological pluralism, are widely acknowledged (Creamer 2018; Creswell & Plano Clark 2018; Johnson & Onwuegbuzie 2004; Ngulube 2022). Methodological pluralism provides researchers an opportunity to explore a phenomenon comprehensively. It should be noted that the scientific method on which most of the research is based is not singular (Love 2006). That implies that the scientific pursuit has a pluralist approach, although that is rarely recognised by many researchers. It is apparent that there is no universal scientific methodology such as qualitative or quantitative research as some methodologists and methodological purists have led us to believe. Methodological purists believe in the sanctity of incompatibility of qualitative and quantitative methodologies and elevate either of them into a dominant status.

Although many researchers fail to acknowledge methodological pluralism, researchers have used multiple methods for many years (Molina-Azorin 2016). However, MMR as a distinct third methodology has emerged in the last few years (Creswell & Plano Clark 2018; Molina-Azorin 2016). Mixed methods research is extremely attractive to researchers because it has the potential of producing knowledge that is anchored on diverse perspectives. Thus, MMR ‘actively invites us to participate in dialogue about multiple ways of seeing and hearing … and multiple standpoints on what is important and valued and cherished’ (Greene 2007:20).

Although the research question largely determines the appropriateness of the research methodology, the use of one methodology is not without its challenges. For instance, some contributors to the South African Journal of Economic and Management Sciences (SAJEMS) acknowledge that the use of a qualitative methodology alone limited their ability to answer certain questions and to develop a complete model that was fit for purpose (Von Loeper et al. 2016). Ultimately, using monomethods limits the researcher’s ability to answer the ‘what’, ‘why’ or ‘how’ type of research questions in a single study. It is important to note that MMR can assist researchers in answering these questions from multiple perspectives and multiple levels of analysis in a single study. Thus, the use of MMR is attractive because of its potential to answer both confirmatory and exploratory questions in one (Teddlie, Johnson & Tashakkori 2021).

Mixed methods research also has the potential of opening opportunities for innovation and productive research in a field (Vogt 2008:455) and giving a holistic picture. However, researchers should be acutely aware that the claim that MMR portrays the whole picture might be misleading because as Johnson and Onwuegbuzie (2004:23) pointed out, research methodologies are ‘all superior under different circumstances’. Research approaches other than MMR might be more appropriate in addressing certain research questions. As Grover and Glazier (1985) supported the utilisation of qualitative research methods over quantitative ones, this study partially supports the use of MMR over monomethods on the proviso that it is used transparently. Mixing research methodology will enable researchers who publish their work in SAJEMS to have many research tools at their disposal and treat research problems differently.

Methodological diversity can assist researchers to innovatively investigate reality without sticking to one solution. There is a compelling reason to treat every research phenomenon as a nail if a hammer is the only available tool (Stange & Zyzanski 1989). The dominance of quantitative research methods in many disciplines (Ngulube 2022) and in SAJEMS (Ngulube & Ngulube 2015) is likely to tempt researchers to think that all research problems are better addressed quantitatively when other methodologies are available. For instance, other methodologies such as qualitative research and MMR can benefit research in a field.

The extent to which a field exploits MMR can be partially determined by prevalence studies. Molina-Azorin and Fetters (2016) called on scholars to conduct prevalence studies in specific cognitive fields to determine the rate of adoption of MMR. Such literature mapping studies uncover what is already known to inform practice and decision-making. Prevalence rate studies such as this one further assist to determine the frequency of quantitative, qualitative and MMR studies occurring in a cognitive discipline.

In the context of MMR, they demonstrate the extent to which researchers in a discipline are aware of the use of MMR and how it can be employed in research (Alise & Teddlie 2010). Prevalence rate studies also help to determine the extent to which paradigm wars have subsided in a discipline (Alise & Teddlie 2010), and Onwuegbuzie and Corrigan (2018) agree. Prevalence rate studies can also demonstrate the extent to which a discipline is adopting the third methodological movement. This is against the backdrop of the potential advantages offered by MMR and the difficulties in using it in practice.

This article is valuable because it contributes to research practice and understanding of MMR. Secondly, it raises awareness of the probable benefits of MMR in Economic and Management Sciences (EMS) research. Thirdly, it shows how MMR as a methodology that is growing in popularity is used by researchers who contribute to SAJEMS. Fourthly, the article provides a guide on how to approach and design an MMR study. Lastly, it contributes to the visibility of MMR in the developing world, as Tashakkori and Teddlie (2010) found the developing world to be underrepresented in the MMR landscape.

Mixed methods research prevalence studies have been conducted in many fields in South Africa, including EMS (Ngulube & Ngulube 2015), information science (Ngulube, Mokwatlo & Ndwandwe 2009; Ngulube & Ukwoma 2021), psychology (Barnes 2012), social sciences (Barnes 2019) and education (Mabila 2017). These studies highlight the underrepresentation of MMR in various fields. Posel (2017:119) underscores the fact that economics will benefit from ‘interdisciplinary collaboration and mixed methods research’. Mixed methods research is very important in the context of South Africa because of its complex history, levels of inequality and cultural diversity (Barnes 2019; Posel 2017). Mixed methods research provides a better way of understanding social and economic processes (Balog 2020).

Theoretical background

Although there is no agreement on the genesis of MMR, the promotion of the use of multiple quantitative methods in determining a psychological construct by Campbell and Fiske (1959) and the subsequent development of triangulation by proponents such as Denzin (1970) were the most important harbingers of mixed methods. The ‘paradigm of choices’ that rejected methodological dogmatism and acknowledged that research questions determine the different methods used in research strengthened the move towards the use of multimethods (Patton 1990). The movement towards the use of multiple methods recognised the limitations of exclusively using one method when conducting research. The thinking undermined the foundation of paradigm wars that emphasised the quantitative and qualitative binary and the incompatibility thesis.

Increasingly, the recognition that the distinction between qualitative and quantitative methodologies was a ‘category mistake’ (Vogt 2008) and a ‘false dichotomy’ (Ridenour & Newman 2008) led to the development of MMR as a third methodological movement (Johnson & Onwuegbuzie 2004). The evolution of MMR has been characterised by ‘methodological plenitude’ (Love 2006) leading to a plethora of frameworks of designing, conducting and reporting MMR studies and ‘divergent conceptual views’ (Anguera et al. 2018) about MMR.

The main characteristics of MMR described in this theoretical background are mainly based on Creswell and Creswell (2018) and the conceptualisation of MMR depicted in Figure 1.

FIGURE 1: Main characteristics of mixed methods research.

An MMR design and methodology comprises research paradigm, methodology, approach and research methods, as illustrated in Figure 1. Based on Figure 1, the following sections of the theoretical background explain the philosophical assumptions of MMR, the rationale for mixing, the specific MMR designs (approach), integration and the value-added of MMR.

Philosophical foundations

Ontological foundations inform the researcher’s epistemological assumptions, which in turn influence the methodological choices made by the researchers (Guba & Lincoln 1988). The paradigm wars that dogged the scientific method before the rise of the mixed methods movement were based on the irreconcilability of the positivist and the interpretivist epistemologies. Mixed methods research as a third methodology emerging from the paradigm wars or science wars is expected to have a philosophical foundation underpinning its epistemology (Teddlie et al. 2021). There is no agreement over the philosophical foundations of MMR, as there are many variations in how scholars describe its philosophical assumptions (Stoecker & Avila 2020). However, the pragmatic approach is the commonly used philosophical stance in MMR (Teddlie et al. 2021). Pragmatism advocates the utilisation of quantitative and quantitative research methods in one study and pays a blind eye to paradigmatic stances.

Some scholars have proposed other philosophical assumptions for MMR in order to contextualise MMR (Creswell & Hirose 2019; Miller 2015). On the other hand, Goodyear-Smith and ‘Ofanoa (2021) proposed the Fa’afaletui cultural perspective, which champions a Samoan research framework as an alternative to pragmatism. In that regard, philosophical transparency is essential in an MMR study. It is also important to declare the philosophical assumption of an MMR study because the paradigm constitutes the MMR integration trilogy (Fetters & Molina-Azorin 2017).

Appropriateness of mixed methods research

Mixed methods research should not be used just for the sake of using it. It is important to determine the appropriateness and the value-added of MMR to a study. Explaining the benefits that researchers derived from using MMR demonstrate the awareness of the value-added of the methodology. Merely arguing that MMR provided a comprehensive or complete picture of the phenomenon under study without giving specific details of the actual benefits that accrue to a study by using MMR and going on to implement it and conclude the study without reflecting on the MMR value-added is not enough.

Greene, Caracelli and Graham (1989) described five main reasons for using MMR: triangulation, complementarity, initiation, development and expansion. Although triangulation is going in and out of MMR language (Fetters & Molina-Azorin 2017; Ngulube 2022), it seems that the five reasons for using MMR advanced by Greene et al. (1989) formed a sound, pioneering foundation. Building on these five reasons for deciding the appropriateness of MMR, scholars have articulated the reasons for using MMR as follows (Creswell & Creswell 2018; Fetters 2020; Ngulube 2022):

• Either qualitative or quantitative data may be inadequate to give another perspective on the research problem. A researcher may concurrently use qualitative and quantitative research methods in order to get a full perspective. Data are collected almost simultaneously to establish convergence or corroboration and divergences in the data (Creswell & Plano Clark 2018; Ngulube 2022; Teddlie et al. 2021).
• Quantitative outcomes need to be explained using a qualitative approach. A quantitative study may uncover patterns and trends of a certain phenomenon without explaining the causes behind the patterns. The qualitative approach may be employed to uncover the reasons behind the trends.
• Qualitative findings need to be generalised to a larger population through a quantitative methodology. Qualitative data are collected either to develop theory or research protocols and then quantitative data are collected later to confirm the theory or to generalise the results to a bigger population.
• Multiple research phases should be conducted to comprehensively understand a research problem. Multiple research phases are prevalent in advanced MMR designs as compared to simple MMR designs described in the first three bullets in this section. Such designs are common in transformative and social justice studies whereby the researcher is interested in involving more participants in order to transform their lives, or when the researcher wants to develop, implement and evaluate a programme (Creswell & Plano Clark 2018; Ngulube 2022).

Ultimately, the use of MMR should enrich and expand the researcher’s appreciation of the research problem by marshalling diverse perspectives employing the complementary advantages of qualitative and quantitative research methods, and they must be reflected on when reporting MMR studies.

Mixed methods research designs

Various MMR designs exist (Gosh 2016; Ngulube 2022). Leech and Onwuegbuzie (2009) describe a three-dimensional typology, including time orientation, emphasis of approaches and level of mixing. There is consensus amongst MMR scholars that the designs can be classified as basic and advanced (Creswell & Creswell 2018). The basic and core MMR designs include the sequential exploratory design, sequential explanatory design and the concurrent convergent design. The advanced or complex designs (Ivankova & Kawamura 2010) include intervention, social justice and multistage evaluation designs. The complex designs are based on the basic ones, which are the starting point whenever the researchers intend to execute the advanced ones.

Qualitative and quantitative data are collected almost at the same time in the parallel convergent designs. A convergent, concurrent design may lead to the divergence, convergence and complementarity of the findings. In the final analysis, the qualitative and quantitative data are integrated to achieve a MMR study. The two components are then integrated to draw inferences. The qualitative phase may result in the development of research instruments or theory in exploratory sequential designs, followed by a quantitative phase to test the theory of the instrument and to generalise the results to a larger population (Creswell & Creswell 2018). The explanatory design begins with the collection and analysis of quantitative data, followed by the qualitative phase to explain the quantitative outcomes.

Integration in mixed methods research

Simply collecting qualitative and quantitative data is not regarded as MMR (Creswell & Creswell 2018) because an integration of approaches is necessary for a study to be considered MMR. Although Uprichard and Dawney (2019) argued that MMR may produce ‘cuts’ that may not be easily integrated, integration remains the ultimate distinguishing feature of MMR studies. However, ‘meaningful integration of qualitative and quantitative data remains elusive and needs further development’ (Guetterman, Fetters & Creswell 2015:554).

Mixed methods research entails the integration or mixing of qualitative and quantitative components at multiple levels, including epistemology and ontology, methodology and methods. Data integration is mandatory in mixed methods studies (Bazeley 2019; Creswell & Plano Clark 2019). Integration is not mandatory in studies that are multimethod (Plano Clark & Ivankova 2016). Viewing MMR as merely mixing or combining quantitative and qualitative research methods undermines the full potential of MMR (Creamer 2018). Integrating qualitative and quantitative research methods is intentionally done in MMR studies. It can occur either concurrently or sequentially in both basic and advanced MMR studies (Creswell & Plano Clark 2018). Plano Clark and Ivankova (2016) distinguished MMR studies as ‘truly mixed’ from ‘quasi-mixed’. In the latter, the quantitative and qualitative approaches remain separate, whilst in the former, the two methodologies are fully integrated.

The points of mixing should occur at the design stages of research questions, data collection, analysis and interpretation (Sandelowski 2014), depending on the MMR design used in the study. A study should have a holistic approach to integration for it to be truly mixed. In other words, it needs to collect ‘evidence for integration’ with four research dimensions including ‘research design, data collection, mixing phase and findings interpretation (DCMF)’ (Zhou & Wu 2020:10).

Methodological transparency

There is a need for transparency in using and reporting MMR studies (Granikova et al. 2020; Ngulube & Ukwoma 2021). The evaluation of the research methodology as depicted in Figure 1 assists researchers to partly achieve methodological transparency because it contains all the ingredients of basic MMR designs, which are the basis of the complex or advanced designs. The issue of quality and methodologically transparency is relatively neglected and under-researched (Guetterman 2017; Ngulube & Ukwoma 2021).

Methodological transparency is key in establishing rigour and quality in the use of this third methodology, whose ‘time has come’. The following strategies for achieving methodological transparency have been suggested in the literature (Creswell & Hirose 2019; Creswell & Plano Clark 2018). Researchers should

• declare philosophical underpinnings of the study (philosophical transparency)
• demonstrate intentionality to combine qualitative and quantitative components by labelling the study as mixed methods
• articulate the purpose of mixing methods
• specify the sequence of methods
• stipulate the weighting of methods (emphasis of approaches)
• state areas of consistency and inconsistency between methods
• specify the stage of integration, including when, where and how it occurred (level of mixing)
• explain the value-added of using mixed methods.

A researcher who reports on some of the outlined aspects will achieve a certain level of methodological transparency. That may help novice MMR researchers and enhance the credibility and value of MMR studies.

Statement of the problem and the research questions

Mixed methods research that merges qualitative and quantitative methodologies can enrich the repertoire of research methods in the methodological toolkit of scholars who publish their work in SAJEMS, considering that it brings together insights from multiple perspectives and provides information that tells a relatively comprehensive story. Because MMR uses data from different methods, it has the potential of enhancing the understanding of EMS problems and questions. Consequently, it has gained popularity amongst many disciplines (Ngulube 2022; Zou et al. 2018). However, little is known about the extent to which scholars who contribute to SAJEMS have taken a methodological shift and embraced MMR ever since the study of Ngulube and Ngulube (2015).

The principle of reviewing existing studies is at ‘the heart of academic scholarship and the philosophy of science’ (Oliver, Thomas & Gough 2018). It is important to conduct such studies to provide scholars with information that illustrates the methodological choices that they make and to reflect on any gaps that may exist in their use of research methods. That has the potential to improve their studies and research skills (Molina-Azorin 2016) and build a better world, as articulated in the United Nations’ Sustainable Development Goals, by using MMR (Molina-Azorin & Fetters 2019). A ‘better world’ can be built on knowledge that is balanced and based on methodologies that give a relatively balanced view of social phenomena such as MMR.

The following research questions were formulated to achieve the purpose of this study:

• What methodologies are employed by researchers contributing articles to SAJEMS?
• Which are the commonly cited MMR authorities in articles published in SAJEMS?
• To what extent are the philosophical assumptions declared in articles published in SAJEMS?
• How is the appropriateness of MMR described in articles published in SAJEMS?
• Which are the MMR designs used in articles published in SAJEMS?
• How was integration achieved in articles published in SAJEMS?
• What is the level of methodological transparency in MMR articles published in SAJEMS?

Research methodology

Content analysis was employed to study the use of MMR in EMS in articles published between 2012 and 2019. Content analysis research approaches include impressionistic, intuitive, interpretive, systematic and textual analyses (Rosengren 1981). Specifically, the type of analysis undertaken was a systematic one. A multilayered sampling scheme was used in this study (Alise & Teddlie 2010). The first phase of the content analysis involved labelling every article as empirical and nonempirical, followed by categorising articles as qualitative, quantitative and MMR. Categorial data analysis resulted in quantitative data, as illustrated in Table 1.

The following sections describe the sample and the coding procedure to make the research method accountable and transparent.

Sample

The SAJEMS was chosen as a case study because this study is based on the previous study conducted on the articles published in the Journal in 2015. Secondly, the content of the Journal is freely accessible, as it is an open access journal. Accessibility of the content of the Journal was one of the major considerations in selecting the journal. The scope of the Journal is interdisciplinary research in EMS and a ‘leading South African–based publication’ (Journal information [SAJEMS] 2020). That makes the Journal a perfect candidate for testing the utilisation of MMR by its contributors to find out if they are exploiting the potential of MMR to support interdisciplinary and multifaceted research and thus break down ‘intellectual silos’ (Journal information [SAJEMS] 2020). The study period was from 2012 to 2019, which is sufficiently long enough to determine the essence of scholarly communication in a field. A generally recommended time span for measuring scholarly communication is 5 years or more (Pendlebury 2010).

Coding procedures

Methodological indicators are highly demanded by scholars who are interested in the development of a discipline. The methodological indicators were determined through coding procedures after determining the level of analysis. Two raters were involved in coding the variables of empirical and nonempirical studies. Following Ngulube and Ngulube (2015), a total of 34 studies that did not report data and provided conceptual insights and literature reviews were categorised as nonempirical (see Table 1). A total of 328 empirical studies were further coded for the variables for ‘qualitative’, ‘quantitative’, ‘triangulation’, ‘multimethods’, ‘mixed methods’ and ‘mixed methods research’.

The degree of consistency of coding amongst the first and second authors was measured using Cohen’s kappa (κ) (Cohen 1960). Kappa values of 0.40–0.60 are considered as fair, 0.60–0.75 as good and over 0.75 as excellent (Bakeman & Gottman 1997). Fifty articles were randomly selected to evaluate the coding decisions of two coders. Coding consistency for the classification of empirical studies between the first author and the second author was 0.855, and that of the second author and a postgraduate fellow was 0.93, giving a median of 0.893. Intercoder reliability for the variables was excellent.

Research in economic and management science can be classified broadly as quantitative, qualitative and MMR. The definition and descriptions of research methodologies were informed by the taxonomy advanced by Creswell and Creswell (2018). The quantitative designs were categorised as descriptive, correlation, experimental, survey, case study and causal comparatives. Indicators for qualitative designs were grounded theory, ethnography, phenomenology, case study and narratives. The MMR designs were classified as basic and advanced. The framework was chosen because it allowed the purpose of the study to be explored, and it is validated.

For the classification of the research methodologies, the consistency between the two authors was 0.672 and between one of the authors and a postgraduate fellow was 0.675. The median of the scores is 0.673. It is evident that the intercoder reliability can be regarded as good. The major source of discrepancies is that identifying studies that utilise MMR is not straightforward, as suggested by Wilkinson and Staley (2019). The variation of the use of the term ‘mixed methods’ is not without its problems. For instance, some studies do not label their studies as mixed methods studies, even if they are utilising MMR. On the other hand, some studies are self-labelled as MMR, even if they are not using any MMR designs.

This is where manually checking the whole article has an added advantage in contrast to automated searches which focus on predefined keywords. Eleven articles out of a total of 328 empirical articles were first classified as MMR. Further analysis revealed that two self-labelled MMR articles had to be reclassified as multimethods, as the integration of qualitative and quantitative research methods was not evident. The remaining nine articles that were finally classified as MMR were the units of analysis of this study (see Table 1). Cook et al. (2019) used a unit of analysis of 12 out of a total of 146 empirical articles. On the other hand, Ngulube and Ngulube (2015) used 4 out of a total of 266 empirical articles as their unit of analysis.

Building on the framework in Figure 1 and the theoretical background discussed in previous sections, the nine studies that were identified as MMR were further coded. One coder checked for the philosophical assumptions, MMR sources that were cited in the articles, justification for mixing, the specific MMR design, data integration and level of methodological transparency exhibited by the article. The second author cross-checked the coding made by the first author to ensure there were no errors in the coding.

Ethical considerations

Ethical clearance to conduct this study was obtained from the University of South Africa College of Graduate Studies Research Ethics Review Committee (ref. no. 2021/CGS/02/R).

Discussion of the results

The results are organised and discussed around the research questions that guided the study.

Methodologies employed by economic and management sciences researchers

Figure 2 and Table 1 illustrate that methodologies used in SAJEMS are rooted in positivism and post-positivism, with most of the research based on quantitative methods.

FIGURE 2: Methodological trends in the Journal.

Figure 2 demonstrates that the use of qualitative methods, multimethods and MMR was limited. As highlighted by Hesse-Biber (2010), the dependence of researchers on one traditional and classical methodology, such as the quantitative one, limits their ability to tackle complex research problems such as race, nationality, class, gender and poverty. This implies that researchers who publish in SAJEMS may face the same predicament, resulting in their research becoming devoid of the transformative agenda and failing to give space to the marginalised in society. Researchers publishing in SAJEMS need the multiple and complex investigatory tools offered by MMR (Morse & Niehaus 2009). It is important to note that researchers who use both qualitative and quantitative methods are ‘on solid epistemological ground’ (Garrison & Shale 1994:25).

A total of 2.74% (see Table 1) of the article published in the journal used MMR. Ngulube and Ngulube (2015) established a prevalence rate of 2% (out of 332 articles). The low prevalence rates are not peculiar to EMS. Zou et al. (2018) revealed that the prevalence rate amongst occupant behaviour researchers was 5.22% (out of 230 articles). The low prevalence rates of MMR studies may partly be explained by the fact that the use of MMR is not without its challenges.

One of the difficulties has to do with reconciling paradigms and positioning oneself paradigmatically (Creamer 2018). Publishing MMR may also be constrained by a lack of reviewers with expertise in the methodology and space limitations in journals, as MMR articles may be too long, owing to the need to report both qualitative and quantitative data (Creswell & Plano Clark 2018). The methodology used in this study was not able to establish why MMR is not prevalent. There is a need for further studies to investigate the under-representation of MMR in SAJEMS.

Most-cited mixed methods research sources

Knowledge of the methodology and its appropriate application relies on the extant literature on MMR. The diversity in the understanding of MMR in the MMR community has resulted in variations in the use of mixed methods in the literature (Creamer 2018). It is important to consult the leading authorities, such as Creswell, Greene, Johnson, Onwuegbuzie, Plano Clark, Tashakkori and Teddlie (Wilkinson & Staley 2019) to fully understand the major characteristics of MMR and the debates associated with the methodology. Table 2 shows that some theorists such as Creswell, Greene and Onwuegbuzie were consulted, although the focus was on the book by Creswell (2003). It is evident from the data presented in the following that some conceptual problems in applying MMR in the various studies might have emanated from the limited consultation of MMR information sources.

Citing sources when carrying out a study demonstrates the link between it and the existing body of knowledge. Four of the nine studies cited MMR-related sources. Three of the cited sources were found in one study. The findings show that the studies that used MMR had a weak link with the existing body of MMR literature and knowledge. Mixed methods research is a methodological perspective with its own philosophical stance, vocabulary and techniques. It implies that researchers who use the methodology should read the literature in order to use the methodology appropriately and transparently (Ngulube 2022).

Philosophical assumptions of the articles

Philosophical assumptions matter when making research decisions (Creamer 2018; Fetters & Molina-Azorin 2017). None of the nine studies declared their epistemological or ontological stances. The failure to declare the philosophical assumptions of studies is not peculiar to EMS. Khoo-Lattimore, Mura and Yung (2019) found that most of the articles in tourism studies also neglected to state their philosophical assumptions. This can be explained in various ways, but the methodology used in this study does not provide for answering the ‘why’ question. However, Creamer (2018) views the inclusion of the philosophical foundations of research as ‘almost obligatory’ when reporting MMR studies. In that light, it is important to reflect on the philosophical stance when reporting MMR studies, despite the views one may hold about the inclusion of such information in a research article.

It is incumbent for MMR researchers to declare their philosophical assumptions, because a diversity of localised MMR philosophical conventions are emerging in the MMR community. In fact, Creswell and Hirose (2019) and Fetters and Molina-Azorin (2019) advocate for the expansion of MMR’s traditional pragmatic philosophical stance, which is Anglo-Saxon oriented, to include world views of other cultures. That implies that MMR researchers should acknowledge that philosophical perspectives can be culturally bound and context-specific, making it important to declare and explain them to achieve philosophical transparency.

Appropriateness of mixed methods research to a study

There are various reasons why researchers use MMR in their studies. It is incumbent upon them to describe the rationale of using it to demonstrate its appropriateness. According to Creamer (2018), transparency about the rationale for using MMR is one of the essential elements in evaluating the quality of MMR studies. Unlike in the study of Ngulube and Ngulube (2015), where all studies gave some rationale of using MMR, only six out of the nine studies clearly articulated the justification for using MMR. Instrument development was given as the reason for using MMR by two studies. Two other studies that used MMR wanted to explain the quantitative data using qualitative methods. One article wanted to expand the study by gathering supplemental data as suggested by Creswell and Creswell (2018). One study stated its reasons of using a convergent MMR design as to obtain data ‘to triangulate data sources and ensure a comprehensive analysis of the research problem’.

At the beginning, the researchers should demonstrate their intention to mix by stating the purpose of mixing. The researchers should also reflect on the insights and inferences they are producing by the utilisation of MMR before concluding their study (Creamer 2018). Results of this study show that five of the nine studies were explicit about how their studies benefited from combining approaches.

Mixed methods research designs

Mixed methods research designs come in many shapes and sizes (Gosh 2016; Ngulube 2022). Researchers show that they can clearly distinguish between multimethods and MMR by specifying their research designs. It also shows their intention to integrate qualitative and quantitative approaches in a certain time orientation with a predetermined weighting of the qualitative and quantitative components of the study. A total of six out of the nine theses that were self-labelled as MMR did not specify the MMR design. However, the theses were classified as MMR because they collected both qualitative and quantitative data and integrated it at one or more phases of the research process. Thus, the research designs were identifiable using the lens described in the theoretical background section. Specifically, the rationale of mixing and the integration of the qualitative and quantitative components were the primary indicators.

Four studies used the convergent MMR design. The sequential MMR designs were used in the remaining five studies. The exploratory design was used in three studies, and the explanatory design accounted for the other two articles. The findings were similar to a study by Khoo-Lattimore et al. (2019) that revealed that sequential mixed methods designs were more prevalent than the convergent designs. Ngulube and Ngulube (2015) made the same conclusions. This is despite the claim by Creswell and Plano Clark (2018) that the convergent design method’s convergent parallel design is the most familiar of the basic MMR designs.

Integration in mixed methods research

Integration distinguishes mixed methods studies from multimethods studies. As Creswell and Plano Clark (2018) pointed out, simply collecting qualitative and quantitative data does not mean a study uses MMR unless there is integration in one or more phases of the study. Morse (1991) developed a notation system to denote the weighting of research approaches. The uppercase letters (i.e. QUAN, QUAL) suggest a major emphasis on the form of data collection and the lowercase ones (i.e. quan, qual) denote less emphasis. An arrow (→) indicates that data collection is sequential. On the other hand, a plus (+) sign shows that quantitative and qualitative data are collected simultaneously. Therefore, mixing can either be sequential or concurrent with varying emphasis on qualitative and quantitative approaches. For example:

• QUAN → qual: dominant quantitative component preceding the qualitative (explanatory sequential)
• QUAL → quan: qualitative strand followed by the quantitative one, whilst it is subsidiary to the qualitative strand (exploratory sequential)
• QUAN + qual: quantitative and qualitative data are collected simultaneously but the quantitative component is dominant (concurrent or parallel)

Table 3 illustrates how the studies integrated methods at the design level.

Table 3 shows that only one study gave equal emphasis on the quantitative and qualitative strands. The study also clearly stated the intention to mix. Either the quantitative or qualitative component was dominant in the other eight studies. The intention to mix either sequentially or concurrently was not stated, as the study designs were not indicated. Ngulube and Ngulube (2015) revealed that one study was QUAL → quan and the other three were QUAN + qual. However, just like in the current study, the MMR designs were not specified. The studies also gave limited details on the integration of qualitative and quantitative methods.

Integration at the interpretation level was evident in all the studies. That was achieved through narrating and discussing qualitative and quantitative results or mixing the data through visual means such as tables, figures and joint displays, as suggested by Guetterman et al. (2015). The findings were not significantly different from those of Ngulube and Ngulube (2015).

Methodological transparency amongst mixed methods studies

Methodological transparency entails being ‘transparent in terms of clarifying the logic underpinning the inquiry’ (Collins, Onwuegbuzie & Johnson 2012:850). Authors should be explicit about the (1) research design that guides the researcher in carrying out the study; (2) research philosophy which shows the stance of the researcher; (3) research approach, whether the study is qualitative, quantitative or a mixed methods; (4) research strategy that the researcher employed in the course of undertaking the research; (5) data collection methods; and (6) analysis of the collected data (Wilson 2014:7). Methodological transparency supports the legitimation of MMR studies by providing conclusions that are ‘credible, trustworthy, dependable, transferable and/or confirmable’ (Onwuegbuzie & Johnson 2006:52). In other words, methodological transparency ensures inference quality and rigour in a study and should appear at the top of any list to assess the quality of MMR studies (Creamer 2018).

The diversity of philosophical assumptions that may influence MMR implies that researchers must be transparent by making the philosophical assumptions open. There are also various MMR designs. This implies that studies must clearly state their philosophical stance and their MMR design to demonstrate their intentionality to use MRR. The philosophical underpinning of the study should also be articulated to demonstrate that the intention is to move away from the classical positivist and interpretivist philosophical assumptions. The results revealed that philosophical assumptions were rarely declared. The labelling of the study as mixed methods and specifying the MMR design of the study is a good indicator of how the researcher understands MMR. Giving reasons for the purpose of using MMR also constitutes methodological transparency. The results show that the studies were deficient in all these aspects, demonstrating that they were low on methodological transparency.

Above all, integration should be described because that is what distinguishes MMR studies from the multimethod ones. It is also important to explain where, how and why integration occurred in a study, because integration is ‘the heart and soul’ of MMR (Guetterman, Molina-Azorin & Fetters 2020:430). Ideally, integration should take place in all the phases of the study (Creamer 2018). Results indicate that integration was mainly confined to the analytic and interpretation phases. Finally, the benefits of using MMR must be reflected upon. Results indicate that only five studies reflected on the benefits of using MMR, but none of the studies stated the limitations of using MMR. The results of this study have implications for methodological transparency and the quality of MMR studies reported in SAJEMS.

Summary and conclusions

This study advanced the new line of research that assesses the pervasiveness of mixed methods in a cognitive discipline. The article identified methodological trends in articles published in SAJEMS and described how qualitative and quantitative researches were integrated in the articles. Although many scholars are of the view that multiple ways of conducting research are commonplace, researchers contributing to the Journal have not adequately adopted multiple ways of conducting research by embracing MMR. The use of MMR and qualitative methodologies was limited. Many studies were not explicit about their philosophical assumptions, appropriateness of MMR, MMR designs, the integration of approaches and the value-added of MMR. Limited consultations with relevant MMR literature were also evident.

Economic and management researchers that contributed to the Journal seem to pay allegiance to positivist epistemologies. They break down the ‘intellectual silos’ as envisaged in the strategic intention of SAJEMS. Being rooted in traditional research methodology such as the quantitative methodology means that researchers who publish in SAJEMS have a limited potential to explore complex problems relating to social justice and social change. A possible explanation for the low uptake is that MMR is still evolving, with various variations based on the perspective of researchers in the MMR community. Using MMR will have the effect of reducing the quantitative-qualitative methodological divide in economic and management science research reported in SAJEMS. Taking into consideration some of the issues raised in this article is likely to assist scholars that publish in the Journal to produce good quality mixed methods studies and strengthen their research by integrating qualitative and quantitative research, despite the limitations that MMR might pose.

Despite its strength, the suggestion is not that MMR should replace established research traditions, including qualitative and quantitative research. This study is careful not to fall into the trap of promoting MMR as a methodology ahead of others, as that might limit innovation and the possible combinations that happen within research methods. However, scholars who contribute to SAJEMS should develop a research agenda that incorporates MMR because of the value-added of MMR. Consequently, researchers are going to overcome the limitations of using one methodology.

Researchers investigate social phenomena to provide a better understanding of the real world. That can be partly achieved by providing a comprehensive picture of social reality. Mixed methods research provides an opportunity for researchers and journalists to disseminate knowledge that paints a relatively complete picture about a phenomenon. Furthermore, MMR supports the ‘mixed methods way of thinking’ (Greene 2007:20), which accommodates other philosophical assumptions. That implies the researchers have a chance of integrating their own philosophical assumptions when conducting MMR, leading to the production of contextually relevant knowledge.

Study limitations

The conclusions made in this article are based on a limited number of articles. The recommendations are drawn from conceptual and methodological literature, and the experience of the authors conducting MMR is ignored. Content analysis is sometimes criticised for ignoring the text despite it being an unobtrusive and low-cost research method. Another methodology that explains the prevalence rates of MMR that were unearthed by this study might paint a different picture. Interviews with productive researchers in EMS may partially address the deficiencies of the reductionist approach inherent in content analysis used in this study. The absence of a universal classification scheme for methodological indicators in scholarly communication is another limitation, as this study relied on various classification frameworks in order to formulate the coding scheme for this study.

Acknowledgements

Thanks to the two anonymous reviewers for their time and very helpful comments on earlier versions of this work and the two independent coders who helped in resolving the coding difference between the researchers.

The authors declare that they have no financial or personal relationships that may have inappropriately influenced them in writing this article.

P.N. contributed 60% of the inputs to the chapter and B.N. accounted for the 40% as described earlier.

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Data used in the research are in the public domain, as the study was based on content analysis of journal articles that are published in an open access environment.

The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of any affiliated agency of the authors.

References