top of page

My Experience Using Quantitative Research Methods During My PhD in Health Psychology

Writer: Cheryl Mazzeo
Cheryl Mazzeo
Nov 10
8 min read
PhD Research Methods Help

My Experience Using Quantitative Research Methods During My PhD in Health Psychology

Reflections on research design, statistical analysis, and the challenges of conducting an independent doctoral study.


When I began my PhD in Health Psychology, I knew that quantitative research would be an important component of my doctoral education. My previous academic and professional experiences had exposed me to statistical analysis, research design, and data collection. However, conducting an independent quantitative dissertation presented challenges that were different from completing statistical assignments or contributing to larger research projects.


My dissertation examined cannabis use, adverse event burden, and quality of life among individuals with gastrointestinal cancers. Specifically, I investigated whether adverse event burden mediated the relationship between cannabis use and quality of life. This required me to develop a research design, select appropriate measurement instruments, recruit participants, prepare a dataset, conduct statistical analyses, and interpret the results.


The experience reinforced my understanding that quantitative research involves much more than selecting statistical tests and reporting p-values. Each stage requires methodological decisions that can influence the validity, interpretation, and usefulness of the findings.

Developing a Quantitative Research Question

One of the first challenges was translating a broad research interest into a question that could be investigated quantitatively.


I was interested in the relationship between cannabis use and quality of life among individuals with gastrointestinal cancers. However, simply examining whether cannabis users reported better or worse quality of life would provide a limited understanding of the relationship.


I wanted to explore whether adverse event burden might help explain this association.


This led me to develop a mediation model with three primary variables:

  • Independent variable: Cannabis use

  • Mediator: Adverse event burden

  • Dependent variable: Quality of life


The distinction between examining a direct association and investigating a potential mediating relationship was important. A mediation analysis could help evaluate whether cannabis use was statistically associated with quality of life through its relationship with adverse event burden.


However, because the study used a cross-sectional design, I also had to recognize that the results could not establish causation or the temporal sequence of these relationships.


This was an important methodological consideration. Quantitative models can be mathematically sophisticated without necessarily providing evidence of causal relationships.


Selecting Measurement Instruments

Another major component of the dissertation involved selecting instruments that could operationalize the variables.


Rather than developing entirely new questionnaires, I selected established measures appropriate for the constructs I wanted to investigate.


Cannabis use was measured using the Daily Sessions, Frequency, Age of Onset, and Quantity of Cannabis Use Inventory (DFAQ-CU). Adverse event burden was assessed using selected items from the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Quality of life was measured using the Functional Assessment of Cancer Therapy–General (FACT-G).


Selecting instruments required considering several issues, including whether the measures aligned with the research questions, how their scores would be calculated, and whether the resulting data would be appropriate for the planned statistical analyses.


I also had to consider the difference between measuring a broad construct and measuring only selected aspects of that construct.


For example, adverse event burden can encompass many different physical symptoms and treatment-related experiences. Using five selected PRO-CTCAE items provided a defined operational measure, but it did not capture every possible adverse event experienced by individuals with gastrointestinal cancers.


This experience reinforced the importance of clearly distinguishing between a theoretical construct and the specific variables used to represent it.


The Reality of Participant Recruitment

Participant recruitment was one of the most practical challenges of conducting the dissertation.


My study required individuals with gastrointestinal cancers who could provide information about cannabis use, adverse events, and quality of life. Recruiting from a specific clinical population introduced considerations that would not necessarily arise in a study involving a general undergraduate population.

I ultimately collected 116 survey responses. After applying the study's screening and data-quality criteria, 85 participants remained in the primary analytic sample.

This reduction illustrated an important distinction between the number of responses collected and the number of observations available for statistical analysis.


Recruitment planning cannot focus exclusively on reaching a target number of responses. Researchers must also anticipate exclusions, incomplete responses, and other factors that may reduce the usable sample.


I also learned that recruitment is a resource-management problem as much as a methodological one. Time, financial costs, eligibility requirements, and data quality all influence the feasibility of a quantitative study.


For doctoral students working independently, these practical constraints can be especially important because they may not have access to the recruitment infrastructure available to large research teams.


Preparing the Data for Analysis

One aspect of quantitative research that is sometimes underestimated is the amount of work required before statistical analysis can begin.


Collecting survey responses does not automatically produce a dataset that is ready for hypothesis testing.


The data must first be examined to determine whether participants meet the eligibility criteria, whether variables have been coded correctly, and whether the measures have been scored according to their intended procedures.


For my dissertation, this included preparing cannabis-use measures, calculating adverse event burden, and scoring quality-of-life responses. I also had to consider the distribution of variables and whether the assumptions of particular statistical procedures were appropriate.


These decisions matter because statistical software will often produce results even when the underlying data or analytic choices are unsuitable.

The responsibility for determining whether an analysis is meaningful remains with the researcher.


My previous experience with statistical software and research data was helpful, but conducting the dissertation required applying those skills within a complete research process rather than addressing isolated analytical questions.


Conducting the Statistical Analyses

The primary analysis used PROCESS Model 4 to investigate whether adverse event burden mediated the relationship between cannabis use and quality of life.

The mediation analysis used 5,000 bootstrap samples to estimate the indirect effect.


I also used nonparametric statistical procedures, including Spearman correlations and Mann–Whitney U tests, to examine relevant relationships and group differences.


One of the most important lessons was that statistical methods should be selected based on the research questions and characteristics of the data, rather than simply choosing the most sophisticated analysis available.


For example, mediation analysis was appropriate for investigating the hypothesized indirect relationship, but it did not eliminate the limitations associated with the study design or sample.


Similarly, nonparametric methods provided alternatives for examining relationships and differences when conventional parametric assumptions were not appropriate.


The challenge was not simply learning how to execute these analyses. It was understanding what each test could establish and how the results should be interpreted.


When the Results Do Not Support the Hypothesis

Perhaps the most valuable experience was interpreting results that did not fully support the proposed mediation model.


The analysis identified a statistically significant negative association between adverse event burden and quality of life. Participants reporting greater adverse event burden generally reported poorer quality of life.


The model explained approximately 21.9% of the variance in quality of life.

However, the indirect effect of cannabis use on quality of life through adverse event burden was not statistically significant. The total effect of cannabis use on quality of life was also not statistically significant.


These findings required careful interpretation. It would have been inappropriate to suggest that the hypothesized mediation relationship existed simply because one component of the model was statistically significant.


Likewise, a nonsignificant result did not establish that cannabis use had no relationship with quality of life under all circumstances.


The results had to be understood within the context of the sample, measurement instruments, research design, and statistical uncertainty.


This experience reinforced one of the most important principles of research: The purpose of statistical analysis is to investigate a question, not to produce a statistically significant result.


A dissertation does not become unsuccessful because its primary hypothesis is unsupported.


In fact, accurately reporting an unsupported hypothesis can make a meaningful contribution by identifying limitations in an existing explanation and informing future research.


Understanding the Limitations of Quantitative Research

Conducting this dissertation also made me more aware of the limitations of quantitative methods.


Quantitative research provides a systematic approach to examining relationships, estimating effects, and evaluating hypotheses. However, numerical data do not necessarily explain the experiences underlying those relationships.

For example, a participant's quality-of-life score may reflect physical symptoms, emotional well-being, functional limitations, social circumstances, and other factors.


Similarly, cannabis use may vary considerably in frequency, quantity, purpose, and individual experience.


Although measurement instruments can quantify aspects of these experiences, they cannot necessarily capture every relevant contextual factor.


My cross-sectional design also limited the conclusions that could be drawn about the direction of relationships.


These limitations did not make the study unimportant. Instead, they helped define what the findings could reasonably contribute to the literature.


I came to view methodological limitations not as weaknesses that should be minimized in the discussion, but as essential information for interpreting the research responsibly.


Quantitative Research Versus Statistical Software

Another lesson was the distinction between knowing how to use statistical software and understanding quantitative research methodology.


Software can calculate correlations, estimate regression coefficients, generate confidence intervals, and perform mediation analyses. However, it cannot independently determine whether the research question is meaningful, whether the instruments adequately represent the constructs, or whether the conclusions are justified.


For doctoral researchers, the greater challenge is often conceptual rather than computational.


A researcher needs to understand why a particular analysis is being performed, which assumptions are relevant, and what alternative explanations might account for the findings.


This is especially important when statistical output contains numerous coefficients, confidence intervals, and significance tests.


It is possible to report these values accurately while misunderstanding their implications.


My dissertation required me to connect statistical results back to the original research questions and explain them in a way that would be understandable to readers who might not have extensive statistical training.


That process was as important as performing the analyses themselves.


What I Would Do Differently

Reflecting on the experience, I can identify several considerations that I would emphasize when planning another quantitative study.


First, I would continue to prioritize the alignment between the research question, measurement instruments, and planned analyses before recruitment begins.


Second, I would account for the possibility that the final analytic sample could be substantially smaller than the total number of responses collected.


Third, I would give particular attention to how the study design influences the interpretation of statistical relationships. A complex statistical model does not compensate for limitations in the underlying design.


Finally, I would continue to approach nonsignificant findings as legitimate research outcomes rather than problems requiring an alternative analysis to produce significance.


These lessons are relevant beyond dissertation research. They apply to clinical research, public health investigations, health psychology, and other fields that rely on quantitative evidence.


Final Reflections

Completing the quantitative components of my PhD dissertation was an opportunity to apply research methods to a substantive health-related question.

The experience involved considerably more than running statistical tests. It required developing a defensible research design, operationalizing variables, managing participant recruitment, preparing data, evaluating assumptions, and interpreting findings within the limitations of the methodology.


It also reinforced the distinction between statistical significance and scientific importance.


Although the hypothesized mediation relationship was not supported, the study identified an important association between adverse event burden and quality of life among individuals with gastrointestinal cancers.


More broadly, the experience strengthened my understanding of what it means to conduct independent quantitative research.


The most important lesson I took from the process was that good quantitative research is not defined by whether the results confirm the researcher's expectations. It is defined by the quality of the methods, the transparency of the analysis, and the accuracy of the conclusions.


For students considering a quantitative dissertation, I would encourage them to think beyond the statistical tests they expect to use. The real work involves understanding the research problem, making defensible methodological decisions, and accepting the findings that emerge from the data—even when those findings differ from what was originally anticipated.

Comments


bottom of page