Choosing a Research Design

Diving Deeper into Limitations and Delimitations

If you are working on a thesis, dissertation, or other formal research project, chances are your advisor or committee will ask you to address the delimitations of your study. When faced with this request, many students respond with a puzzled look and then go on to address what are actually the study’s limitations.

If you’re wondering what the difference between these two terms is, don’t worry—you’re not alone!

In a previous article , we covered what goes into the limitations, delimitations, and assumptions sections of your thesis or dissertation. Here, we will dive a bit deeper into the differences between limitations and delimitations and provide some helpful tips for addressing them in your research project—whether you are working on a quantitative or qualitative study.

Acknowledging Weaknesses vs. Defining Boundaries

These concepts are easy to get confused because both limitations and delimitations restrict (or limit) the questions you’ll be able to answer with your study, most notably in terms of generalizability.

However, the biggest difference between limitations and delimitations is the degree of control you have over them—that is, how much they are based in conscious, intentional choices you made in designing your study.

Limitations occur in all types of research and are, for the most part, outside the researcher’s control (given practical constraints, such as time, funding, and access to populations of interest). They are threats to the study’s internal or external validity.

Limitations may include things such as participant drop-out, a sample that isn’t entirely representative of the desired population, violations to the assumptions of parametric analysis (e.g., normality, homogeneity of variance), the limits of self-report, or the absence of reliability and validity data for some of your survey measures.

Some limitations are inherent to your research design itself. For example, you won’t be able to infer causality from a correlational study or generalize to an entire population from a case study. Likewise, while an experimental study allows you to draw causal conclusions, it may require a level of experimental control that looks very different from the real world (thus lowering external validity). Of course, your choice of research design is within your control; however, the limitations of the design refer to those aspects that may restrict your ability to answer the questions you might like to answer.

Limitations can get in the way of your being able to answer certain questions or draw certain types of inferences from your findings. Therefore, it’s important to acknowledge them upfront and make note of how they restrict the conclusions you’ll be able to draw from your study. Frequently, limitations can get in the way of our ability to generalize our findings to the larger populations or to draw causal conclusions, so be sure to consider these issues when you’re thinking about the potential limitations of your study.

Delimitations are also factors that can restrict the questions you can answer or the inferences you can draw from your findings. However, they are based on intentional choices you make a priori (i.e., as you’re designing the study) about where you’re going to draw the boundaries of your project. In other words, they define the project’s scope.

Like limitations, delimitations are a part of every research project, and this is not a bad thing. In fact, it’s very important! You can’t study everything at once. If you try to do so, your project is bound to get huge and unwieldy, and it will become a lot more difficult to interpret your results or come to meaningful conclusions with so many moving parts. You have to draw the line somewhere, and the delimitations are where you choose to draw these lines.

One of the clearest examples of a delimitation that applies to almost every research project is participant exclusion criteria. In conducting either a quantitative or a qualitative study, you will have to define your population of interest. Defining this population of interest means that you will need to articulate the boundaries of that population (i.e., who is not included). Those boundaries are delimitations.

For example, if you’re interested in understanding the experiences of elementary school teachers who have been implementing a new curriculum into their classrooms, you probably won’t be interviewing or sending a survey to any of the following people: non-teachers, high-school teachers, college professors, principals, parents of elementary school children, or the children themselves. Furthermore, you probably won’t be talking to elementary school teachers who have not yet had the experience of implementing the curriculum in question. You would probably only choose to gather data from elementary school teachers who have had this experience because that is who you’re interested in for the purposes of your study. Perhaps you’ll narrow your focus even more to elementary school teachers in a particular school district who have been teaching for a particular length of time. The possibilities can go on. These are choices you will need to make, both for practical reasons (i.e., the population you have access to) and for the questions you are trying to answer.


Of course, for this particular example, this does not mean that it wouldn’t be interesting to also know what principals think about the new curriculum. Or parents. Or elementary school children. It just means that, for the purposes of your project and your research questions, you’re interested in the experience of the teachers, so you’re excluding anyone who does not meet those criteria. Having delimitations to your population of interest also means that you won’t be able to answer any questions about the experiences of those other populations; this is ok because those populations are outside of the scope of your project. As interesting as their experiences might be, you can save these questions for another study. That is the part of the beauty of research: there will always be more studies to do, more questions to ask. You don’t have to (and can’t) do it all in one project.

Similarly, the focus of the research problem itself (and the associated research questions) is another common source of delimitations. By choosing to focus your research on a particular problem or question, you are necessarily choosing not to examine other problems or questions. Remember: You can’t answer all possible questions with one project. While this may seem obvious, it’s worth acknowledging. There may be other related problems or questions that are equally worthy of study, but you must choose which one(s) you are and which ones you are not looking into with your project.

Continuing with the previous example, for instance, let’s suppose that the problem you are most interested in addressing is the fact that we know relatively little about elementary school teachers’ experiences of implementing a new curriculum. Perhaps you believe that knowing more about teachers’ experiences could inform their training or help administrators know more about how to support their teachers. If the identified problem is our lack of knowledge about teachers’ experiences, and your research questions focus on better understanding these experiences, that means that you are choosing not to focus on other problems or questions, even those that may seem closely related. For instance, you are not asking how effective the new curriculum is in improving student test scores or graduation rates. You might think that would be a very interesting question, but it will have to wait for another study. In narrowing the focus of your research questions, you limit your ability to answer other questions, and again, that’s ok. These other questions may be interesting and important, but, again, they are beyond the scope of your project.

Common Examples of Limitations

While each study will have its own unique set of limitations, some limitations are more common in quantitative research, and others are more common in qualitative research.

In quantitative research, common limitations include the following:

- Participant dropout

- Small sample size, low power

- Non-representative sample

- Violations of statistical assumptions

- Non-experimental design, lack of manipulation of variables, lack of controls

- Potential confounding variables

- Measures with low (or unknown) reliability or validity

- Limits of an instrument to measure the construct of interest

- Data collection methods (e.g., self-report)

- Anything else that might limit the study’s internal or external validity

In qualitative research, common limitations include the following:

- Lack of generalizability of findings (not the goal of qualitative research, but still worth mentioning as a limitation)

- Inability to draw causal conclusions (again, not the goal of qualitative research, but still worth mentioning)

- Researcher bias/subjectivity (especially if there is only one coder)

- Limitations in participants’ ability/willingness to share or describe their experiences

- Any factors that might limit the rigor of data collection or analysis procedures

Common Examples of Delimitations

As noted above, the two most common sources of delimitations in both quantitative and qualitative research include the following:

- Inclusion/exclusion criteria (or how you define your population of interest)

- Research questions or problems you’ve chosen to examine

Several other common sources of delimitations include the following:

- Theoretical framework or perspective adopted

- Methodological framework or paradigm chosen (e.g., quantitative, qualitative, or mixed-methods)

- In quantitative research, the variables you’ve chosen to measure or manipulate (as opposed to others)

Whether you’re conducting a quantitative or qualitative study, you will (hopefully!) have chosen your research design because it is well suited to the questions you’re hoping to answer. Because these questions define the boundaries or scope of your project and thus point to its delimitations, your research design itself will also be related to these delimitations.

Questions to Ask Yourself

As you are considering the limitations and delimitations of your project, it can be helpful to ask yourself a few different questions.

Questions to help point out your study’s limitations:

1. If I had an unlimited budget, unlimited amounts of time, access to all possible populations, and the ability to manipulate as many variables as I wanted, how would I design my study differently to be better able to answer the questions I want to answer? (The ways in which your study falls short of this will point to its limitations.)

2. Are there design issues that get in the way of my being able to draw causal conclusions?

3. Are there sampling issues that get in the way of my being able to generalize my findings?

4. Are there issues related to the measures I’m using or the methods I’m using to collect data? Do I have concerns about participants telling the truth or being able to provide accurate responses to my questions?

5. Are there any other factors that might limit my study’s internal or external validity?

Questions that help point out your study’s delimitations:

1. What are my exclusion criteria? Who did I not include in my study, and why did I make this choice?

2. What questions did I choose not to address in my study? (Of course, the possibilities are endless here, but consider related questions that you chose not to address.)

3. In what ways did I narrow the scope of my study in order to hone in on a particular issue or question?

4. What other methodologies did I not use that might have allowed me to answer slightly different questions about the same topic?

How to Write About Limitations and Delimitations

Remember, having limitations and delimitations is not a bad thing. They’re present in even the most rigorous research. The important thing is to be aware of them and to acknowledge how they may impact your findings or the conclusions you can draw.

In fact, writing about them and acknowledging them gives you an opportunity to demonstrate that you can think critically about these aspects of your study and how they impact your findings, even if they were out of your control.

Keep in mind that your study’s limitations will likely point to important directions for future research. Therefore, when you’re getting ready to write about your recommendations for future research in your discussion, remember to refer back to your limitations section!

As you write about your delimitations in particular, remember that they are not weaknesses, and you don’t have to apologize for them. Good, strong research projects have clear boundaries. Also, keep in mind that you are the researcher and you can choose whatever delimitations you want for your study. You’re in control of the delimitations. You just have to be prepared—both in your discussion section and in your dissertation defense itself—to justify the choices you make and acknowledge how these choices impact your findings.

Research Design 101: Research Methods for Students

Which of the many different types of research design is best for you?

One thing that you will want to consider early in your dissertation process is the design of your research study. By the time you start your dissertation or thesis, you have probably taken graduate and undergraduate courses about research methods; however, it has probably been a while since you have taken these courses, and you may need help sorting through all the different types of research design. Below is a brief refresher on different research designs and methodologies.

General Types of Research Designs

Descriptive: Researchers use descriptive research designs to describe particular phenomena or relationships within a single group sample. Descriptive designs are typically used as either pilot or preliminary studies and generally have rather basic statistical procedures. By nature, descriptive studies do not and cannot be used to explain causation.

Descriptive research designs usually provide researchers with information about a group or phenomenon about which there has been little research (e.g., mating patterns of Martians). However, descriptive studies lack randomization and control and cannot be used to determine causation and other implications; in other words, descriptive research designs can only be used to determine “who” and “what,” not “why.”

Quasi-Experimental: Researchers use quasi-experimental research designs to identify differences between two or more groups in an attempt to explain causation. What keeps these types of experiments from being true experiments is lack of randomization. For example, researchers cannot randomly assign gender to participants; therefore, any study in which researchers are investigating differences between genders is inherently quasi-experimental.

Quasi-experimental designs allow researchers more control to make assumptions about causation and implications of findings. Quasi-experimental designs are also useful when researchers want to study particular groups in which group members cannot be randomly assigned (e.g., persons with depression, single mothers, people from different races or ethnic groups, etc.). A major drawback to using quasi-experimental designs is that quasi-experimental research designs typically have less internal validity than do true experimental designs.

Experimental: Experimental research designs have the most control, and, thus, allow researchers to explain differences between groups. One of the key features of an experimental design is that participants are randomly assigned to groups. Experimental designs can be used to test differences between groups (e.g., treatment a group, treatment b group, and control group) or factorial differences within multiple levels of each group (e.g., a drug group [Xanax or Valium] and a psychotherapy group [Cognitive Behavioral Therapy or Rational Emotive Behavioral Therapy]).

True experimental research designs are understood to be the gold standard of research because experimental research designs are the best designs for researchers to predict causation. However, true experimental designs often require more resources than do other research designs and will not work with all research questions.

Specific Types of Research Designs

Single-Sample Repeated Measures: A design method in which the same group is tested at multiple points in time. Giving students an assessment of knowledge the first day of class and giving the same assessment on the last day of class is an example of a research design based on a single-sample repeated measures.

ABA: A specific single-sample repeated measures design in which participants are measured at baseline (A), after an intervention (B), and again after the intervention has been removed (A).

Between Groups: A design in which researchers compare the scores of two or more groups. Between-group designs can be used as either a single or repeated measure.

Matched Sample: A specific between-groups design in which researchers match participants across groups based on criteria determined by the researchers (e.g., age, IQ, gender, etc.). After matching participants based on the predetermined criteria, researchers examine differences between matched pairs (not between group means).

Stating the Obvious: Writing Assumptions, Limitations, and Delimitations

During the process of writing your thesis or dissertation, you might suddenly realize that your research has inherent flaws. Don’t worry! Virtually all projects contain restrictions to your research. However, being able to recognize and accurately describe these problems is the difference between a true researcher and a grade-school kid with a science-fair project. Concerns with truthful responding, access to participants, and survey instruments are just a few of examples of restrictions on your research. In the following sections, the differences among delimitations, limitations, and assumptions of a dissertation will be clarified.

Delimitations

Delimitations are the definitions you set as the boundaries of your own thesis or dissertation, so delimitations are in your control. Delimitations are set so that your goals do not become impossibly large to complete. Examples of delimitations include objectives, research questions, variables, theoretical objectives that you have adopted, and populations chosen as targets to study. When you are stating your delimitations, clearly inform readers why you chose this course of study. The answer might simply be that you were curious about the topic and/or wanted to improve standards of a professional field by revealing certain findings. In any case, you should clearly list the other options available and the reasons why you did not choose these options immediately after you list your delimitations. You might have avoided these options for reasons of practicality, interest, or relativity to the study at hand. For example, you might have only studied Hispanic mothers because they have the highest rate of obese babies. Delimitations are often strongly related to your theory and research questions. If you were researching whether there are different parenting styles between unmarried Asian, Caucasian, African American, and Hispanic women, then a delimitation of your study would be the inclusion of only participants with those demographics and the exclusion of participants from other demographics such as men, married women, and all other ethnicities of single women (inclusion and exclusion criteria). A further delimitation might be that you only included closed-ended Likert scale responses in the survey, rather than including additional open-ended responses, which might make some people more willing to take and complete your survey. Remember that delimitations are not good or bad. They are simply a detailed description of the scope of interest for your study as it relates to the research design. Don’t forget to describe the philosophical framework you used throughout your study, which also delimits your study.

Limitations

Limitations of a dissertation are potential weaknesses in your study that are mostly out of your control, given limited funding, choice of research design, statistical model constraints, or other factors. In addition, a limitation is a restriction on your study that cannot be reasonably dismissed and can affect your design and results. Do not worry about limitations because limitations affect virtually all research projects, as well as most things in life. Even when you are going to your favorite restaurant, you are limited by the menu choices. If you went to a restaurant that had a menu that you were craving, you might not receive the service, price, or location that makes you enjoy your favorite restaurant. If you studied participants’ responses to a survey, you might be limited in your abilities to gain the exact type or geographic scope of participants you wanted. The people whom you managed to get to take your survey may not truly be a random sample, which is also a limitation. If you used a common test for data findings, your results are limited by the reliability of the test. If your study was limited to a certain amount of time, your results are affected by the operations of society during that time period (e.g., economy, social trends). It is important for you to remember that limitations of a dissertation are often not something that can be solved by the researcher. Also, remember that whatever limits you also limits other researchers, whether they are the largest medical research companies or consumer habits corporations. Certain kinds of limitations are often associated with the analytical approach you take in your research, too. For example, some qualitative methods like heuristics or phenomenology do not lend themselves well to replicability. Also, most of the commonly used quantitative statistical models can only determine correlation, but not causation.

Assumptions

Assumptions are things that are accepted as true, or at least plausible, by researchers and peers who will read your dissertation or thesis. In other words, any scholar reading your paper will assume that certain aspects of your study is true given your population, statistical test, research design, or other delimitations. For example, if you tell your friend that your favorite restaurant is an Italian place, your friend will assume that you don’t go there for the sushi. It’s assumed that you go there to eat Italian food. Because most assumptions are not discussed in-text, assumptions that are discussed in-text are discussed in the context of the limitations of your study, which is typically in the discussion section. This is important, because both assumptions and limitations affect the inferences you can draw from your study. One of the more common assumptions made in survey research is the assumption of honesty and truthful responses. However, for certain sensitive questions this assumption may be more difficult to accept, in which case it would be described as a limitation of the study. For example, asking people to report their criminal behavior in a survey may not be as reliable as asking people to report their eating habits. It is important to remember that your limitations and assumptions should not contradict one another. For instance, if you state that generalizability is a limitation of your study given that your sample was limited to one city in the United States, then you should not claim generalizability to the United States population as an assumption of your study. Statistical models in quantitative research designs are accompanied with assumptions as well, some more strict than others. These assumptions generally refer to the characteristics of the data, such as distributions, correlational trends, and variable type, just to name a few. Violating these assumptions can lead to drastically invalid results, though this often depends on sample size and other considerations.

To Dig or Not To Dig: What is Archived Data?

Deciding on a dissertation or thesis topic can be a difficult task in and of itself. Deciding on a dissertation or thesis topic and collecting data based on your topic can at times seem impossible. It may be tempting to find an archival data set that might be tailored to fit your research study. There are a few advantages to using archived data. For example, on the surface, archival data sets save you resources and time because you don’t have to collect your own data. However, though archival data sets may appear to be time-savers, there are many other issues that you should take into account before you choose to use an archival data set.

Often, archived data sets were created to answer a specific set of research questions. Though this seems intuitive, what does not seem so intuitive is the impact that this will have on your study. Because the goal of your dissertation or thesis is to prove to your committee that you have been able to contribute new knowledge to your field of study, it is likely that you are not asking research questions that have already been answered in the literature. It is likely that you will struggle trying to get a data set that was designed to answer specific questions to fit with your specific research questions.

Another common issue with archived data is the sheer amount of data. Large archival data sets often contain data collected over many years. On the surface, it might sound like a benefit to have such a large amount of data; however, this does become problematic when the codebook is difficult to follow and the data collected have changed from year to year. Additionally, archival data can have misleading sample sizes. When large questionnaires are given, participants are only asked certain questions based on their previous answers. If you want to run analyses on a particular item from the questionnaire, chances are that not everyone who took the survey has answered that item. This becomes particularly problematic when you want to investigate relationships among multiple items that may not have been answered by all participants in the study.

Though archival data may seem like an easier way to collect dissertation data, there are additional factors that might actually cause you more work in the long run. If you do choose to use archived data for your dissertation or thesis, be mindful of the potential difficulties you might face.

 
 
 

Finding the Minimum Sample Size

When completing your thesis or dissertation, you will most likely be collecting data and running some statistical analysis on the data that you collect. When writing proposals for their theses and dissertations, students commonly overlook a priori power analysis which can be critical in their study.

What is power?

Statistical power is the ability for statisticians to use statistical tests to determine if significance exists between variables in a study. Power (though in a different metric) is the inverse of the alpha level. Insufficient statistical power increases the likelihood of a type II (or beta) error, which occurs when researchers fail to reject null hypotheses [link to article Introduction to Null Hypothesis Significance Testing] when alternative hypotheses are discovered to be true.

What are the factors relating to power?

Many factors relate to statistical power, such as sample size, significance level, effect size, beta level, number of groups being compared, etc. Increasing the sample size, significance level, or effect size will increase statistical power. Having more group comparisons will lower statistical power.

What is a priori power?

The term a priori comes from Latin and means “from the earlier.” It is a type of power analysis that researchers calculate prior to data collection to determine the minimum sample size to find significance (if significance exists). Calculating a minimum sample size helps researchers maximize their resources. By knowing the minimum number of participants needed for significance, researchers do not waste time collecting more data than they need to determine significance between variables. Additionally, knowing the minimum sample size also helps researchers confirm that they have sufficient data to find significance, which decreases chances of a type II error.

How do you calculate a priori power?

A priori power is calculated from the factors related to power, factors that were previously discussed in the “What are the factors relating to power?” section. Before collecting your data, you will need to determine your alpha level (typically .05), to estimate the expected effect size (it is best to be moderate in your estimation), and to count number of groups being compared (if applicable). The last step is to determine how much power you want for your study. Ideally, when running this type of power analysis, you should set your power to .90; however, some statisticians argue that power as low as .80 is acceptable.

There are many software programs researchers and students can use; G*Power is the most common free software program used.

 
 
 

More Articles...

  1. Data Collection Methods

More Articles on PhDStudent.com

  • Default
  • Title
  • Date
  • Random
load more hold SHIFT key to load all load all

  Still can't find what you're looking for?

Post your question to the forum where grad students, faculty, & more can respond.

Receive instant access to the forum when you create your free account on PhDStudent.com.
Become a member today and join an exclusive academic network to connect with other students and faculty.

Sign Up Now
image

  Still can't find what you're looking for?

Post your question to the forum

Sign Up Now
image

PhDStudent