What is the formula for calculating inter-rater reliability?

There isn’t just one formula for calculating inter-rater reliability. The right one depends on your data type (e.g., nominal data, ordinal data) and the number of raters.

Cohen’s kappa (κ) is commonly used for two raters
Fleiss’ kappa is typically used for three or more raters
The Intraclass Correlation Coefficient (ICC) is used for continuous data (interval or ratio). This is based on analysis of variance (ANOVA)

The most used formula (for Cohen’s kappa) is:
$\kappa = \dfrac{{{P}_o}-{{P}_e}}{{1}-{P_e}}$
Po is the observed proportion of agreement, and Pe stands for the expected agreement by chance.

Research: Other interesting questions

What is the first step in the research process?: Before you can conduct a research project, you must first decide what topic you want to focus on. In the first step of the research process, identify a topic that interests you. The topic can be broad at this stage and will be narrowed down later.

Do some background reading on the topic to identify potential avenues for further research, such as gaps and points of debate, and to lay a more solid foundation of knowledge. You will narrow the topic to a specific focal point in step 2 of the research process.
How many steps make up the research process?: The research process comprises five steps.

Choosing a topic

Identifying a problem

Formulating research questions

Creating a research design

Writing a research proposal

Once you’ve written your proposal, you may need your advisor’s approval of your plan before you can dive into the research process.
What is a research design?: The research design is the backbone of your research project. It includes research objectives, the types of sources you will consult (i.e., primary vs secondary), data collection methods, and data analysis techniques.

A thorough and well-executed research design can facilitate your research and act as a guide throughout both the research process and the thesis or dissertation writing process.
What are examples of action research papers?: Examples of action research papers are:

“Participatory Action Research for Conservation and Development: Experiences from the Amazon” by Perz et al. (2022)

“Exploring Social Innovation through Co-creation in Rural India Using Action Research” by Cornet and Barpanda (2021)
What is a list of topics for action research in education?: Topics for action research in education are:

Developing a standards-based grading system to improve students’ understanding of assessment criteria

Designing a technology-enhanced curriculum to improve student learning outcomes and engagements

Developing a co-teaching model to improve student outcomes for students with special needs

If you’re looking for even more topic ideas, ask QuillBot’s AI Writer to generate a list of action research topics for you.
Why is systematic random sampling sometimes used in place of simple random sampling?: Systematic sampling is sometimes used in place of simple random sampling because it’s easier to implement.

With systematic sampling, you only draw one random number and then select subjects at regular intervals. This is especially helpful when the population is large.
What are the pros and cons of simple random sampling?: Simple random sampling is one of the most commonly used probability sampling methods.

The most important pros of simple random sampling are:

Ease of implementation. This method is relatively easy to implement. You don’t have to think about strata (like with stratified sampling) or clusters (like with cluster sampling).

Representative sample. Simple random sampling provides a representative sample of the population, with each unit having an equal chance of being selected.

Lack of bias. Because of the random nature of this technique, the risk of research biases is minimized. Researchers can’t influence the selection process.

The most important cons of simple random sampling are:

Limited flexibility. This sampling method is a fixed-probability sampling method, which means it can’t be adapted to changing circumstances during the sampling process.

Requirement of a large sample size. This technique typically requires large sample sizes to achieve acceptable levels of precision and accuracy, which can be expensive and time-consuming.

Difficulty of obtaining a list of entire population. It can be very difficult to obtain an exhaustive list of the entire population. This means some individuals who should be on the list have no chance of ending up in the sample.
Is simple random sampling probability or nonprobability sampling?: Simple random sampling is a common probability sampling technique.

In probability sampling, each individual in the population has the same chance of being selected for the sample. With simple random sampling, individuals are chosen from a list at random, which makes it a probability sampling method.

Other examples of probability sampling are stratified sampling, systematic sampling, and cluster sampling. Examples of nonprobability sampling are convenience sampling, quota sampling, self-selection sampling, snowball sampling, and purposive sampling.
What’s the difference between stratified and systematic sampling?: Stratified sampling and systematic sampling are both probabilistic sampling methods used to obtain representative samples from a population, but they differ significantly in their approach and execution.

Stratified sampling involves dividing the population into distinct subgroups (strata) based on specific characteristics (e.g., age, gender, income level) and then randomly sampling from each stratum. It ensures representation of all subgroups within the population.

Systematic sampling involves selecting elements from an ordered population at regular intervals, starting from a randomly chosen point. For example, you have a list of students from a school and you choose students at an interval of 5. This is a useful method when the population is homogeneous or when there is no clear stratification. It’s much easier to design and less complex than stratified sampling.
What is disproportionate stratified sampling?: Disproportionate sampling in stratified sampling is a technique where the sample sizes for each stratum are not proportional to their sizes in the overall population.

Instead, the sample size for each stratum is determined based on specific research needs, such as ensuring sufficient representation of small subgroups to draw statistical conclusions.

For example, the population you’re interested in consists of approximately 60% women, 30% men, and 10% people with a different gender identity. With disproportionate sampling, your sample would have 33% women, 33% men, and 33% people with a different gender identity. The sample’s distribution does not match the population’s.
What is proportionate stratified sampling?: Proportionate sampling in stratified sampling is a technique where the sample size from each stratum is proportional to the size of that stratum in the overall population.

This ensures that each stratum is represented in the sample in the same proportion as it is in the population, representing the population’s overall structure and diversity in the sample.

For example, the population you’re investigating consists of approximately 60% women, 30% men, and 10% people with a different gender identity. With proportionate sampling, your sample would have a similar distribution instead of equal parts.
Is stratified sampling random?: Yes, stratified sampling is a random sampling method (also known as a probability sampling method). Within each stratum, a random sample is drawn, which ensures that each member of a stratum has an equal chance of being selected.
What are the different types of cluster sampling?: In all three types of cluster sampling, you start by dividing the population into clusters before drawing a random sample of clusters for your research. The next steps depend on the type of cluster sampling:

Single-stage cluster sampling: you collect data from every unit in the clusters in your sample.

Double-stage cluster sampling: you draw a random sample of units from within the clusters and then you collect data from that sample.

Multi-stage cluster sampling: you repeat the process of drawing random samples from within the clusters until you’ve reached a small enough sample to collect data from.
What are the advantages of cluster sampling?: Cluster sampling is generally more inexpensive and efficient than other sampling methods. It is also one of the probability sampling methods (or random sampling methods), which contributes to high external validity.
What are the disadvantages of cluster sampling?: Cluster sampling usually harms internal validity, especially if you use multiple clustering stages. The results are also more likely to be biased and invalid, especially if the clusters don’t accurately represent the population. Lastly, cluster sampling is often much more complex than other sampling methods.
What are advantages of using a within-participant design in experimental research?: A within-participant design, also known as a repeated-measures design, is a type of experimental design where the same participants are assigned to multiple groups or conditions. Some advantages of this design are:

Increased statistical power: By using the same participants across multiple conditions, you can reduce the number of participants needed to detect a significant effect, which can lead to increased statistical power.

Reduced between-participants variability: Since each participant is tested multiple times, the variability between participants is reduced, which can result in more accurate and reliable estimates of the effect.

Better control over extraneous variables: By using the same participants across multiple conditions, you can better control for extraneous variables that might affect the outcome, as these variables are likely to be constant across conditions.

Increased precision: Within-participant designs can provide more precise estimates of the effect size, as the same participants are used across all conditions.

Reduced sample size: Depending on the research question and design, a within-participant design can require fewer participants than a between-participants design, which can reduce costs and increase efficiency.

It’s important to note that within-participant designs also have some limitations, such as increased risk of order effects (where the order of conditions affects the outcome) and carryover effects (where the effects of one condition persist into another condition).
What are the two groups in an experimental design?: In experimental design, the two main groups are:

Treatment group: This group is exposed to the manipulated independent variable, and the researcher measures the effect of the treatment on the dependent variable.

Control group: This group is not exposed to the manipulated independent variable (the variable being changed or tested). The control group serves as a reference point to compare the results of the experimental group to.

In other words, the control group is used as a baseline to compare with the treatment group, which receives the experimental treatment or intervention.

Two groups in experimental design example
You want to test a new medication to treat headaches. You randomly assign your participants to one of two groups:

The treatment group, who receives the new medication

The control group, who receives a placebo
Why is randomization important in an experimental design?: Randomization is a crucial component of experimental design, and it’s important for several reasons:

Prevents bias: Randomization ensures that each participant has an equal chance of being assigned to any condition, minimizing the potential for bias in the assignment process.

Controls for confounding variables: Randomization helps to distribute confounding variables evenly across conditions, reducing the risk of spurious correlations between the independent variable and the outcome.

Increases internal validity: By randomly assigning participants to conditions, you can increase the confidence that any observed differences between conditions are due to the independent variable and not some other factor.
Are data at the nominal level of measurement quantitative or qualitative?: Data at the nominal level of measurement is qualitative.

Nominal data is used to identify or classify individuals, objects, or phenomena into distinct categories or groups, but it does not have any inherent numerical value or order.

You can use numerical labels to replace textual labels (e.g., 1 = male, 2 = female, 3 = nonbinary), but these numerical labels are random and are not meaningful. You could rank the labels in any order (e.g., 1 = female, 2 = nonbinary, 3 = male). This means you can’t use these numerical labels for calculations.
What are the 4 principles of experimental design?: The four principles of experimental design are:

Randomization: This principle involves randomly assigning participants to experimental conditions, ensuring that each participant has an equal chance of being assigned to any condition. Randomization helps to eliminate bias and ensures that the sample is representative of the population.

Manipulation: This principle involves deliberately manipulating the independent variable to create different conditions or levels. Manipulation allows researchers to test the effect of the independent variable on the dependent variable.

Control: This principle involves controlling for extraneous or confounding variables that could influence the outcome of the experiment. Control is achieved by holding constant all variables except for the independent variable(s) of interest.

Replication: This principle involves having built-in replications in your experimental design so that outcomes can be compared. A sufficient number of participants should take part in the experiment to make sure that randomization allows for groups with a similar distribution. This increases the chance of detecting true differences.
What is an experimental design diagram?: An experimental design diagram is a visual representation of the research design, showing the relationships among the variables, conditions, and participants. It helps researchers to:

Clarify the research question and hypotheses

Identify the independent, dependent, and control variables

Determine the experimental conditions and treatment levels

Plan the sampling and data collection procedures

Visualize the flow of participants through the study
What is a pre-experimental design?: A pre-experimental design is a simple research process that happens before the actual experimental design takes place. The goal is to obtain preliminary results to gauge whether the financial and time investment of a true experiment will be worth it.

Pre-experimental design example
A researcher wants to investigate the effect of a new type of meditation on stress levels in college students. They decide to conduct a small pre-experiment with 10 college students who are already interested in meditation.

The students are asked to participate in a 30-minute meditation session once a week for 4 weeks. The students’ stress levels are measured before and after the meditation sessions with a standardized questionnaire.

The researcher compares the outcomes and notices significant differences in stress scores. They decide to move forward with a more costly and time-consuming experiment where they take into account all criteria for an experimental design (e.g., random assignment of participants, control group, controlling for extraneous variables).
Can you use nominal data in an ANOVA test?: You can’t use an ANOVA test if the nominal data is your dependent variable. The dependent variable needs to be continuous (interval or ratio data).

The independent variable for an ANOVA should be categorical (either nominal or ordinal data).
Does nominal data involve the use of variables that have been rank ordered?: No, nominal data can only be assigned to categories that have no inherent order to them.

Categorical data with categories that can be ordered in a meaningful way is called ordinal data.
What type of information does data at the nominal level describe?: Data at the nominal level of measurement typically describes categorical or qualitative descriptive information, such as gender, religion, or ethnicity.

Contrary to ordinal data, nominal data doesn’t have an inherent order to it, so you can’t rank the categories in a meaningful order.
What is the difference between nominal and ordinal data?: Nominal data and ordinal data are similar because they can both be grouped into categories. However, ordinal data can be ranked in a logical order (e.g., low, medium high), whereas nominal data can’t (e.g., male, female, nonbinary).
Is ordinal data qualitative or quantitative?: Ordinal data is usually considered qualitative in nature. The data can be numerical, but the differences between categories are not equal or meaningful. This means you can’t use them to calculate measures of central tendency (e.g., mean) or variability (e.g., standard deviation).
What is the difference between ordinal and interval data?: Ordinal data and interval data are similar because they can both be ranked in a logical order. However, for interval data, the differences between adjacent scores are equal.
Is age ordinal data?: The variable age can be measured at the ordinal or ratio level.

If you ask participants to provide you with their exact age (e.g., 28), the data is ratio level.

If you ask participants to select the bracket that contains their age (e.g., 26–35), the data is ordinal.

Ordinal data and ratio data are similar because they can both be ranked in a logical order. However, for ratio data, the differences between adjacent scores are equal and there’s a true, meaningful zero.
What are properties of ordinal data?: Ordinal is the second level of measurement. It has two main properties:

Ordinal data can be grouped into categories

Ordinal data can be ranked in a logical order (e.g., low, medium, high)
What is the difference between content validity and face validity?: Content validity and face validity are both types of measurement validity.

Content validity refers to the degree to which the items or questions on a measure accurately reflect all elements of the construct or concept that’s being measured. It assesses whether the items are accurate, relevant, and comprehensive in measuring the construct.

Face validity refers to the degree to which a measure seems to be measuring what it claims to measure. It assesses whether the measure appears to be relevant.
What is the difference between construct validity and face validity?: Face validity refers to the extent to which a research instrument appears to measure what it’s supposed to measure. For example, a questionnaire created to measure customer loyalty has high face validity if the questions are strongly and clearly related to customer loyalty.

Construct validity refers to the extent to which a tool or instrument actually measures a construct, rather than just its surface-level appearance.
What is the best way for a researcher to judge the face validity of items on a measure?: The best way for a researcher to judge the face validity of items on a measure is by asking both other experts and test participants to evaluate the instrument.

The combination of experts with background knowledge and research experience, along with test participants who form the target audience of the instrument, provides a good idea of the instrument’s face validity.
In which ways are content and face validity similar?: Content validity and face validity are both types of measurement validity. Both aim to ensure that the instrument is measuring what it’s supposed to measure.

However, content validity focuses on how well the instrument covers the entire construct, whereas face validity focuses on the overall superficial appearance of the instrument.
What is the difference between construct validity and internal validity?: Construct validity refers to the extent to which a study measures the underlying concept or construct that it is supposed to measure.

Internal validity refers to the extent to which observed changes in the dependent variable are caused by the manipulation of the independent variable rather than other factors, such as extraneous variables or research biases.

Construct validity vs. internal validity example
You’re studying the effect of exercise on happiness levels.

Construct validity would ask whether your measures of exercise and happiness levels accurately reflect the underlying concepts of physical activity and emotional state.

Internal validity would ask whether your study’s results are due to the exercise itself, or if some other factor (e.g., changes in diet or stress levels) might be causing changes in happiness levels.
What are the 12 threats to internal validity?: The 12 main threats to internal validity are:

History: Changes in the environment or events that occur outside of the study can affect the outcome.

Maturation: Changes in the participants over time (e.g., age, skill level) can affect the outcome.

Testing: The act of testing or measurement itself can affect the outcome (testing effect, practice effect, or carryover effect).

Instrumentation: Changes in the measuring instrument or tool used to collect data can affect the outcome.

Statistical regression to the mean: The tendency of extreme scores to regress towards the mean, which can lead to a loss of statistical significance.

Selection: The selection of participants for the study can affect the outcome (selection bias), especially in the case of non-probability sampling.

Experimental mortality or attrition bias: The loss of participants or dropouts during the study can affect the outcome.

Multiple-treatment interference: The interaction between different treatments or conditions can affect the outcome.

Social desirability bias: The participants’ awareness of being in a study and their desire to be well-liked by researchers can affect the outcome.

Social interaction: The participants’ awareness of being treated differently than people in other groups can affect the outcome.

Residual confounding: The presence of unmeasured or uncontrolled extraneous or confounding variables that affect the outcome and are not accounted for in the analysis.

Order effect: The order of the independent variable levels affects the dependent variable.

There are several ways to counter these threats to internal validity, for example, through randomization, the addition of control groups, and blinding.
Is systematic sampling random?: Systematic sampling is a random sampling method. Another name for random sampling is probability sampling.

In systematic sampling, the researcher chooses a random starting point in a list of the population (e.g., by using a random number generator) before selecting subjects for the sample at a regular sampling interval (n). The random starting point and regular interval ensure the random nature of this sampling method.
When is it inappropriate to use systematic random sampling?: It is inappropriate to use systematic random sampling when your population has a periodic or cyclic order. This could result in only including individuals with a specific characteristic (e.g., age) in your sample.

Systematic sampling example: Unrepresentative sample
Your list of employees alternates between men, women, and nonbinary people. You select every third individual, which means you’re only selecting nonbinary people. This wouldn’t be a representative sample because the sample doesn’t contain any people who identify as men or women, whereas they make up most of the population.
Why might a researcher choose purposive sampling over systematic sampling?: Purposive sampling is often chosen over systematic sampling in situations where the researcher wants to select subjects that have specific traits that are needed in their sample.

Systematic sampling is a probability sampling method where the researcher systematically selects every nth member of the population with a random starting point. The researcher is unable to influence the characteristics of the people that end up in the sample.

Purposive sampling is a non-probability sampling method where the researcher selects specific subjects with traits that can provide the best information to achieve the research aims.
Is systematic sampling biased?: Systematic sampling is a probability sampling method, which typically ensures a lower risk of bias than nonprobability sampling methods.

However, systematic sampling can be vulnerable to sampling bias, especially if the starting point isn’t truly random. The choice of sampling interval can also introduce bias:

If the interval is too small, the sample can lack representativeness of the population.

If the interval is too large, the sample might not capture all the variation that exists in the population.
What is the formula for systematic sampling?: You can use a formula to calculate the sampling interval in systematic sampling, which is a probability sampling method where the researcher systematically selects subjects for their sample at a regular interval.

You can calculate the sampling interval (n) by dividing the total population by the desired sample size.

Formula: n = population size/sample size

Example: I = 2,000/200 = 10

In some cases, people might use a different letter to indicate the sampling interval (e.g., k). This is irrelevant to the use of the formula.
What are some major mistakes to avoid when writing a research proposal?: These are three major mistakes to avoid when writing a research proposal:

Failing to connect your potential research to previous studies, from the research question to the contribution your research will make.

Failing to maintain a clear and cohesive focus on the research topic throughout your research questions, aims, objectives, and methods.

Failing to determine realistic research steps and explain them clearly enough.

You also should tailor your research proposal to its audience. If the people approving your study do not have much technical knowledge, it may be helpful to run your proposal through a humanizer to reduce jargon.
What are the 3 chapters of a research proposal?: A research proposal has three main parts: the introduction, the literature review, and the methods section.

For help structuring and refining your research proposal, try QuillBot’s free AI project proposal generator.
What are the parts of a research paper?: Most research papers contain at least an introduction and sections for methodology, results, discussion, and references. Many also include an abstract and a literature review. Some other common elements are a title page, a table of contents, tables and figures, and appendices.

A title is an important part of a research paper that can sometimes get lost in the shuffle. QuillBot’s free title generator can help you come up with a compelling title quickly.
How do I cite sources for a research paper?: As you research, write down citation information for any sources you plan to use. Record quotes and ideas carefully, along with the page numbers where you found them. You can write them on note cards, on paper, or in a digital document.

When writing your first draft, include enough citation information in the text to ensure accurate referencing. After finishing the draft, you can go through your paper and add the full citations, following the style guide.

QuillBot’s Citation Generator can help you automatically generate in-text citations and a reference list for your paper.

Finally, use QuillBot’s Plagiarism Checker to double-check your work and avoid plagiarism.
Why is it important for a study of eyewitness memory to have a high level of ecological validity?: When a study has high ecological validity, the findings are more likely to generalize to real-world situations, making them more applicable and useful for practical purposes, such as improving witness testimony and investigative procedures.

High ecological validity minimizes the influence of factors that can affect results, such as laboratory settings or overly structured procedures, which can lead to biases or unrepresentative data.

Ecological validity is a subtype of external validity.
What is the difference between ecological validity and external validity?: Ecological validity is a subtype of external validity that is specifically concerned with the extent to which the study environment, tasks, and conditions reflect the real-world settings in which the behavior naturally occurs.

External validity also consists of population validity, which refers to the extent to which the results of a study can be generalized to the larger population from which the sample was drawn.
What is the difference between content validity and predictive validity?: Content validity and predictive validity are two types of validity in research:

Content validity ensures that an instrument accurately measures all elements of the construct it intends to measure.

A test designed to measure anxiety has high content validity if its questions cover all relevant aspects of the construct “anxiety.”

Predictive validity demonstrates that a measure can forecast future behavior, performance, or outcomes. It is a subtype of criterion validity.

A test designed to predict student retention has high predictive validity if it accurately predicts which students still participate in the study program 2 years later.
What is the difference between content and criterion validity?: Content validity and criterion validity are two types of validity in research:

Content validity ensures that an instrument measures all elements of the construct it intends to measure.

A survey to investigate depression has high content validity if its questions cover all relevant aspects of the construct “depression.”

Criterion validity ensures that an instrument corresponds with other “gold standard” measures of the same construct.

A shortened version of an established anxiety assessment instrument has high criterion validity if the outcomes of the new version are similar to those of the original version.
Does random assignment increase external validity?: Random assignment can increase external validity, but it has a bigger impact on internal validity.

Random assignment helps to reduce confounding variables and ensures that the treatment and control groups are comparable in all aspects except for the independent variable.

This increases the confidence that any observed differences between the groups can be attributed to the treatment rather than other factors, which means an increase in internal validity.

It can also improve external validity because random assignment of participants prevents researchers from inadvertently selecting participants who may be more or less likely to respond to the treatment.

However, the external validity may still be limited by sampling bias if the participants are not representative of the target population, which is why choosing the appropriate sampling method is also important to ensure external validity.

A probability sampling method, such as simple random sampling, stratified sampling, cluster sampling, or systematic sampling, is always the best choice.
What kind of sample is best for external validity?: To ensure high external validity, it’s important to draw a sample that’s representative of the population you want to generalize to. It’s always best to choose a probability sampling (also known as random sampling) method for this.

The most popular sampling methods are stratified sampling, systematic sampling, simple random sampling, and cluster sampling.

A probability sampling method also increases other types of validity, such as internal validity, and it reduces bias.
Which type of interview has been shown to have the highest predictive validity?: The interview type with the highest predictive validity differs based on the goal of the interview.

Generally speaking, a structured interview has the highest predictive validity.

Unstructured interviews have the lowest predictive validity, especially in recruitment or job performance settings.

Semi-structured interviews have adequate predictive validity but not as high as structured interviews.

Situational questions, work sample requests, and interview questions about past behavior are the best question types in the case of job interviews.

When designing job interview questions, make sure to minimize bias and to also account for other types of validity, such as construct validity and content validity.

You can use QuillBot’s Grammar Checker to make sure your interview questions are error-free.
What is the difference between construct validity and predictive validity?: Construct validity assesses how well a test measures the concept it was meant to measure, whereas predictive validity evaluates to what degree a test can predict a future outcome or behavior.
What are the two types of criterion validity?: Criterion validity measures how well a test corresponds to another measure, or criterion. The two types of criterion validity are concurrent and predictive validity.

Concurrent validity compares two measures obtained at the same time.

Predictive validity indicates how well a test correlates with a measurement taken later on.
What is the difference between convergent and concurrent validity?: Convergent and concurrent validity both indicate how well a test score and another variable compare to one another.

However, convergent validity indicates how well one measure corresponds to other measures of the same or similar constructs. These measures do not have to be obtained at the same time.

Concurrent validity instead assesses how well a measure aligns with a benchmark or “gold-standard,” which can be a ground truth or another validated measure. Both measurements should be taken at the same time.
What is a construct?: A construct is a phenomenon that cannot be directly measured, such as intelligence, anxiety, or happiness. Researchers must instead approximate constructs using related, measurable variables.

The process of defining how a construct will be measured is called operationalization. Constructs are common in psychology and other social sciences.

To evaluate how well a construct measures what it’s supposed to, researchers determine construct validity. Face validity, content validity, criterion validity, convergent validity, and discriminant validity all provide evidence of construct validity.
What is the difference between concurrent and predictive validity?: Concurrent validity and predictive validity are both types of criterion validity. Both assess how well one test corresponds to another, theoretically related, test or outcome. However, the key difference is when each test is conducted:

Concurrent validity compares one measure to a second, well-established measure that acts as a gold-standard. Both measures should be obtained at the same time, or concurrently.

Predictive validity instead captures how well a measure corresponds to a measure taken later in time (i.e., how well one measure predicts a future measure).
How do you measure construct validity?: Construct validity assesses how well a test reflects the phenomenon it’s supposed to measure. Construct validity cannot be directly measured; instead, you must gather evidence in favor of it.

This evidence comes in the form of other types of validity, including face validity, content validity, criterion validity, convergent validity, and divergent validity. The stronger the evidence across these measures, the more confident you can be that you are measuring what you intended to.
What is the difference between construct and criterion validity?: Construct validity evaluates how well a test reflects the concept it’s designed to measure.

Criterion validity captures how well a test correlates with another “gold standard” measure or outcome of the same construct.

Although both construct validity and criterion validity reflect the validity of a measure, they are not the same. Construct validity is generally considered the overarching concern of measurement validity; criterion validity can therefore be considered a form of evidence for construct validity.
Why are convergent and discriminant validity often evaluated together?: Convergent validity and discriminant validity (or divergent validity) are both forms of construct validity. They are both used to determine whether a test is measuring the thing it’s supposed to.

However, each form of validity tells you something slightly different about a test:

Convergent validity indicates whether the results of a test correspond to other measures of a similar construct. In theory, there should be a high correlation between two tests that measure the same thing.

Discriminant validity instead measures whether a test is similar to measures of a different construct. There should be a low correlation between two tests that measure different things.

If a test is measuring what it is supposed to, it should correspond to other tests that measure the same thing while differing from tests that measure other things. To assess these two qualities, you must determine both convergent and discriminant validity.
What is the difference between concurrent validity and convergent validity?: Convergent validity and concurrent validity both indicate how well a test score and another variable compare to one another.

Convergent validity indicates how well one measure corresponds to other measures of the same or similar constructs. These measures do not have to be obtained at the same time.

Concurrent validity instead assesses how well a measure aligns with a benchmark or “gold-standard,” which can be a ground truth or another validated measure. Both measurements should be taken at the same time.
Are discriminant and divergent validity the same thing?: In short, yes! The terms discriminant validity and divergent validity are often used synonymously to refer to whether a test yields different results than other tests that measure unrelated concepts. However, “discriminant validity” is the more commonly used and accepted term.
Why is validity so important in psychology research?: Psychology and other social sciences often involve the study of constructs—phenomena that cannot be directly measured—such as happiness or stress.

Because we cannot directly measure a construct, we must instead operationalize it, or define how we will approximate it using observable variables. These variables could include behaviors, survey responses, or physiological measures.

Validity is the extent to which a test or instrument actually captures the construct it’s been designed to measure. Researchers must demonstrate that their operationalization properly captures a construct by providing evidence of multiple types of validity, such as face validity, content validity, criterion validity, convergent validity, and discriminant validity.

When you find evidence of different types of validity for an instrument, you’re proving its construct validity—you can be fairly confident it’s measuring the thing it’s supposed to.

In short, validity helps researchers ensure that they’re measuring what they intended to, which is especially important when studying constructs that cannot be directly measured and instead must be operationally defined.
What is the difference between test validity and experimental validity?: Test validity refers to whether a test or measure actually measures the thing it’s supposed to. Construct validity is considered the overarching concern of test validity; other types of validity provide evidence of construct validity and thus the overall test validity of a measure.

Experimental validity concerns whether a true cause-and-effect relationship exists in an experimental design (internal validity) and how well findings generalize to the real world (external validity and ecological validity).

Verifying that an experiment has both test and experimental validity is imperative to ensuring meaningful and generalizable results.
What is an experiment?: An experiment is a study that attempts to establish a cause-and-effect relationship between an independent and dependent variable.

In experimental design, the researcher first forms a hypothesis. They then test this hypothesis by manipulating an independent variable while controlling for potential confounds that could influence results. Changes in the dependent variable are recorded, and data are analyzed to determine if the results support the hypothesis.

Nonexperimental research does not involve the manipulation of an independent variable. Nonexperimental studies therefore cannot establish a cause-and-effect relationship. Nonexperimental studies include correlational designs and observational research.
What is an outcome variable?: An outcome variable, or outcome measure, is another term for a dependent variable.

Dependent variables are the outcome or response that is measured in a study. Independent variables are manipulated by the researcher, and changes in the dependent variable are recorded and analyzed. An experiment explores cause-and-effect relationships between dependent and independent variables.
What is a dependent variable synonym?: You may encounter different terms for independent and dependent variables in different contexts. Some common synonyms for dependent variables are as follows:

Dependent measure

Outcome

Response variable

Predicted variable

Output variable

Measured variable
What is an independent variable synonym?: Independent and dependent variables are called by various names across different contexts and fields. Some common synonyms for independent variables include the following:

Predictor variable

Regressor

Covariate

Manipulated variable

Explanatory variable

Exposure variable

Feature

Input variable
What is the difference between a conceptual framework, a theoretical framework, and a literature review?: The literature review, conceptual framework, and theoretical framework are all important steps in defining a research project.

A literature review is conducted early in the research process. Its purpose is to describe the current state of a research area, identify gaps, and emphasize the relevance of your own research question or study.

A theoretical framework is the lens through which a research question is viewed and answered. Different fields have their own assumptions, methods, and interpretations related to the same phenomenon that influence the choice of a theoretical framework.

Consider a neuroscientist and a social psychologist studying the construct “love.” They will each take a different approach, applying specialized methods and interpretations. In other words, they each use a unique theoretical framework that is guided by the existing theories of their field.

A conceptual framework describes the variables relevant to a study and how they relate to one another. This may include dependent and independent variables as well as any confounding variables that could influence results.
What is the difference between dependent variables, independent variables, control variables, and confounding variables?: A variable is something that can take on different values. A study contains independent and dependent variables, control variables, and confounding variables that influence its results.

Dependent variables represent the outcome of a study. Researchers measure how they change under different values of the independent variable(s).

Independent variables are manipulated by the researcher to observe their effect on dependent variables.

Control variables are variables that are held constant to isolate the effect of the independent variable.

Confounding variables are variables that have not been controlled for that may influence a study’s results.

The expected relationship between these variables can be illustrated using a conceptual framework.
What is the difference between mediator and moderator variables?: A mediator (or mediating variable) is a variable that falls between a dependent and independent variable; that is, it connects them.

For example, the dependent variable “academic performance” is influenced by the independent variable “exercise” via the mediator variable “stress.” Exercise reduces stress, which in turn improves academic performance. Stress therefore mediates the relationship.

A moderator (or moderating variable) influences how an independent variable influences a dependent variable; in other words, it impacts their relationship.

For example, the relationship between the dependent variable “mental health” and the independent variable “social media use” may be influenced by the moderator “age.” The impact that social media has on mental health depends on someone’s age.

The expected influence of mediator and moderator variables can be captured in a conceptual framework.
Why are research ethics important?: Research ethics are principles that guide scientists, helping them distinguish right from wrong when conducting research. Research ethics help protect the people involved in scientific studies and ensure the integrity of scientific research.
What are the APA’s 5 principles of research ethics?: The American Psychological Association (APA) has five principles to guide psychologists in conducting ethical research and scientific work.

Beneficence and nonmaleficence: protect the welfare of research participants and do no harm.

Fidelity and responsibility: serve the best interests of society and the specific communities impacted by research and scientific work.

Integrity: conduct and teach psychology in an accurate and honest manner.

Justice: ensure that all people have equal access to the benefits of psychology services and research.

Respect for people’s rights and dignity: show consideration for people’s dignity and their right to privacy, confidentiality, and autonomy.
What are NIH’s 7 principles of ethics in research?: The National Institutes of Health (NIH) has defined seven principles to protect clinical research participants and promote research ethics:

Social and clinical value: the scientific advances of a research study should justify the costs or risks of conducting this research.

Scientific validity: a study should be designed to address an answerable question using feasible and accepted research methods.

Fair subject selection: participants should be selected based on the scientific aims of the study and should not be included or excluded for reasons unrelated to research goals.

Favorable risk-benefit ratio: the potential risks to participants should be minimized and should be outweighed by potential benefits.

Independent review: an independent review panel should ensure a study is ethical before research begins.

Informed consent: participants should decide whether to voluntarily participate in a study after learning about its research question, methods, potential risks, and benefits.

Respect for potential and enrolled subjects: individuals should be treated with respect throughout the research process.
What is an institutional review board (IRB)?: An institutional review board (IRB) is a committee that reviews proposed studies involving human participants to ensure research ethics are being followed. In most countries, a study must be approved by an IRB before data can be collected.

An IRB is sometimes called a research ethics board (REB), an ethical review board (ERB), or an independent ethics committee (IEC).
What’s the difference between anonymity and confidentiality?: Anonymity and confidentiality are both important aspects of research ethics.

Anonymity means that researchers do not collect personal information that can be used to identify a participant or that someone’s responses cannot be linked to their identity.

Confidentiality means that only the researchers conducting a study can link study responses or data to individual participants.

If you run a study and do not know who your participants are (i.e., you collect no identifying information), your data are anonymous. If you know who your participants are but no one else does (i.e., you collect identifying information but don’t publish it), your data are confidential.
What is triangulation in qualitative research?: Triangulation involves using a combination of data or techniques to answer a research question. Triangulation can help you confirm the validity of your findings. This can be helpful in qualitative research, which is often subjective and vulnerable to bias.

Types of triangulation include the following:

Data triangulation: uses data from different sources

Investigator triangulation: has multiple people analyze data

Theory triangulation: uses different theoretical frameworks to analyze data

Methodological triangulation: uses different approaches to study the same phenomenon
What is grounded theory?: Grounded theory is a systematic approach that can be applied in qualitative research. Its goal is to create new theories that are grounded in data.

With a grounded theory approach, data collection and analysis occur at the same time (this is called theoretical sampling). This approach can be helpful when you are conducting research in a new area and do not have a hypothesis related to study outcomes.
How do I analyze qualitative data?: Qualitative data are generally narrative in nature. They may include interview transcripts or experimenter observations. Different approaches exist to analyze qualitative data, but common steps are as follows:

Organize the data

Code the data, sorting it into relevant categories

Extract and interpret key ideas

Common qualitative data analysis techniques include content analysis, thematic analysis, and discourse analysis.
What is member checking in qualitative research?: Member checking is when participants are allowed to review their data or results to confirm accuracy. This process can happen during or after data collection.

In qualitative research, data are often collected through interviews or observations. Allowing a participant to review their data can help build trust and ensure that their thoughts and experiences are being accurately expressed.
What are the types of qualitative research?: There are many types of qualitative research. The following are five common approaches:

Ethnography: the researcher immerses themself in a group to understand its culture

Grounded theory: data are collected to create new theories that may guide additional data collection

Phenomenology: the experiences of individuals are used to study a phenomenon

Narrative research: the way stories are told is studied to understand human experiences

Action research: a problem is studied and solutions are generated simultaneously

Choosing the right approach depends on the research question you are studying.
How do I decide between qualitative and quantitative research?: Qualitative and quantitative research methods are used to investigate different types of research questions.

Quantitative methods are best if:

The topic you’re studying can easily be expressed using numbers

You want to use statistics to test a hypothesis

Qualitative methods are best if:

The topic you’re studying cannot be expressed numerically

You’re studying a new research area with limited existing theories

You want to gain deep insight into a nuanced issue
What is operationalization?: Operationalization is when you define how a variable will be measured. Operationalization is especially important in fields like psychology that involve the study of more abstract ideas (e.g., “fear”).

Because fear is a construct that cannot be directly measured, a researcher must define how they will represent it. For example, in studies involving mice, fear is often operationalized as “”how often a mouse freezes (i.e., stops moving) during an experiment.”

Operationalization can be used to turn an abstract concept into a numerical form for use in quantitative research.

Some operationalizations are better than others. It is important to consider both reliability and validity (how consistent and accurate a measurement is, respectively) when operationalizing a construct.
What is data collection?: Data collection is the process of gathering data (measurements, observations, and other information) to answer a research question. Though many different methods of data collection exist, all are systemic and follow a procedure defined before data collection begins. Data can be qualitative or quantitative.
What are some mixed methods research questions?: Mixed methods research questions combine qualitative methods and quantitative methods to answer a research question. Examples of mixed methods research questions include the following:

How does the proximity of student housing to college campuses (quantitative) relate to student perceptions of school culture (qualitative)?

To what extent do air pollution levels (quantitative) correspond to residents’ perception of air quality (qualitative) in urban areas?

What barriers do community members describe to accessing mental health services (qualitative), and how do these barriers correlate with use of these services within this community (quantitative)?
What are the types of mixed methods research designs?: The four main types of mixed methods research designs differ in when the quantitative and qualitative data are collected and analyzed:

Convergent parallel design: Qualitative and quantitative data are collected and analyzed simultaneously and analyzed separately. The results are then compared, and similarities and dissimilarities between the findings are discussed.

Embedded design: Either qualitative methods or quantitative methods are the focus of a study, and additional methods are used to support these main findings.

Explanatory sequential design: Quantitative data are first collected and analyzed, then the results or patterns are explained using qualitative methods.

Exploratory sequential design: Qualitative data are first collected and analyzed, then emergent themes are explored in more detail using quantitative methods.
When should I use quasi-experimental design?: Practical or ethical concerns may prevent researchers from using a true experimental design:

Practical concerns that prevent researchers from conducting a true experiment may include the cost of a study or the time required to design the experiment and collect and analyze data.

Ethical concerns may also limit the feasibility of true experimental research. It would be unethical to intentionally prevent study participants from accessing medication or other treatments that the researcher knows would benefit them.

In these cases, a quasi-experimental design may be more appropriate.
What is the difference between a true experiment and a quasi-experiment?: In a true experiment, participants are randomly assigned to different study conditions. A quasi-experiment lacks this random assignment.

True experiments are also usually conducted in controlled laboratory settings, which facilitates control of confounding variables that may impact study results. Quasi-experimental designs often collect data in real-world settings, which increases external validity but reduces control of confounds.

Finally, both true experiments and quasi-experiments generally involve the manipulation of an independent variable to determine its causal effect on a dependent variable. However, in a quasi-experimental study, researchers may have less control over this manipulation (e.g., they may be studying the impact of an intervention or treatment that has already happened).
What is pretest-posttest design?: A pretest-posttest design is a quasi-experimental research design. Two data points are collected for a participant: one from before an intervention is introduced and one from after an intervention. A difference in these scores may indicate that the intervention was effective.

For example, imagine you complete a depression inventory before and after a 6-week therapy program. An improvement in your score may indicate that the program worked.
What is regression discontinuity design?: Regression discontinuity design is a quasi-experimental approach that compares two groups of participants that are separated based on an arbitrary threshold. This method assumes that people immediately above and immediately below this threshold are quite similar. Any subsequent differences between these groups can therefore be attributed to interventions that one group does or does not receive.

For example, imagine you’re testing the efficacy of a cholesterol medication. You administer this medication only to patients whose cholesterol levels exceed 200 mg/dl. You then compare heart health indicators of patients with cholesterol levels slightly over 200 mg/dl, who do receive the medication, to patients with cholesterol levels slightly below 200 mg/dl, who do not receive the medication. If the heart health of the former group improves relative to the latter group, you may conclude that the treatment worked.
What is an interrupted time series design?: An interrupted time series design is a quasi-experimental research method. It is similar to a pretest-posttest design, but multiple data points, called a time series, are collected for a participant before and after an intervention is administered. The intervention “interrupts” the time series of observations.

If scores taken after the intervention are consistently different from scores taken before the intervention, a researcher can conclude that the intervention was successful. Considering multiple measurements helps reduce the impact of external factors
What are the types of case studies?: Though case studies can be classified in many ways, the most common types are intrinsic, instrumental, and collective case studies.

Intrinsic case studies focus on a specific subject (i.e., case). The point of such studies is to learn more about this specific subject rather than to generalize findings.

Instrumental case studies also focus on a single subject, but the intent is to generalize findings to a broader population.

Collective case studies have the same purpose as instrumental case studies—to use findings to increase one’s understanding of a broader topic—but they include multiple cases.
What is a case study example in psychology?: Case studies have historically been used in psychology to understand rare conditions. For example, Phineas Gage was a railroad worker who had an iron rod driven through his head in an accident and miraculously survived. However, this accident drastically altered his personality and behavior for the remaining 12 years of his life.

Detailed studies of Phineas Gage helped scientists realize that different areas of the brain have specific functions. This famous case study is an example of how studying one individual in detail can provide insights that drive the formation of broader theories.
In research, what is the difference between methods vs methodology?: Research methods are the steps you follow when conducting research. A methods section should describe the type of research you’re conducting, sampling techniques, data collection methods, and data analysis.

Research methodology instead focuses on the theory behind your research methods and why you chose them to address your research question.

Though people sometimes use the terms method and methodology interchangeably, they are not the same. Methods describe how you conduct your research, and methodology describes why you chose these methods.
What are common research methods in psychology?: Most research methods in psychology are quantitative: numerical data are used to address a research question. Quantitative approaches include the following:

Experimental research

Quasi-experimental research

Survey research

Some psychology research is qualitative (i.e., uses nonnumerical data to explore a research question). Qualitative research approaches include the following:

Interviews and focus groups

Case studies

Survey research

Observational research

Mixed-methods research combines qualitative and quantitative approaches.
What are descriptive research methods?: Descriptive research is a research method that aims to uncover the characteristics of a population or phenomenon. Research questions can be addressed using techniques like surveys and observation.

Examples of descriptive research questions include the following:

What percentage of people in a community experience food insecurity?

What is the most popular social media platform for people under the age of 25?

How many cars does the average American family own?

Descriptive research can answer what, where, when, and how questions but not why questions. Both quantitative and qualitative methods can be used for descriptive research.
What’s the difference between a survey and a questionnaire?: The terms survey and questionnaire are often used interchangeably, but they are not the same.

A survey is a research method that involves collecting a large amount of information about a topic.

A questionnaire is a tool used to collect data. It comprises a series of questions that someone answers. A questionnaire may be used to collect data when a researcher is conducting a survey.
What is question-order bias?: People are highly sensitive to question order when filling out questionnaires. Question-order bias refers to how earlier questions may influence how someone responds to later questions (this is sometimes called “priming”).

For example, when people are first asked to rate their general life satisfaction and then asked about relationship satisfaction, they respond more positively than when these questions are reversed.
Is a Likert scale ordinal?: Yes—the Likert scale, which is often included in questionnaires, is an example of an ordinal measurement.

Ordinal data have a clear order (items can be logically sorted), but the distance between items is not uniform nor quantifiable.

A Likert scale asks someone to rate how much they agree with a series of statements. Response options typically include “Strongly disagree,” “Disagree,” “Neutral,” “Agree,” and “Strongly agree.”

Because these options have a clear rank (we can easily and objectively order them) but unclear spacing (i.e., the distance between “Disagree” and “Neutral” isn’t necessarily the same as the distance between “Neutral” and “Agree”), a Likert scale is considered an ordinal measurement.

Note: For analysis, Likert scale data are sometimes converted to numbers and treated as integer data. This process allows the research to use certain analysis techniques. It’s always important to carefully consider and provide justification for any analyses you decide to conduct.
What is a rating scale?: A rating scale is a type of multiple-choice question often included in a questionnaire. Rating scales include ordered response options that range from low to high (unipolar) or negative to positive (dichotomous). Rating scales are often used to assess frequency, satisfaction, or other subjective experiences.

A Likert scale is a special type of rating scale that is used to assess a person’s level of agreement with a series of statements about a topic.
What’s the difference between open-ended and close-ended questions?: Open-ended and close-ended questions can both be included in questionnaires, but they serve different purposes.

Open-ended questions have no set response: the respondent can respond in their own words. This freedom may encourage more detailed or unpredicted responses. However, responses to open-ended questions are often more time-consuming and difficult to analyze.

Close-ended questions have set response options. They can be answered quickly but limit the detail the respondent is able to provide. Types of close-ended questions include multiple-choice questions, rating scales, and Likert scales.
How do you pronounce Likert scale?: The term Likert scale is commonly mispronounced. The correct pronunciation is “Lick-urt,” not “Lie-kert.” The scale is named after its creator, Rensis Likert, and this is how you correctly say his last name.
Should I use a 5- or 7-point Likert scale?: Though traditional Likert scales include a 5-point response scale, some research has indicated that 7-point scales provide more reliable results.

As a rule of thumb, 5-point scales are better for unipolar constructs, which range from zero to positive, such as frequency. You may want to use 7-point scales for bipolar (or dichotomous) constructs that range from negative to positive, such as quality—some evidence suggests that doing so can increase reliability.
What is a Likert type scale?: A Likert-type scale resembles a Likert scale—the respondent is presented with a series of statements, and they select their response from a set of ranked options.

However, for a true Likert scale, the respondent rates their level of agreement with these statements using a 5- or 7-point scale. Likert-type scales may instead ask people to rate constructs like frequency, satisfaction, or likelihood.

An example of a true Likert scale item is as follows:

Please rate your agreement with the following statement: I am most productive in the morning.

Strongly disagree

Disagree

Neither agree nor disagree

Agree

Strongly Agree

A Likert-type scale may include items such as the following:

How frequently do you wake up before 8 a.m.?

Never

Rarely

Sometimes

Often

Always
What is the difference between a cross sectional study and a longitudinal study?: In a cross-sectional study, researchers collect data from individuals at a single point in time.

In a longitudinal study, researchers collect data from individuals repeatedly or continuously over an extended period of time (often years).

Cross-sectional studies are faster and less expensive to conduct than longitudinal studies. However, because they collect data at a single point in time, cross-sectional studies are not the best option for establishing cause-and-effect relationships.

A common practice is to conduct a cross-sectional study to generate hypotheses. You can then use this information to design a longitudinal study.
What is the difference between a case-control study and a cross-sectional study?: Case-control and cross-sectional studies differ in how participants are recruited and the types of questions they can answer.

In a case-control study, participants are recruited based on outcome status. Data are collected from two groups. The “case” group has an outcome of interest (e.g., a diabetes diagnosis), and the “control” group does not. These groups can be compared to understand what differences may have contributed to the outcome.

In a cross-sectional study, a sample of participants is recruited from a population without considering outcome status (often using random sampling). Data on outcomes and risk factors are then collected simultaneously from the sample. Cross-sectional studies are helpful for assessing the prevalence of an outcome.
What is the difference between a cross-sectional study and a cohort study?: In a cross-sectional study, researchers recruit a group of participants (often using random sampling), then measure exposure variables (e.g., risk factors—such as smoking) and outcomes (e.g., lung cancer). Cross-sectional studies are helpful for determining the prevalence of an outcome in a population.

Cohort studies instead recruit participants based on their exposure status. Cohort studies are longitudinal. They follow participants over time to observe the effect of this exposure (e.g., how many people who were exposed to asbestos go on to develop lung cancer). Cohort studies are helpful for establishing cause-and-effect relationships.
What is a repeated cross-sectional study?: In a repeated cross-sectional study, the same population is studied at multiple time points. At each time point, data are collected from a different sample of the population.

A repeated cross-sectional study is a type of longitudinal study because data are collected repeatedly over a period of time.

However, as the name suggests, it also resembles a cross-sectional study. Data are obtained from each group of participants at a single time point, and this process is repeated several times.

Repeated cross-sectional studies are helpful for studying changes in a population over time.
How long is a longitudinal study?: There is no minimum duration for longitudinal studies. Some are only a few days or weeks long, but many last for years or even decades.

The only requirement for a study to be considered longitudinal is that data are collected repeatedly from participants.
What are the types of longitudinal studies?: Longitudinal studies are broadly categorized as prospective (researchers recruit participants and collect their data over the following years) or retrospective (researchers use an existing longitudinal dataset to “look back in time” at factors contributing to an outcome of interest).

Specific longitudinal study designs include the following.

Repeated cross-sectional study: Researchers collect the same data from distinct samples of the same population over time (essentially a cross-sectional study that is repeated with different people).

Cohort study: Participants who share one or more characteristics (or “exposures”) are followed over time to observe which outcomes they develop.

Panel study: A group of participants, often randomly selected, is studied over time to observe changes in their attitudes or behaviors.
What is the difference between descriptive and experimental research?: Descriptive research describes something as it naturally occurs. Because researchers do not manipulate any variables, descriptive research cannot be used to characterize “cause-and-effect” relationships.

In contrast, experimental research involves manipulating an independent variable to see how it impacts a dependent variable. Experimental research can be used to investigate “why” questions and establish cause-and-effect relationships.
What are the types of descriptive research?: Consider the following common descriptive research methods.

Case studies are detailed explorations of a single subject. The purpose of case studies is to gain a deep understanding of a person, place, or thing. Case studies on several subjects are called case series studies.

Cross-sectional studies capture the characteristics of multiple subjects at a single point in time. Descriptive cross-sectional studies, which describe the prevalence of a trait or a condition (how frequently it occurs) are descriptive; analytical studies, which explore associations between risk factors and outcomes, are not.

Ecological studies are similar to cross-sectional studies but describe groups of people. For example, a cross-sectional study might assess the prevalence of insomnia in American adults, whereas an ecological study would describe the prevalence of insomnia in each of the 50 American states.
What is the goal of descriptive research?: As its name suggests, descriptive research describes something. A researcher conducting descriptive research does not attempt to manipulate any variables; they measure a phenomenon of interest as it naturally occurs.

Descriptive research cannot establish cause-and-effect relationships, but it can be useful for generating hypotheses or learning more about an unfamiliar topic.
What are some descriptive research examples?: Descriptive research is used to describe a person, place, or phenomenon as it naturally occurs. Descriptive research can answer “who,” “what,” “when,” “where,” or “how much” questions but not “why” questions.

Consider the following examples of descriptive research questions:

Who are the primary caregivers for adults with dementia?

What are the main barriers to public transit use in Toronto, Canada?

When do students feel most engaged during online lectures?

Where do young adults shop for groceries?

How much sleep do undergraduate students get during final exams?
What is a theme?: A theme is an idea or pattern that recurs throughout a dataset and is related to a specific research question.

The identification of themes is a core component of thematic analysis, which is a research method commonly used to analyze qualitative data.
What are Braun and Clarke’s 6 steps to thematic analysis?: In their 2006 paper, researchers Virginia Braun and Victoria Clarke outlined the following 6 steps for conducting thematic analysis:

Familiarization

Generating codes

Searching for themes

Reviewing themes

Defining and naming themes

Writing up results
What is reflexive thematic analysis?: Reflexive thematic analysis is a type of thematic analysis that centers the researcher’s interpretation of the data.

Reflexive thematic analysis acknowledges the subjective nature of data interpretation—rather than focusing on uncovering some “ground truth” in the data, researchers are encouraged to engage with their data and use their own knowledge and experiences for interpretation and analysis.
What is thematic content analysis?: Thematic content analysis is often defined in different ways, and the term is sometimes used interchangeably with thematic analysis or qualitative content analysis.

Qualitative content analysis focuses on systematically summarizing the prevalence of specific codes in a dataset, whereas thematic analysis not only codes data but also identifies themes and underlying narratives.

Thematic content analysis is sometimes defined as a hybrid between these two methods: much like qualitative content analysis, its purpose is to describe a body of qualitative data, but data are broken down into themes rather than more simple codes.

Because “thematic content analysis” can be interpreted in different ways, it’s important to provide a detailed description of your methodology if you choose to use this term.
Can explanatory research be qualitative?: Explanatory research examines why something happens. This is usually done by assessing the statistical relationship between two variables.

Because statistical methods like correlation require quantitative (numeric) values, explanatory research generally involves quantitative data.

However, these quantitative explanations may be supplemented by data from, for example, interviews. This mixed methods approach may offer a more comprehensive explanation of a phenomenon.
What’s the difference between explanatory and descriptive research?: The aim of explanatory research is to determine why a phenomenon occurs. This may be done using correlational or experimental research.

On the other hand, descriptive research captures the characteristics of something as is, without intervention.

Though these approaches may share data collection techniques (e.g., they both might use questionnaires), their overarching purpose is distinct.
What are some examples of explanatory research questions?: The goal of explanatory research is to understand why something happens. This is often done by exploring a cause-and-effect relationship between two variables.

Examples of explanatory research questions include the following:

Does talking to plants (cause) make them grow faster (effect)?

Are people more likely to buy chocolate (effect) when they’re sad (cause)?

Does listening to music while studying (cause) improve students’ exam performance (effect)?
What is the difference between correlation and causation?: Causation and correlation are two ways variables can be related.

Causation means changes in one variable directly lead to changes in another (i.e., there is a cause-and-effect relationship). For example, eating food (the cause) satisfies hunger (the effect).

Correlation means there is a statistical relationship between two variables—as one changes, so does the other. However, this relationship is not necessarily causal. For example, although a child’s shoe size and their reading ability are correlated, one does not cause the other (instead, they’re both influenced by a third variable, age).
What are examples of correlation vs causation?: A correlation is a relationship between two variables: as one changes, the other tends to change as well. For example, coffee consumption is correlated with productivity: people who drink more coffee often report getting more done.

Causation, on the other hand, means that a change in one variable directly causes changes in another. To test whether coffee actually increases productivity, you could conduct an experiment: assign some people to drink coffee and others to drink water, and compare their task performance.

It’s important to remember that correlation does not imply causation. Even if coffee consumption and productivity are correlated, it doesn’t mean one causes the other. It’s possible that people who are working more are tired, so they drink more coffee.
What is Pearson’s r?: Pearson’s r (the “Pearson product–moment correlation coefficient,” or simply “r”), is the most common way to compute a correlation between two variables. It tells you how two variables are related. Most statistical tools (like R or Excel) have a built-in correlation function.

The value of r ranges from -1 to +1. The sign of r (+ or –) indicates the direction of a relationship (whether a correlation is positive or negative), and the magnitude of r indicates the strength of the relationship (sometimes called the effect size).

What is considered a strong, moderate, or weak correlation varies by field. Many researchers use Cohen’s size criteria as a guideline:

Cohen’s size criteria

r value Direction Strength

Between –1 and –0.5 Negative Strong

Between –0.5 and –0.3 Negative Moderate

Between –0.3 and –0.2 Negative Weak

Between –0.2 and +0.2 N/A No correlation

Between +0.2 and +0.3 Positive Weak

Between +0.3 and +0.5 Positive Moderate

Between +0.5 and +1 Positive Strong
What is regression?: Correlation tests the strength and direction of a relationship between two variables.

Regression goes a step further: it lets you model the relationship between a dependent variable and one or more independent variables, often using a line of best fit that lets you make predictions about your data.
What are random sampling methods?: Random sampling (also called probability sampling) is a category of sampling methods used to select a subgroup, or sample, from a larger population. A defining property of random sampling is that all individuals in the population have a known, non-zero chance of being included in the sample.

Random sampling methods include simple random sampling, systematic sampling, stratified sampling, and cluster sampling. All of these methods require a sampling frame (a list of all individuals in the population).

The opposite of random or sampling is non-probability sampling, where not every member of the population has a known chance of being included in the sample.
What is sampling?: Sampling is the process of selecting a subset of individuals (a sample) from a larger population.

Because it’s often not feasible to collect data from every individual in a population, researchers study a sample instead. The goal is to use this sample to make predictions (or inferences) about the broader population.

For example, if you want to study consumer attitudes towards a brand, you might survey a subset of customers rather than every single one.

There are different sampling methods that can be used to select a sample.
Is simple random sampling probability or nonprobability sampling?: Simple random sampling is a probability sampling method. Individuals are selected randomly from a list of all members of the population (the sampling frame), so everyone has an equal chance of being included in the sample..

This method has reduced sampling bias compared to other sampling methods, but it can be more difficult to conduct. It requires a complete list of the population and does not consider how easy or difficult it is to reach selected individuals.
What is sampling bias?: Sampling bias occurs when some individuals in the population are more likely to be included in a sample than others. This can limit how well results generalize to the broader population.

Sampling methods like probability sampling help reduce sampling bias because every individual in the population has a known, non-zero chance of being included in the sample. However, it’s difficult to eliminate sampling bias entirely, so results from a sample should always be interpreted with caution.
What are sample statistics vs population parameters?: In statistics, population parameters are characteristics that describe a population (such as mean, standard deviation, and variance). They are calculated using the data from every member of the group you want to learn about, so they provide a completely accurate description of that population.

Sample statistics, on the other hand, are calculated from a sample (a subset of the population). Sample statistics provide an estimate of population parameters, but because they do not include data from every member of the population, they may be biased or inaccurate.
How do I know if my data are from a population or a sample?: Knowing whether your data are from a population or a sample is key to properly analyzing or interpreting your results.

If your data are from a subset of the group you are studying, your data represent a sample. If instead your data have been collected from every single individual you are interested in studying, your data are from a population.

If you are analyzing data you did not collect yourself, consider how likely it is that the researchers who collected this data gathered measurements from every single individual they were interested in studying.

Researchers generally collect data from a smaller group and use the results to make inferences about the population, so there’s a good chance that these data are from a sample rather than a population.
What is an example of purposive sampling?: Purposive sampling, or judgment sampling, is a non-probability sampling method that involves hand-picking individuals to include in a study based on certain characteristics.

For example, a researcher studying how cancer patients cope with terminal illness may directly recruit several late-stage cancer patients who are receiving palliative care at their clinic.

Unlike in probability sampling, not every cancer patient has an equal chance of being selected. Instead, the researcher can choose the cases they feel will be most informative.

Purposive sampling can be helpful when the researcher is very familiar with the population they are studying, as it allows them to select individuals who best represent this group. However, any biases the researcher holds may be reflected in the sample.
What is the difference between purposive sampling and convenience sampling?: Purposive sampling and convenience sampling are two non-probability sampling methods, meaning not every individual from the population has an equal chance of being selected. Sampling methods are ways of choosing individuals from a population to study.

Purposive sampling is when a researcher hand-picks individuals because they possess specific traits or characteristics. For example, someone studying successful teaching techniques might only include teachers who have recently won awards in their sample.

On the other hand, convenience sampling involves selecting individuals simply because they are easily accessible. For example, a business might ask their social media followers to complete a survey.

Convenience sampling and purposive sampling are not mutually exclusive—a researcher might use some combination of both techniques when obtaining a sample for their study.

Both of these techniques are susceptible to sampling bias. Individuals who are readily accessible or who the researcher chooses to participate may not be fully representative of the broader population.
What is the difference between random sampling and purposive sampling?: Random sampling—or probability sampling—includes a range of sampling methods used to select a subgroup of individuals from a larger population. A key characteristic of random sampling is that every individual has a known chance of being selected.

Purposive sampling, on the other hand, is a non-probability sampling technique. In this method, not every individual has a known chance of being included in the sample. Instead, the researcher chooses who they include in their sample based on certain traits or experiences. This can be helpful if the researcher is very familiar with the population they are studying and wants to gain rich, targeted insight rather than generalizable information.

Purposive sampling is often easier and more efficient than random sampling. However, purposive sampling is highly susceptible to sampling bias. Random sampling is a better approach for obtaining a representative sample that reflects the broader population.

To learn more about random sampling vs purposive sampling, use QuillBot’s free AI search.
What are the types of purposive sampling?: Purposive sampling is a sampling method where the researcher intentionally selects individuals to study based on desired characteristics or experiences relevant to their research question.
There are several common approaches to purposive sampling:

Maximum variation (heterogeneous) sampling: includes individuals who differ from each other as much as possible to capture a range of experiences

Homogeneous sampling: includes individuals who are very similar to each other to enable a detailed exploration of a certain subgroup

Typical case sampling: includes individuals who best reflect the average or norm of a population

Extreme (deviant) case sampling: includes outliers who fall significantly above or below the norm

Critical case sampling: includes individuals whose results are likely to generalize—if it happens to them, it would probably happen to anyone

Expert sampling: includes individuals with specialized knowledge or expertise relevant to the research topic
What is the difference between sampling bias and selection bias?: There’s not a universally agreed-upon distinction between sampling bias and selection bias, but sampling bias is often considered a subtype of selection bias.

Sampling bias occurs when a sample is not random (i.e., it differs from the target population). It impacts external validity—how well the results generalize from the sample to the population.

Selection bias, on the other hand, refers more broadly to bias introduced when selecting who to include in a study. It impacts internal validity—whether your results can be explained by the independent variable you manipulated (and not by other confounds).

The distinction between sampling and selection bias is complex. AI tools like QuillBot’s Paraphrasing Tool can be helpful when trying to parse difficult concepts.
What are some types of sampling bias?: Sampling bias occurs when the sample collected for a study systematically differs from the target population. Below are some common types of sampling bias:

Self-selection bias: People who choose to participate in a study differ from the general population in an important way (e.g., motivation, interest).

Nonresponse bias: Those who are unable or unwilling to respond often share key characteristics, and their absence may skew results.

Healthy user bias: Individuals who are able or willing to participate are often healthier or more health-conscious than nonparticipants.

Survivorship bias: Data are only available for individuals or outcomes that pass a certain filter (e.g., those who survive an event); those that didn’t are ignored.

Undercoverage bias: Certain subgroups are systematically excluded from the sample, leading to skewed representation.

Prescreening bias: Eligibility criteria (e.g., age, language) may unintentionally exclude relevant parts of the population.

Not sure which types of sampling bias are applicable to your study? AI tools are a great way to generate ideas and receive dynamic feedback on study design. Try QuillBot’s free AI Chat the next time you’re feeling short on inspiration.
How do you avoid sampling bias?: Though it’s difficult to fully eliminate sampling bias, it can be minimized through careful research design and sampling methods.

Probability sampling methods (where every member of the population has a known chance of being selected) are less susceptible to sampling bias than nonprobability methods.

Looking for ways to minimize sampling bias that are tailored to your specific situation? Get ideas from QuillBot’s free AI Chat.
What is inter-rater reliability in psychology?: In psychology, inter-rater reliability refers to the degree of agreement between different observers or raters who evaluate the same behavior, test, or phenomenon.

It ensures that measurements are consistent, objective, and not dependent on a single person’s judgment, which is especially important in research, clinical assessments, and behavioral studies.

High inter-rater reliability indicates that results are dependable and reproducible across different raters.
What is a good inter-rater reliability score?: A good inter-rater reliability score depends on the statistic used and the context of the study.

For Cohen’s kappa (two raters), common guidelines are:

< 0.20: Poor agreement

0.21–0.40: Fair agreement

0.41–0.60: Moderate agreement

0.61–0.80: Substantial agreement

0.81–1.00: Almost perfect agreement

For the Intraclass Correlation Coefficient (interval or ratio data), similar thresholds are used:

< 0.50: Poor agreement

0.51–0.75: Moderate agreement

0.76–0.90: Good agreement

> 0.91: Excellent agreement

Paraphraser

Grammar Checker

AI Detector

Plagiarism Checker

AI Humanizer

AI Chat

Translate

Summarizer

Citation Generator

What is the formula for calculating inter-rater reliability?

Research: Other interesting questions

*r value*	Direction	Strength
Between –1 and –0.5	Negative	Strong
Between –0.5 and –0.3	Negative	Moderate
Between –0.3 and –0.2	Negative	Weak
Between –0.2 and +0.2	N/A	No correlation
Between +0.2 and +0.3	Positive	Weak
Between +0.3 and +0.5	Positive	Moderate
Between +0.5 and +1	Positive	Strong