DIVISION OF SURGERY AND INTERVENTIONAL SCIENCE
FACULTY OF MEDICAL SCIENCES
Lecture 2: Structure of Research and
How Research Works
Prof Kurinchi Gurusamy
Lecture 2: Structure of Research and How Research Works
Research Methodology and Transferable
Skills: Lecture 2 1
Learning objectives
Understand the structure of research
Understand how research works
Distinguish between external validity and internal validity
List the different types of errors in research
Research Methodology and Transferable Skills: Lecture 2 2
After having understood what research means, the next step is to understand the
structure of research and how research works. We will learn those in this lecture. In
addition, we introduce you to the different types of errors in research.
2
Research Methodology and Transferable
Skills: Lecture 2
Structure of research
The major components of any research are the following.
Research question(s)
Background and significance
Design
Subjects
Variables
Statistical issues
Hulley, Stephen B., et al. Designing Clinical Research, Wolters Kluwer, 2013
Research Methodology and Transferable Skills: Lecture 2 3
The major components of any research are listed in this slide. Some explanations of
these terms follow in the next few slides.
3
Research Methodology and Transferable
Skills: Lecture 2
Research Question
Objective of the study
Uncertainty that the researcher wants to resolve
Origin can be a general question, but it should then be narrowed into a
researchable issue
Hulley, Stephen B., et al. Designing Clinical Research, Wolters Kluwer, 2013
Research Methodology and Transferable Skills: Lecture 2 4
Research question is the objective of the study: the uncertainty that the researcher
wants to resolve. The origin can be a general question, but it should then be
narrowed into a researchable issue.
4
Research Methodology and Transferable
Skills: Lecture 2
What is a good research question
A good research question should pass the “So what ” test. Getting the
answer should contribute usefully to our state of knowledge
A good research question should be Feasible, Interesting, Novel, Ethical,
and Relevant (FINER)
More details in the next few slides
Hulley, Stephen B., et al. Designing Clinical Research, Wolters Kluwer, 2013
Research Methodology and Transferable Skills: Lecture 2 5
A good research question should pass the “So what ” test. Getting the answer should
contribute usefully to our state of knowledge. It should be Feasible, Interesting,
Novel, Ethical, and Relevant (FINER). More details about the FINER aspects of
research will be provided in the next few slides.
Research Methodology and Transferable
Skills: Lecture 2 5
Feasibility
Adequate sample size (details in later lecture)
Some information on the number of participants who can be recruited
Surveys
Chart reviews
Interviews
Can also help with the type of study design
Sufficient technical expertise
Sufficient money and time
Avoid asking interesting side questions that can decrease feasibility
Hulley, Stephen B., et al. Designing Clinical Research, Wolters Kluwer, 2013
Research Methodology and Transferable Skills: Lecture 2 6
The research should plan to include an adequate sample size to answer the research
question (the question ‘what is an adequate sample size’ is covered in a later lecture).
In clinical trials, you should include some information on the number of participants
who can be recruited. This may necessitate surveys, chart reviews, and/or interviews.
This can also help with the type of study design used and single centre or multicentre
study. There should be sufficient technical expertise available to deliver the study. In
addition to ensuring that adequate personnel are available, there should be sufficient
money and time. While it is tempting to answer the interesting side questions (‘let’s
ask that too while we are here’), it is usually better to focus on the main questions.
Research Methodology and Transferable
Skills: Lecture 2 6
Interesting
Many reasons for performing a study
Research provides financial support
Logical or important next step in building a career
Getting at the truth of the matter is interesting
Important to confirm that you are not the only one who finds a question
interesting
Speak with mentors, outside experts, and representatives of potential
funders
Hulley, Stephen B., et al. Designing Clinical Research, Wolters Kluwer, 2013
Research Methodology and Transferable Skills: Lecture 2 7
A researcher may perform the study for various reasons. Some of the reasons include
the research provides financial support, because it is a logical or important next step
in building a career, or because getting at the truth of the matter is interesting.
However, it is wise to confirm that you are not the only one who finds a question
interesting. Speaking with mentors, outside experts, and representatives of potential
funders before devoting substantial energy can help with avoiding development of a
research plan or grant proposal that is unlikely to be funded.
Researchers do not realise how useful it is to get informal feedback from the
programme managers of the funding streams. The programme managers provide lot
of useful information, as they want to fund high quality research. They may have
previous experience as part of the funding panel and are usually willing to share their
knowledge.
Research Methodology and Transferable
Skills: Lecture 2 7
Novel
Covered in previous lecture
Research projects should pursue entirely new advancements in knowledge
May result from a project to reproduce an existing result that finds
potential discrepancies
Also includes an experimental development project aimed at creating:
Knowledge in support of the development of new concepts
Ideas related to the design of new products or processes
Research Methodology and Transferable Skills: Lecture 2 8
The novelty aspect of research was covered in the last lecture. In the context of
healthcare, research projects should pursue entirely new advancements in
knowledge. Novelty could result from a project to reproduce an existing result that
finds potential discrepancies. An experimental development project aimed at creating
knowledge in support of the development of new concepts and ideas related to the
design of new products or processes should be included in R&D.
Research Methodology and Transferable
Skills: Lecture 2 8
Ethical
If the study design poses unacceptable risk to people or invasion of privacy
Find other ways of finding the answer to the research question
For example, using a different study design
If you are not sure whether your study is ethical
Speak to mentors
Seek preliminary advice from ethics committee
Hulley, Stephen B., et al. Designing Clinical Research, Wolters Kluwer, 2013
Example: Research to find out whether hand washing after public transport
can decrease coronavirus transmission
Unethical: to ask people to avoid hand washing
Ethical: mobile app reminder to remind people to wash hands/check
how clean the hands are versus standard advice to wash hands
Research Methodology and Transferable Skills: Lecture 2 9
A good research question should be ethical. If the study design poses unacceptable
risk to people or invasion of privacy, the researcher should find other ways of finding
the answer to the research question, for example, using a different study design. If
you are not sure whether your study is ethical, it is useful to speak to mentors and
seek preliminary advice from ethics committee.
An example
For a research question ‘Does washing hands after travelling on public transport
decrease coronavirus transmission ‘, it is unethical to perform an experiment by
asking some people not to wash hands after travelling on public transport and others
to wash hands after travelling on public transport. However, it may be ethical to test a
mobile app that reminds the participants to wash their hands after they travelled on
public transport (and tests how clean the hands are) versus standard advice to wash
hands after travelling on public transport.
Research Methodology and Transferable
Skills: Lecture 2 9
Relevant
Imagine what the possible outcomes of research are
How the research might advance scientific knowledge, influence practice
guidelines and health policy, or guide further research
Hulley, Stephen B., et al. Designing Clinical Research, Wolters Kluwer, 2013
For example, in most grant applications
Section on outputs from the research
How this might impact on patients, researchers, health services, and/or
general public
You could speak to the potential beneficiaries to find out how
relevant/important the research might be to them
Research Methodology and Transferable Skills: Lecture 2 10
A good research question must be relevant. One way to test this is to imagine what
the possible outcomes of research are and how this might advance scientific
knowledge, influence practice guidelines and health policy, or guide further research.
An example
In most grant applications, there is a section on outputs from the research and how
this might impact on patients, researchers, health services, and/or general public (the
beneficiaries may vary depending upon whether is basic research, applied research,
or experimental development). You could speak to the potential beneficiaries to find
out how relevant or important the research might be to them.
Research Methodology and Transferable
Skills: Lecture 2 10
Secondary Research Question
There can be additional research questions
May make the study more complex to design and conduct
Focus the study design on the primary research question
Include other secondary research questions which might lead to valuable conclusions
Hulley, Stephen B., et al. Designing Clinical Research, Wolters Kluwer, 2013
For example:
Primary research question: Does washing hands after travelling on
public transport decrease coronavirus transmission
Secondary research question: Does washing hands after travelling on
public transport decrease other infections
Research Methodology and Transferable Skills: Lecture 2 11
There can be additional research questions. However, this can make the study more
complex to design and conduct. A good strategy is to focus the study design on the
primary research question and include other secondary research questions which
might lead to valuable conclusions.
An example
The primary research question might be whether washing hands after travelling on
public transport decreases coronavirus transmission. The secondary research
question might be whether washing hands after travelling on public transport
decreases other infections.
Research Methodology and Transferable
Skills: Lecture 2 11
: Background and significance
Background and significance
What is already known
Previous research on the topic (including the researcher’s own research
on the topic)
Problems with prior research
Uncertainties in prior research
Why is the research question important
What kind of answers will the study provide
Hulley, Stephen B., et al. Designing Clinical Research, Wolters Kluwer, 2013
Research Methodology and Transferable Skills: Lecture 2 12
There are various other components of research. These are shown in this slide and
the next few slides. It should be noted that many of these aspects are covered in later
lectures. The aspects related to background and significance are shown on this slide.
Research Methodology and Transferable
Skills: Lecture 2 12
Other components of research: Study
design
Study design
Various advantages and disadvantages in different study designs are
described in later lectures
Research Methodology and Transferable Skills: Lecture 2 13
The various advantages and disadvantages in different study designs are described in
later lectures.
Research Methodology and Transferable
Skills: Lecture 2 13
Other components of research:
Variables
A factor in a scientific experiment that may be subject to change
A quantity that may assume any one of a set of values
Merriam-Webster Dictionary (online)
May be predictor variables, outcome variables, or confounding variables
Covered in later lecture
May or may not change over time for the same individual
For example, age changes over time for the same individual, but ethnic
origin does not change over time
Still called variable as it can assume different values for different
subjects
Research Methodology and Transferable Skills: Lecture 2 14
Merriam-Webster Dictionary (online) defines variable as a factor in a scientific
experiment that may be subject to change. Another definition for variable is that it is
a quantity that may assume any one of a set of values.
In the context of research, variables can be predictor variables (independent
variables), outcome variables (dependent variables), or confounding variables. These
types of variables will be described in a later lecture.
It is important to note that some variables change over time for a specific
individual, for example, age, while others do not change over time for a specific
individual, for example, ethnic origin. Regardless of whether the values change or do
not change over time for a specific individual, these are called variables, since these
may take different values for different research subjects or research materials.
Research Methodology and Transferable
Skills: Lecture 2 14
Other components of research
Constants
Opposite of variable
Do not vary in different research subjects
Statistical issues
Covered in relevant lecture
Research Methodology and Transferable Skills: Lecture 2 15
Constants
In the context of research, constants are the opposite of variables and do not vary
with different individuals.
Statistical issues
These are discussed under the relevant lecture.
Research Methodology and Transferable
Skills: Lecture 2 15
How research works
Main purpose of clinical research: arrive at conclusions about a population
based on what you observe in a study (‘Inferences’)
Two types of inferences
Internal validity: degree to which the researcher makes the correct
conclusions about the study sample.
External validity (also called generalisability or applicability): degree to
which these conclusions about the study sample can be applied in the
population
Relation between research question, research plan, and internal and
external validity: explained with examples in next few slides
Research Methodology and Transferable Skills: Lecture 2 16
Having learnt the structure of research, we will learn next about how research works.
The main purpose of clinical research is to arrive at conclusions about a population
based on what you observe in a study (‘Inferences’). The two types of inferences are
internal validity and external validity. Internal validity is the degree to which the
researcher makes the correct conclusions about the study sample. External validity
(also called generalisability or applicability) is the degree to which these conclusions
about the study sample can be applied in the population.
The relation between research question, research plan, and internal and external
validity and some examples explaining these concepts are shown in the next few
slides.
Research Methodology and Transferable
Skills: Lecture 2 16
validity
Research Methodology and Transferable Skills: Lecture 2 17
This slide shows the relation between research question, research plan, and internal
and external validity. While converting the research question to the research plan, it
is possible that the research plan does not match the research question. This means
that the research does not have good external validity. To have good external validity,
the sample studied must be representative of the population in whom the results of
the study are applied. Even if the research plan reflects the research question, there
may be ‘systematic errors’ in the conduct of the study (described later in this lecture)
because of which the results of the study do not reflect the truth in the sample
studied. This means that the research does not have good internal validity. Only the
results of a study with good internal and external validity are applicable in the
population of interest.
In the next few slides, some examples of studies with poor external validity are
shown. Internal validity will be discussed in the lecture related to ‘bias’ and examples
of poor internal validity will be provided in that lecture.
Research Methodology and Transferable
Skills: Lecture 2 17
Poor external validity: Example 1
Research question: Does washing hands after travelling on public
transport decrease coronavirus transmission
Research plan: Study washing hands in people who travel to office on
bikes.
Problem with external validity
Results are only applicable in people who travel to office on bikes and
not in people travelling on public transport
Research Methodology and Transferable Skills: Lecture 2 18
An example of a study with poor external validity is shown in this slide. If you perform
a study of washing hands in people who travel to office on bikes, the results are
applicable only in people who travel to office on bikes and not in people travelling on
public transport. Therefore, the study has poor external validity to answer the
research question, “Does washing hands after travelling on public transport decrease
coronavirus transmission”
Research Methodology and Transferable
Skills: Lecture 2 18
Poor external validity: Example 2
Research question: Does the antihypertensive drug (‘Drug A’) prevent
deaths in people aged 75 years
Research plan: Study only people aged 75 years without any other
illnesses.
Problem with external validity
Results are only applicable in people aged 75 years without any other
illnesses
Most 75-year-olds have illnesses besides hypertension
Research Methodology and Transferable Skills: Lecture 2 19
Another example of a study with poor external validity is shown in this slide. If you
perform a study in only people aged 75 years without any other illnesses, then the
results are only applicable in people aged 75 years without any other illnesses.
However, most 75-year-olds have illnesses besides hypertension. This means that the
study has poor external validity in answering the research question “Does the
antihypertensive drug (‘Drug A’) prevent deaths in people aged 75 years”
Research Methodology and Transferable
Skills: Lecture 2 19
Poor external validity: Example 3
Research question: Is ‘Material A’ better than ‘Material B’ in manufacturing
3-D printed scaffolds
Research plan: Use inferior quality of ‘Material B’ and compare this with
high quality ‘Material A’.
Problem with external validity
Results are only applicable in the comparison between high quality
‘Material A’ versus inferior quality ‘Material B’
We cannot make a general conclusion that ‘Material A’ is better than
‘Material B’
Research Methodology and Transferable Skills: Lecture 2 20
A third example of a study with poor external validity is shown in this slide. If you
perform a study in which you used inferior quality of ‘Material B’ for manufacturing 3-
D printed scaffolds and compare this with 3-D printed scaffolds manufactured using
high quality ‘Material A’, then the results are only applicable in the comparison of 3-D
printed scaffolds manufactured with high quality ‘Material A’ versus inferior quality
‘Material B’. Therefore, we cannot make a general conclusion that ‘Material A’ is
better than ‘Material B’ for manufacturing 3-D printed scaffolds. This means that the
study has poor external validity in answering the research question “Is ‘Material A’
better than ‘Material B’ in manufacturing 3-D printed scaffolds”
Research Methodology and Transferable
Skills: Lecture 2 20
Why must you to make inferences
Usually not possible to study the whole population
Highly resource intensive to study the whole population
In addition, the main purpose of preclinical and clinical research: make
some predictions about future participants or material based on our
knowledge of studying the sample
Cannot study the whole population (‘Universe’), as it includes the future
Research Methodology and Transferable Skills: Lecture 2 21
One question that you may have, is that why must we make inferences Why can’t
we study the whole population
It is not possible to study the whole population in most research. This is because it is
highly resource intensive to study the whole population. In addition, the main
purpose of preclinical and clinical research is to make some predictions about future
participants or material based on our knowledge of studying the sample. Therefore,
we cannot study the whole population, as it includes the future. Some use the term
‘universe’ to denote population.
Research Methodology and Transferable
Skills: Lecture 2 21
Errors
Introduction
Research Methodology and Transferable Skills: Lecture 2 22
Errors in research plan can lead to observations that may not reflect the truth. There
are two major types of errors: random errors and systematic errors. We provide a
brief introduction to these errors in the next few slides.
Research Methodology and Transferable
Skills: Lecture 1 22
Random errors
Research Methodology and Transferable Skills: Lecture 2 23
Random errors can be best explained by an analogy of tossing a ‘fair coin’. A ‘fair coin’
is a coin where there is equal probability of getting ‘heads’ and ‘tails’. This is in
comparison with a ‘biased coin’ where the coin has been altered, for example
attaching a thin layer of wax on one side, say the ‘heads’ side, which makes the coin
heavier on the ‘heads’ side and the coin to stick to the surface on the ‘heads’ side
resulting in more ‘tails’.
Let us toss a ‘fair coin’ 10 times. This can be considered a single experiment or a
single clinical study of 10 participants. Even if you toss the ‘fair coin’ ten times
without cheating and you expect to get 5 heads and 5 tails, you may sometimes get 4
heads and 6 tails or 6 heads and 4 tails. You may even get 9 heads and 1 tail or 10
heads purely by chance, although the probability of such an observation purely due
to chance is very low. The figure in this slide shows how many heads were obtained in
25 experiments of 10 coin tosses each. This type of figure which shows how often an
event occurred (in this case, number of heads in 10 coin tosses) is called a
histogram.
It is clear from this figure of 25 experiments (of 10 coin tosses each) that you do not
get 5 heads and 5 tails in all the experiments, despite the expected value being 5
Research Methodology and Transferable
Skills: Lecture 2 23
heads and 5 tails. If you conduct coin toss experiments of 100,000 coin tosses in each
experiment, the results will be different. In each experiment, the number of heads
will be very close to 50%.
Most preclinical or clinical research studies involve only a few participants and not
hundreds of thousands of participants. You may observe that 5% of elderly people
might have had silent heart attacks in a particular clinical study. However, this does
not mean that the true percentage or proportion of people with silent heart attacks
in the elderly population from which the sample was drawn is 5%. Just the same way
that getting 4 heads in 10 coin tosses does not mean that the probability of getting
heads in a fair coin is 40% (it is 50% and the observed value of 40% is because of
random error), observing a value of 5% in the sample does not mean that the true
value in the population is 5% even in the absence of other types of errors.
Similarly, while testing a medical device, you might find that the battery in the device
lasts 2 years, but in truth, the battery could last only for 1 year even in the absence of
other types of errors.
These errors that occur because of ‘chance’ are called ‘random errors’. There are
different types of random errors (these are covered in later lectures). The random
errors can be reduced by increasing the sample size.
Research Methodology and Transferable
Skills: Lecture 2 23
Systematic errors
Another name: ‘bias’
In the context of preclinical and clinical research:
Bias = prejudice or preconceived judgement or opinion.
Bias has different meaning in the context of voltage applied to a device.
Merriam Webster Dictionary Online
Research Methodology and Transferable Skills: Lecture 2 24
In the previous slide, we learnt about random errors. The other major type of error in
research is systematic error. Another name for systematic error is ‘bias’. In the
context of preclinical and clinical research, bias means prejudice or preconceived
judgement or opinion. Note that this has a different meaning in the context of voltage
applied to a device.
Research Methodology and Transferable
Skills: Lecture 2 24
Other errors
Sampling errors
Refers to random error or systematic error
Hulley, Stephen B., et al. Designing Clinical Research, Wolters Kluwer, 2013
Measurement errors
Errors in accuracy, precision, or sampling
Covered in a later lecture
Could be random errors or systematic errors
Design errors
Some use this term to refer to external validity
Keus et al. BMC Med Res Methodol 10, 90 (2010). https://doi.org/10.1186/1471-2288-10-90
Research Methodology and Transferable Skills: Lecture 2 25
While most methodologists agree on what random and systematic errors are, there
are other terms which are used differently by different methodologists.
Sampling errors
‘Sampling errors’ is a term that refers to random error or systematic error.
Measurement errors
These refer to errors in accuracy, precision, or sampling. These terms are described in
a later lecture on measurement errors. Measurement errors could be random errors
or systematic errors.
Design errors
Some use this term to refer to external validity.
Research Methodology and Transferable
Skills: Lecture 2 25
Summary
Structure of research
How research works
Distinguish between external validity and internal validity
Different types of errors in research
Research Methodology and Transferable Skills: Lecture 2 26
This brings us to the end of lecture 2. In this lecture, we learnt the concepts shown in
this slide. In the next lecture, we will learn some skills related to searching literature
and reference management.
Research Methodology and Transferable
Skills: Lecture 1 26
Reality check
Check your understanding
Research Methodology and Transferable Skills: Lecture 2 27
Check your understanding of the concepts explained in this lecture by answering the
questions in the next few slides.
Research Methodology and Transferable
Skills: Lecture 2 27
Reality Check 1
State True or False. A research project that attempts to reproduce the
findings of another research study is not novel.
Research Methodology and Transferable Skills: Lecture 2 28
Reality check 1: State True or False.
Research Methodology and Transferable
Skills: Lecture 2 28
Reality Check 1 (answer)
State True or False. A research project that attempts to reproduce the
findings of another research study is not novel.
Attempts to reproduce the findings of previous research is novel as it
is aimed at finding potential discrepancies. Therefore, the correct
answer is ‘False’.
Research Methodology and Transferable Skills: Lecture 2 29
Attempts to reproduce the findings of previous research is novel as it is aimed at
finding potential discrepancies. Therefore, the correct answer is ‘False’.
Research Methodology and Transferable
Skills: Lecture 2 29
Reality Check 2
State True or False. A researcher wants to conduct an experimental
study to test a device. The device could potentially harm the research
participant. No immediate benefits to research participants are
expected, but the research may lead to an advancement in science.
This research is ethical.
Research Methodology and Transferable Skills: Lecture 2 30
Reality check 2: State True or False.
Research Methodology and Transferable
Skills: Lecture 2 30
Reality Check 2 (answer)
State True or False. A researcher wants to conduct an experimental
study to test a device. The device could potentially harm the research
participant. No immediate benefits to research participants are
expected, but the research may lead to an advancement in science.
This research is ethical.
The research poses unacceptable risk to participants with no
prospects of benefits. Therefore, this research is unethical. The
correct answer is ‘False’.
Research Methodology and Transferable Skills: Lecture 2 31
The research poses unacceptable risk to participants with no prospects of benefits.
Therefore, this research is unethical. The correct answer is ‘False’.
Research Methodology and Transferable
Skills: Lecture 2 31
Reality Check 3
A researcher wants to test the iron content per gram of different
polymers at a specific time point. Using an advanced technology, it is
possible to control the exact iron content in each different type of
polymer and keep it at a constant level for the particular polymer. In
this research, is iron content a variable or constant
Research Methodology and Transferable Skills: Lecture 2 32
Reality check 3: Is iron content in this research a variable or constant
Research Methodology and Transferable
Skills: Lecture 2 32
Reality Check 3 (answer)
A researcher wants to test the iron content per gram of different
polymers at a specific time point. Using an advanced technology, it is
possible to control the exact iron content in each different type of
polymer and keep it at a constant level for the particular polymer. In
this research, is iron content a variable or constant
Even though the iron content can be kept at a constant level for the
polymer, it differs from one polymer to another. Therefore, iron
content is a variable in this research.
Research Methodology and Transferable Skills: Lecture 2 33
Even though the iron content can be kept at a constant level for the polymer, it differs
from one polymer to another. Therefore, iron content is a variable in this research.
Research Methodology and Transferable
Skills: Lecture 2 33
Reality Check 4
A researcher wants to find out whether daily aspirin is useful in
preventing heart attacks in people above 40 years of age. In the
clinical study, they excluded people with allergies and
contraindications for aspirin. Which of the following statements is
correct
The findings of this research are applicable in all people above 40
years of age
The findings of this research are applicable in all people above 40
years of age who do not have allergies or contraindications to
aspirin
The findings of this research are not applicable in anybody
regardless of their age
The findings of this research are applicable in people below 40 years
of age
Research Methodology and Transferable Skills: Lecture 2 34
Reality check 4: Which of the statements shown in this slide is correct
Research Methodology and Transferable
Skills: Lecture 2 34
Reality Check 4 (answer)
A researcher wants to find out whether daily aspirin is useful in
preventing heart attacks in people above 40 years of age. In the
clinical study, they excluded people with allergies and
contraindications for aspirin. Which of the statements in the previous
slide is correct
Since the researchers excluded people with allergies and
contraindications for aspirin, the findings of this research are only
applicable in people above 40 years of age without allergies or
contraindications to aspirin.
Research Methodology and Transferable Skills: Lecture 2 35
Since the researchers excluded people with allergies and contraindications for aspirin,
the findings of this research are only applicable in people above 40 years of age
without allergies or contraindications to aspirin.
Research Methodology and Transferable
Skills: Lecture 2 35
Reality Check 5
State True or False. Even in the absence of systematic errors, it is not
possible that a researcher concludes that a ‘treatment A’ is better than
‘treatment B’ because of random error.
Research Methodology and Transferable Skills: Lecture 2 36
Reality check 5: State True or False.
Research Methodology and Transferable
Skills: Lecture 2 36
Reality Check 5 (answer)
State True or False. Even in the absence of systematic errors, it is not
possible that a rese
