MODULE 3

METHODS OF SCIENTIFC RESEARCH


NATURALISTIC OBSERVATION

In many scientific studies, naturalistic observation is the only tool available

In the early phases of scientific inquiry, it is necessary to begin by examining the variety of natural events that seem relevant to the subject of the inquiry. By noting whatever arises naturally, the scientist begins to generate a body of knowledge in which he can find patterns and detect possible cause-and-effect relationships.

Such naturalistic observation is a method of investigation in which an interested person simply observes and records events without trying to influence them in any way. It was the historical precursor of scientific experimentation as we know it today, and such procedures still have value in scientific study. Astronomy, for example, must employ naturalistic observation as its major research tool, since it cannot influence the objects observed. When a biological researcher sets up a camera to photograph the nest-building behavior of the crane, he is engaging in naturalistic observation. He has not in any way disturbed the birds; he has merely observed and recorded their behavior. We observe behavior around us every day; to some extent such observations help us predict and control our own behavior and affect the behavior of others.

Also, such observations help us form ideas and hypotheses about behavior that can be tested experimentally. Have you ever tried to housebreak a puppy? Perhaps you noticed that if you watched him carefully he would give certain signs before he needed to be put outside. Or you may have observed that a pat and some attention after a satisfactory performance outdoors worked better than punishment after an accident. Such observations can lead to a scientific study of learning. For example, you might start to investigate the effect of giving more (or less) reward, or the relationship between the frequency of punishment and the number of days required to train the puppy. When a scientist begins to influence the things he observes, and record the effect of those changes, he has taken a step beyond naturalistic observation. He has imposed some control on the situation in order to refine his knowledge of cause and effect in this case, the effect of reward and punishment on a puppy's behavior.

THE SCIENTIFIC METHOD

The experimental method is often used to test a hypothesis formed as a result of naturalistic observation

One problem with naturalistic observation is that it may take a long time to gather enough relevant data to reach a conclusion; the exact combination of events we need to observe may not occur often. Also, in order to establish beyond doubt just what factors determine the occurrence of an event, it is necessary to make observations that are reproducible, so that anyone can repeat the experiment and verify the results. For this reason, researchers began to specify their procedures precisely, so that others-who were interested in their field could carry out the same operations and obtain the same results. Following their hunches and sharing their results with each other, scientists evolved a method of doing research which came to be called the experimental method Although this is not the only method used by scientists (events such as a comet's appearance or a baby's development cannot be experimentally manipulated for natural or cultural reasons), it is the scientist's basic research tool.

The experimental method is used when a researcher has observed that certain factors, or variables, seem to be more important than others, and formed at least a rough hypothesis about cause and effect. These variables may be incorporated into an experiment. Basically, the experiment is conducted by changing the value of one variable and measuring the changes in another variable while holding everything else constant.

Controlled experiments seek to demonstrate cause-and-effect relationships between independent and dependent variables

For example, to study the effect of pay increases on the output of factory workers, we might increase the pay of some workers by a certain number of dollars and measure their output. The two variables in our experiment would be the rate of pay and the amount of work produced. The rate of pay is the independent variable; we manipulate it according to our experimental design. The amount produced is the dependent variable, which we would measure as we changed the value of the independent variable. We would try to make all other factors that might be relevant, such as the workers' attitudes, their working conditions, and their outside incomes, as similar as possible throughout the period of experimentation. What we know in an experiment is the change we make in the independent variable

Students often have trouble keeping the terms independent end dependent straight. It may help you to remember that the dependent variable is the one which depends on what happens in the experiment. Consider the everyday problem of cooling your coffee. The temperature of the coffee may be considered a dependent variable, which depends on many factors: e.g., the temperature of the pot it was poured from, the length of time since it was poured, and the thickness and the temperature of the cup.

What we want to learn is the change produced in the dependent variable

One easily controllable factor is the amount of cream one pours into the coffee. If we were to devise an experiment to determine how fast coffee cools when we add cream, we would make the amount of cream the independent variable (we can directly control the amount of cream to be added). The temperature of the coffee would then be the dependent variable (it is affected by how much cream we add).

Or consider an experimenter who is testing a new drug and wants to know how this drug affects the time a person takes to react to a particular stimulus. The independent variable is the amount of the drug he administers. The dependent variable is each subject's reaction time.

Independent variables are manipulated according to the purpose of the experiment. Often, as in the experiment to determine the effect of a drug, only one independent variable is used (i.e., the amount of the drug administered). However, it is possible to use more than one independent variable if the experiment is carefully designed. There are statistical procedures that separate the effects of several independent variables. But they are beyond the scope of this course.

As he manipulates the value of the independent variable, the experimenter observes, measures, and records the dependent variable. In psychology the dependent variable is usually some measure of a behavior anything from the number of errors made in a learning maze to changes in blood pressure.

ORGANIZING AND SUMMARIZING THE DATA

When submitting the results of our research for review by others, it is usually necessary to summarize the data in order to clarify its meaning and significance. In some cases, we may want to present the observations themselves; actual films of the birds at work may be useful in supporting conclusions about nest-building behavior. But, if we present only the raw observations, the reviewer has to duplicate all of the thought processes that led from the observations to the conclusions. Continuing with the nest-building example, the conclusion might have more credibility if we also present a summary of such observations is the frequency with which the birds left the nest to bring in new material, or the amount of time spent in building the nest.

A frequently used format of summarization is the tab/e. The results of the experiment to determine the effects of a particular drug on reaction time might appear as in Figure 1.

_
Figure 1. The effects of different dosages of drug X on reaction time
Amount of Drug GivenAverage Reaction Time of Subject
10 cc of drug X.40 sec
20 cc of drug X.58 sec.
30 cc of drug X.70 sec.

The same data could be displayed in a graph The graph is particularly useful when we are determining the effect of several levels, degrees, or amounts of an independent variable on the dependent variable. The conventional form of a graph requires that the independent variable form the horizontal axis and the dependent variable form the vertical axis. Thus we could graph the results of the drug experiment like this:
Figure 2.

We will describe many experiments in this course, and present the results in graphic form. In each case we will follow the convention of presenting the independent variable on the horizontal axis and the dependent variable on the vertical axis.

EXPERIMENTAL CONTROL

An experiment has an independent and a dependent variable. The experimenter must be sure that changes in the dependent variable are not caused by chance alone or by any factor except the changes he is making in the independent variable.

The control condition os one in which evary factor is identical to the experimental conditions except that the Independent variable is eliminated or held constant. If the experimenter wishes to test the effects of a drug then ha must give a control group of subject a 'placebo" to be sure that It Is the Independent variable that is producing the effect on the dependent variable

In the drug experiment there may be many factors which affect reaction time other than the amount of the drug given. Just a few factors which may confound the results are:

The purpose of a control condition is to ensure that only the change in the independent variable has influenced the result

To be certain that none of these factors will influence the results of our study, we control for them by making sure that each group is alike in every possible way except for the amount of drug given. An experiment is only as good as its controls.

One way to control for factors which may affect a dependent variable is to use a control group. In a control group, every relevant variable has exactly the same value as in the experimental situation, except the independent variable. For example, it has often been found that just taking a pill produced an effect, no matter what the pill contained. To make sure the drug experiment shows the effect of the drug, we might have a control group that is identical in every way with the experimental group except that it is given a sugar pill instead of the drug. The sugar pill is called a placebo because it is identical in appearance to the experimental drug, but it does not contain the active ingredients. Such a controlled experiment could be described as follows:
Administration
(lndependent Variable)
Observation
(Dependent Variable)
Experimental Group 1 10 cc of Drug x30 min. Iater record reaction times
Experimental Group 2 20 cc of Drug X30 min. Iater record reaction times
Experimental Group 3 30 cc of Drug X30 min. later reaction times
Control Group Placebo 30 mint Iater record reaction time.

You should note that this is a highly oversimplified description of an extremely complicated experiment. Experiments involving human behavior pose incredibly difficult problems just in determining what all the variables are that affect the dependent variable, much less making sure that all but one are held constant. In this experiment, for example, the experimenter would need to decide how long to wait between the time the drug was administered and when the reaction time test was given.

Often the validity of one's conclusion is suspect, more because of faulty experimental design than inaccuracies in the observations of the behavior. One reason psychologists have made so little progress in resolving the heredity vs. environment controversy is that no one has been able to demonstrate that all the relevant independent variables have been identified and controlled.

AN EXAMPLE OF EXPERIMENTAL METHODOLOGY

We have selected the subject of the "effectiveness of psychotherapy" to use as an example of the methodological considerations one must make in conducting research. Individual and group therapy have been practiced extensively for over half a century in a wide variety of forms, e.g., psychoanalysis, encounter groups, psychosurgery, behavior therapy, transactional analysis, and even electrical shock. And yet there are still highly conflicting claims as to the therapeutic effectiveness of any of the varieties now practiced.

But this choice of subject has certain dangers and your cooperation is necessary if we are to avoid losing sight of the teaching value of the example. First of all, the points made here do not show and are not intended to show that psychotherapy does or does not help people, and secondly, the methodological problems illustrated are not specific to psychotherapy. They apply equally well to other causal investigations such as the study of different learning methods, of the effect of religious beliefs on truthfulness, of astrological signs on sexual compatability, and so on.

Hypothesis Under Investigation

The hypothesis or claim to be investigated is that therapy produces significant improvement in people suffering from a variety of troubles, primarily those called neurotic conditions, but also including many psychosomatic disorders; such troubles include cases of sexual impotence, insomnia, psychologically caused paralysis or blindness, over aggressiveness, tics, agoraphobia, exhibitionism, compulsive cleanliness, or compulsive dirtiness. Some of these terms may be unfamiliar to you but you will probably be familiar with enough of them to think of some examples. The claim is that a recognizable and substantial proportion of such cases can be materially improved by therapy.

Designing the Experiment: Survey of Those Who Have Completed Therapy

If we want to discover whether therapy does any good, we might start by asking the people who have completed this kind of treatment whether it helped them. Let us suppose that such a study has been done, and that the response was unanimously favorable; each person asked was enthusiastically in favor of the treatment, believing it to have been of great benefit. Given such evidence, we might be convinced of the effectiveness of therapy. If we did accept this evidence as conclusive, however, we might be making a mistake. So far we have not even defined what we mean by "improvement." Also, the people surveyed might have lied; many people are reluctant to admit that they spent time, effort and money for something from which they received little or no value. Thus, we must phrase our questions with great care in order to get meaningful answers and identify any biases in the responses.

The Necessity for a Control Group

Also, even assuming there was significant improvement, the individuals we survey may have improved for some reason other than the therapy. Neurotic conditions often result in symptoms so bizarre that the person creates many changes in his own environment. Since, in many types of therapy, treatment often continues for several years, it is quite possible that a patient's troubles might have cleared up by that time without therapy. Perhaps his difficulties finally led the patient to "take himself in hand," or to change his mode of life in some way that produced a cure, e.g., by marriage or divorce. It would be absurd to pretend that we can find the answer to this question just by thinking about it. We must discover how people with similar troubles do without receiving any therapy.

Notice that the therapist is in no better position than his patient to contribute useful data for this investigation. He only has experience with those he treats; his own experience does not enable him to make comparisons with people who have had no treatment. He cannot tell from such an experience whether he produces the cure or whether his treatment merely accompanies the cure. Of course, he is probably responsible for some changes in the patients (e.g., in the terms they use when they talk about their troubles) but then the question remains whether those changes actually caused any improvement or merely happened at the same time. There is no way of telling, unless we can say how well such people would have done without any treatment, which clearly indicates a need to make some observations of a control group of untreated subjects. If the members of the untreated group improved just as much without benefit of therapy, then the therapy has negligible merit.

As we saw before, the comparison group used in such cases is called the control group, and the group receiving the treatment which the experiment is designed to investigate is called the experimental group. The purpose of this study is to identify the variation between the two groups in terms of the well-being of the individuals concerned. The factor of psychological health is the dependent variable in our study; it is presumed to be dependent on the application of therapy-the independent variable. We might achieve experimental control by providing therapy to the experimental group and withholding it from the control group, or by reviewing the records of large numbers of people who had, and had not, received therapy in the past. Either way, it would be difficult to hold all other independent variables constant.

It should be clear that virtually every assertion about the effect of any influence, treatment, teaching method, drug indeed, any claim that one circumstance causes another be supported only by evidence which includes the results of observations under controlled conditions. The demand for such controlled experiments is not narrow minded or stultifying, but a demand for the only kind of evidence which can possibly establish the credibility of such claims.

NOW TAKE THE Progress Check for Module 3

August 31, 2001