The Duffy and Kabance study consisted of four experiments that examined the effects of changing only word and sentence length on comprehension. It used a "reading to do" task and a "reading to learn" task. The study used four versions of the text:

  1. The original version (a narrative or expository passage from the 1973 Nelson-Denny reading tests).
  2. One with vocabulary that they simplified using The Living Word Vocabulary
  3. One with only simplified and shortened sentences.
  4. One with both vocabulary and sentences simplified.

The effect was a 6-grade drop in reading level of the changed passages from the 11th to the 5th grade.

Following Klare's research protocols (1976), they attempted to maximize the readability effects by using readers who were low motivated, unfamiliar with the topic, and widely varying in reading skills.

Using the Nelson-Denny reading tests, they tested the reading ability of the 1,169 subjects, male Navy trainees between 17 and 20 years old, of which 80% were high-school graduates. They divided them into two groups, one with a median reading grade of 8.7 and the other 10.3. The experiments took place in groups of 40 to 70.

In the first two experiments, they simulated a "reading-to-do" situation. In the first experiment, they first showed the questions, then had the subjects read the text. After that, they were shown the questions again, which they answered. In the second experiment, they limited the reading time but let the subjects have access to the text while answering the questions. The third experiment was a standard cloze test. The fourth experiment was a standard multiple-choice test with the subjects first reading the text and then answering the questions without the text.

The first three experiments showed no significant improvements. The fourth experiment resulted in significant improvement but only with the low-ability group using the changed-vocabulary text, an improvement of 13 percent. The authors concluded that simplifying the text made no difference to the advanced readers. This is not a surprising result, when we consider the reading ability of the advance group was at grade 10.3 while the difficult text was at 11th grade.

The vocabulary variable is significant for the low-ability group, they stated, but only in reading-to-learn tasks but not reading-to-do tasks, where memory is less important. This correlation was also suggested by Fass and Schumacher (1978).

Duffy and Kabance concluded that the increased readability is not required for technical documents, in which the emphasis is on "reading to do" and memory is not required.

This is sometimes true. At other times, serious errors have taken place because of memory failure. Many, if not most, technical tasks involve learning a skill that can be repeated, as Redish (1988) emphasizes. Besides reading-to-learn and reading-to-do tasks, she writes, many technical tasks require "reading to learn to do." Technical texts may require more memory than do most other kinds of literature such as magazines, newspapers, or fiction.

When we look at the methods of these experiments, difficulties appear that explain their inconsistent results. In their report, Duffy and Kabance provide four sample passages used in the study. The re-written passages appear disjointed and stilted, not what one would expect of a 5th-grade text (See Fig. 11). If these studies are representative of the other passages, we must assume that judges were not used to control for the coherence and content of the text.