Welcome to Allen Conferences, Inc.’s web page,
The official web site of Allen Cognitive Levels.
Summary of ACL Research
If you are a graduate student and are doing a project or thesis on the Cognitive levels, Claudia would love to hear from you. If you are not sure where to start or what to work on, please call Allen Conferences we will be extremely pleased to work with you. We can inform you of studies done or being done on the Allen’s Cognitive Levels. We will be happy to publish your paper and put it on the web site. If you have already finished a thesis, we would love to have a copy of it. We would like to put it on the web. The office phone number is 800-853-2472.
Claudia Kay Hoover Allen, MA, OTR, FAOTA
This is an extract from the book Cognition and Occupation in Rehabilitation, edited by Naomi Katz, published by AOTA in 1998. The chapter from which this is extracted is entitled “Cognitive Disabilities: How to Make Clinical Judgments”.
A lot of work has been done to improve the credibility of the components of the Allen Battery, with the most studies done on several versions of the screening tools, the ACLS (Allen Cognitive Level Screen) and the LACLS (Larger Allen Cognitive Level Screen). Most of the quantitative studies have used correlations to establish the strength of the association between measures. The strongest correlation is reported as r = + or – 1.0. For reliability, the r number should be higher than + or- .8. The r for validity is usually lower, between .3 and .7. A small p represents the statistical significance, with the customary minimum at .05, meaning that the result is incorrect 5 out of 100 times. Lower p numbers are better, reducing the number of incorrect results. N is the number in the sample; bigger Ns increase the confidence in the study.
Interrater reliability has always been high. The original six cognitive levels were used in the first studies that found nearly perfect interrater reliability (r = .99, n = 32, range of levels 2-6). At that time the ACLS was used to place clients in a group that matched their score; the predictive validity was r = .76, N = 23 ( Moore, 1978). Newman (1989) examined the next version of the ACLS and found a percentage of agreement between two raters of 95.2% ( N= 21). Test-Retest reliability for Newman’s sample of chronic schizophrenic patients was r = .75 (N = 22, p.<.01). Partida looked at the interrater reliability of the large and small ACL with a small sample and two raters, finding perfect reliability (N = 4, r = 1.0, p = 0.0). Howell (1993) found similar interrater reliability with the most recent version of the small ACLS (r = .91, p.<.0001, N = 20). Eight raters trained by Penny, Messer, & North (1995) achieved an impressive r = .98. The first step in establishing a credible use of the Allen Battery is to be sure that the rating of the screening tool is accurate. These studies suggest that therapists and their students should be able to establish accurate interrater reliability.
Kehrberg (1992) reported significant correlations between the large leather lacing and the small one, with higher associations with the senile dementia population ( r = .95, p<.001) than the control subjects (r = .58, P < .001). The client population was more impaired when for gender and test order were controlled. With both sizes, both the dementia population and the controls over the age of 75 did not do as well as those under 75.
Validity is the extent to which a test measures what it says it measures, in this case cognitive processes, global ability to function, and severity of a disability. The ACLS has been correlated with well known instruments commonly used with a variety of diagnostic categories to check the validity of the scale. The cognitive measure that is most widely used is the Wechlser Adult Intelligence Scale (WAIS), which is often reported as verbal, performance and full scale intelligence (IQ). Katz (1979) found a moderate correlation between the Block Design and the ACLS (r = .45, p , .001) Mayer (1988) used all of the subtests of the WAIS to clarify the type of information processing that is related to adaptation. The strongest correlations were between the ACLS and the Block Design and Object Assembly ( r = .729, p< .0001). Performance IQ also showed a high correlation with the ACLS (r = .55, p < .0003). The most enlightening use of the WAIS was to divide the test into crystallized and fluid abilities. Fluid abilities “subsume information processing functions such as attention, perception, flexibility, and problem-solving. Crystallized abilities are dependent on previous training, education, and acculturation (e.g. vocabulary)” (Mayer, 1988, p. 176). The correlations with fluid abilities gave credence to the notion the a pattern of performance in adapting to a changing environment was being tested. No significant correlations were found with vocabulary, arithmetic, or picture completion, which are part of crystallized intelligence that is influenced by cultural background. This has been further supported by a lack of significant correlations with age, sex, education, occupation, work history, or socioeconomic status in several disabled populations. While these demographic factors do influence intelligence in normal populations, the benefits of cultural experiences do not overcome difficulties in processing the information necessary for planning and problem-solving with new information (Alsberg, 1987, Averbuch & Katz, 1988, Breeding, 1993, Camp & Person, 1987, David & Riley. 1990, Gokey, 1987, Heinmann, Allen, & Yerxa, 1989, Herzig, 1978, Heying, 1983, Howell, 1993, Josman & Katz, 1991, Kaeser, 1992, Katz, 1979, Katz, Josman, & Steinmetz, 1988, Mayer, 1988, Moore, 1983,. Newman, 1987, Partida, 1993, Penny, Musser, & North, 1995, Richards, 1983, Shapiro, 1992, Skinner, Denton, & Levy, 1989, Williams, 1981, Wilson, Allen, McCormack, & Burton, 1989).
Shapiro’s (1992) finding that leather lacing did not correlate with the Perceptual Memory Task (PMT), but did correlate with Visual Motor Integration (VMI) was surprising. A deeper look at underlying mental processes was required. The input of PMT is similar to the Block Design of the WAIS, using colored blocks and designs of increasingly complex designs printed on cards. With the PMT, subjects are given cards for 10 seconds, and then the designs are removed. To match or recognize a pattern, subjects must store an image of the design. The formation and storage of images is not required with leather lacing. With leather lacing people are given examples of stitches to copy or match. With the VMI, subjects are also given geometric patterns to copy. Shapiro’s study suggests a need to look beyond the input of visual and auditory cues and consider the underlying mental processes that must be applied to the cues.
The relationship between leather lacing and hand dexterity has been investigated, using the Purdue Pegboard, without a significant correlation as expected. With a sample of depressed patients the time to complete the ACL was used to successfully remove the ceiling effect on the ACLS, which has been a problem with other studies of depression. (Carmel, Katz, & Modai, 1996). When the major problem is slowness, or loss of energy, timing the ACLS may provide objective data.
The relationship between leather lacing and verbal abilities has also been studied. David and Riley (1990) found modest correlations between the ACLS and the Shipley Institute of Living Scale, which is a paper and pencil test for choosing synonyms and writing responses to a sequence of terms. With an N of 57, modest correlations were found with vocabulary r = .25, p,.02; abstraction, r =.35, p, .001, and IQ r = .31, p,.005. A stronger correlation was found with the Symbol Digit Modalities Test, which times the translation of novel geometric shapes in to written responses. (r = .52, p< .001). As suspected, leather lacing is more related to novel learning of perceptual motor tasks than to abstract reasoning and verbal abilities. A similar modest correlation was found between leather lacing and the Social Interaction Test (SIT) (Penny, Musser, and North, 1995). The total SIT correlation was r = -.32, p<.01; nonverbal r = -.27, p,.03; conversation r = -.32, p<.01, and voice quality was not significant r = -.16. A low ACLS score was associated with reduced social competencies. The scales go the opposite directions which explains the expected negative correlations. While there is a relationship between what people say and what people do, the relationship is modest.
In an effort to examine the cognitive abilities that emerge during Piaget’s period of concrete operations, the Riska Object Classification (ROC) was developed with interrater reliability at r – .83, P < .01. The structured part of this test required to great a leap in cognitive abilities and has not produced significant correlations. The spontaneous portion has been more productive, with a significant association with leather lacing, education, occupation, and social position in a non-disabled population (Williams, 1991). Wilson also found a significant correlation between the spontaneous portion of the ROC and leather lacing (r = .66, p<.001) and the Mini Mental Status Exam in her population with senile dementia (r= .90, p< .001). In a study of depressed people, Katz ( 1979) found significant correlations with leather lacing (r = .42, p<.001) and the Block Design portion of the WAIS (r=.60, p<.0001). The results of the early studies with the ROC were uneven, suggesting that the developmental continuum needed to be replaced with a different way of organizing our understanding of cognitive abilities. Katz began to work on this by including a series of class inclusion questions in the Lowenstein Occupational Therapy Assessment Battery (Katz, Josman, Steinmetz, 1988; Averbuch & Katz, 1988; Landsmann & Katz, 1988; Katz, et. al. 1988; Ceramic et. A. 1995). The convenient explanation of regression to lower levels of development, in a sequence taken from normal children, was not working. The differences between the way disabled people think, as distinguished from normal children and adults, needed clearer explanations.
Some differences between a psychological view of intelligence and occupational therapy view of function are emerging from these studies. Psychological tests of intelligence tend to include words, concepts, paper and pencil tasks, and even when visual cues are used they must be processed and stored in a sequential manner, within a timed period. As a whole, psychological tests have a strong tradition in favoring the left hemisphere abilities. Leather lacing, crafts and ADLs probably favor the right hemisphere’s contribution to visual spatial abilities. Visual spatial cues are demonstrated and copied while doing an activity. Learning may be instantaneous or acquired through practice in doing the activity. Right hemisphere processes prepare people for immediate action, containing spontaneity not seen in left hemisphere processes. Shapiro’s (1992) discussion of different responses to the tests she used provides insight into these differences. Because both hemispheres are important, an effort has been made to include both types of ability in the description of the modes to provide a global view of ability to function (Allen, Earhart, & Blue, 1992).
The idea that the scale measured the severity of a disability was first investigated with the Brief Psychiatric Rating Scale; the concurrent validity with the ACLS was r = .53 at admission and r = .43 at discharge ( Moore, 1983) In another sample with a schizophrenic population, Newman (1987) found a correlation with the Global Assessment Scale (r = .46, p,.01, N = 34). Differences between schizophrenic, depressed and control populations were found in California, and replicated in Israel ( Williams, 1981, Katz, 1979 , Katz & Heinmann, 1990). Heyings (1985) expanded the investigation the ACLS to senile dementia, finding a correlation with the Mini-Mental Status Exam (r. = .66, p< .001, N = 33). The results were replicated by Wilson (1985) with an N of 20 and an r = .59. The Cognitive Performance Test has been used to predict institutionalization in low functioning patients and trace the functional decline and death in years for people with senile dementia (Burns, 1992). At the other end of the age range, emotionally disturbed boys showed decreased ability to do leather lacing in a school in New York (N = 24). A moderate correlation with Developmental Test for Visuomotor Impairment (r = .44, p < .04) was found on a test design to screen children for neurological impairments and learning disabilities. (Shapiro, 1992). Breeding (1993) found a difference between age groups in a study of normal 6 to 9 year olds from a school with parents in high socioeconomic status. ( N = 84). The ACLS has been used to differentiate between adolescents, aged 12 to 18, with psychiatric disorders (N = 94) and controls (N = 29) matched for age, sex, education and place of residence (Katz, Josman, and Steinmetz, 1988; Josman & Katz, 1991). In non-disabled adults aged 18 to 65, Williams (1981) found a positive correlation between ACLS and social position based on education, social and occupational status as determined by the Hollinghead’s Two-Factor Index of Social Position. Cultural factors influence cognitive ability in normal populations but have not had an impact on the disabled scores. The cognitive levels do correlate with well known measures of the severity of a variety of mental disorders.
The relationship between activities that therapists typically use in practice and leather lacing has also been examined. Newman (1989) found a correlation ( r = .63, p,.01 with the Task Oriented Assessment of the Bay Area Assessment of Functional Performance. The study of activities of daily living was started by Heying’s ( 1983) finding of a high correlation (r = 82, p< .001 ) with the Physical Self-Maintenance Scale and Instrumental Activities of Daily Living Scale presented by Lawton and Brody (1969). Lawton and Brody’s activities of daily living were modified and turned into a quantitative measure by Heimann ( 1985), in the form of the Routine Task Inventory (RTI). Heimann studied psychiatric outpatients, finding an ACLS correlation with the total RTI of r = .64, p< .001 (Heimann, Allen, & Yerxa, 1989). The test-retest reliability of the RTI was established at r= .999, p< . 0001 and interrater reliability was established at r = .99, p, .001 (Heimann, 1995). Wilson found similar test retest reliability after a two week interval (r = .99, p< .0001). Wilson ( 1985) described a community based sample of people with senile dementia, finding an ACLS correlation with the RTI similar to Heimann’s at r = .56, p< . 01 (Wilson, Allen, McCormack, & Burton, 1989). Gokey (1987) found a modest correlation between leather lacing and the RTI ( r = .44, p < .006). The strength of the correlation between leather lacing and activities of daily living was given an in depth investigation by Gokey (1987), Heinmann (1985), and Wilson (1985), leading to the suggestion that eating is one level lower than all other activities. Gokey (1987) found that the RTI had a stronger correlation with working than leather lacing did in a sample of schizophrenic patients. By comparing the RTI correlations with leather lacing and classification abilities, a distinction between motor and verbal abilities began to emerge. (Heimann, 1985; Wilson, 1985; Gokey (1987) . The safety concerns implicit in the RTI received their first explicit investigation by Alsberg (1987) finding an association between leather lacing and the errors made while making macaroni and cheese in a sample functioning at ACL 4 and 5. Burns (1992) translated 6 tasks from the RTI into a Cognitive Performance Task, finding a correlation with the Mini-Mental Status Exam (r= .67, N = 77). At 4 weeks with the CPT, interrater reliability was r = .91 (N = 18) and test-retest was r = .89 (N = 36). The strength of the validity correlations were not great, suggesting that prior experience, motivation, social situations and underlying mental processes can have important influences on ADLs.
The investigation of craft activities was initiated by Kaeser (1992), showing improved performance when activities are matched to the cognitive level. The sample was elderly persons with dementia doing tiling tasks. Subjects at cognitive level 3 performed better on level 3 tiling activities than on a level 4 tiling activities (F (1,14) = 125, p<.001). As expected there was so significance difference in the way the level 4 subjects performed on the level 3 and 4 activities.
The first controlled research investigation of treatment effectiveness has been completed in Israel with two groups of schizophrenics in post-acute care. The research group (N = 11)was given activities from the ADM that matched their ability to function and probed for higher abilities. The control group (N = 8) was in a sheltered workshop with tasks given according to the work to be done. Both groups showed a significant improvement on the RTI (z = 2.80, p<.005 for research; z = 2.52, p< ,01 for control), but the research group showed a higher gain. The research group showed a significant improvement on the ACLS (z = 2.52. P< .01) but there was no significant improvement in the control group (z = .13). While the sample size is very small, the study supports the idea that greater gains may be made when the therapist’s treatment methods match the capacities of the client ( Raweh, 1996).
These quantitative studies have shaped the understanding of the psychological mechanism that explain ability to function. Confidence in using the cognitive levels to describe the severity of the disability in varied populations has been enhanced. Respect for what the person can do, will do, and may do has been heightened because what the person can do only explains a limited percentage of the variance. With the disabled populations, the overwhelming conclusion is that cultural background, as studied with the demographic variables, does not compensate for the severity of a cognitive disability.