EjSBS - The European Journal of Social & Behavioural Sciences

The European Journal of Social & Behavioural Sciences

Online ISSN: 2301-2218
European Publisher

Reliability and Validity of the Malay Version of the Brief Assessment of Cognition in Schizophrenia (BACS): Preliminary Results

Abstract

Cognitive impairment is a main feature of schizophrenia. Schizophrenia patients demonstrate impairments in several key dimensions of cognitions and these impairments are closely associated with functional outcome. The importance of cognitive assessment has been broadly accepted today; therefore, an easy administered test battery is needed by researchers and in daily clinical practice. The Brief Assessment of Cognition in Schizophrenia (BACS), a cognitive assessment tool has been validated internationally in English, French, German, Spanish, Japanese, and Italian. This preliminary study was performed to test the reliability and validity of the Brief Assessment of Cognition in Schizophrenia (BACS) as an assessment tool in a Malay- language version (BACS-M). Participants Data were collected from 26 outpatients with schizophrenia who had given written informed consent to participate in the research. Measurements Tests included in the BACS-M are as follows: verbal fluency, verbal memory list learning, token motor test, symbol coding, digit sequencing task, and Tower of London. Results The mean duration of completion for the BACS Malay version was 39.27 min (S.D. = 9.03 min). The Malay version of the BACS showed high test-retest reliability (ICC = 0.89). The BACS-M composite score was significantly correlated with all primary measures of the BACS-consecutive assessments. Thus, this preliminary study indicates that the BACS-M is a reliable and a practical scale to evaluate cognitive function in schizophrenia patients for Malay speaking patients.

Keywords: Brief Assessment of Cognition in Schizophrenia (BACS), Malay version, Cognitive function, Schizophrenia

Introduction

Cognitive impairment is a core feature of schizophrenia (Mohamed et al., 1999; Saykin et al., 1994). Assessment of cognitive function is an important step in evaluating patients with schizophrenia. Cognitive impairment also gives an impact on functional outcome (Green, 1996; Green et al., 2000) A few components of the cognition, especially verbal memory, working memory, motor speed, attention, executive functions and verbal fluency are correlated with poor functional abilities (Keefe, 1995; Harvey & Keefe, 1997; Heinrichs & Zakzanis, 1998; Saykin et al., 1991).

Many studies on the neurocognitive impact of the second generation of antipsychotic medications have been completed. While most of these studies have concluded that second generation antipsychotic improve neurocognitive function, the interpretation of these results has been challenged, because of the variable test batteries used in each study (Harvey & Keefe, 2001). Current test batteries differ widely in content, duration and procedures. Most of the neurocognitive assessments’ batteries used are long and complex, mostly adapted from clinical neuropsychology tests which assess the entire profile of neuropsychological strengths and weaknesses in individuals. Long hours are often required to administer these batteries.

A number of test batteries are currently available for the purpose of brief cognitive assessment. Several computerized batteries have been applied to schizophrenia samples, such as the Cambridge Neuropsychological Test Automated Battery (CANTAB) (Robbins et al., 1996), the CDR Cognitive Assessment System (Hunter et al., 1997), and the CogTest Battery (Cogtest, 2002). Another option is the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) (Gold et al., 1999; Wilk et al., 2004). The RBANS is capable of providing reliable and valid assessments of patients with schizophrenia for a wide variety of cognitive functions (Gold et al., 1999; Hobart et al., 1999; Wilk et al., 2002) even though it was originally developed as a screening measure for elderly subjects and therefore, favours the evaluation of cognitive domains in more severely impaired patients, and leaves out important measures such as motor, executive and working memory tasks, which are important targets for cognitive enhancement in schizophrenia.

The Brief Assessment of Cognition in Schizophrenia (BACS) (Keefe et al., 2004) on the other hand, preserves the desirable features of the RBANS, but overcomes its limits, as it was specifically designed for use in schizophrenia clinical trials. It meets all the required criteria such as brief administration and scoring time, portability, repeatability and availability of alternate forms. The BACS has been validated in English (Keefe et al., 2004), French (Bralet et al., 2007), German (Sachs et al., 2011), Spanish (Salgado et al., 2007), Japanese (Kaneda et al., 2007) and Italian (Anselmetti et al., 2008). It was designed as a short battery for detecting cognitive deficits in people with schizophrenia and measures verbal memory, working memory, verbal fluency, attention motor speed, and problem solving.

The BACS assesses the cognitive aspects that are most affected in these patients such as verbal memory (Verbal Memory List Learning), working memory (Digit Sequencing Task), speed of information processing (Symbol Coding), motor speed (Token Motor Task), executive functions (Tower of London) and verbal fluency (Category instances, supermarket items or animals, and the Controlled oral word association test, letters T and S). The reliability, validity and comparability of original forms of BACS have been established empirically (Keefe et al., 2004). The composite score has a high test-retest reliability in patients with schizophrenia and healthy controls (ICCs>.80). The composite score is strongly correlated to functional measures such as independent living skills (r+.45), performance-based assessments of performance of everyday living skills (r=.56), and interview-based assessments of cognition in patients with schizophrenia (r=.48) (Keefe et al., 2006). Thus, it shows that BACS has a clear functional significance. Hence, for its clinical application for the Malay speaking population, we have developed the Malay-language version of BACS: BACS-M) (Hilwa et al., unpubl. data, 2013). The aim of the present study was to test the reliability, validity and usefulness of the BACS-M for the evaluation of cognitive function in schizophrenia.

Subjects and methods

Subjects

The study included 26 patients diagnosed with schizophrenia (DSM -IV). Patients were outpatients of the Psychiatry Clinic, University Putra Malaysia (UPM) at the Kajang Hospital in Kajang, Selangor. The study was approved by the UPM ethics committee and all participants gave their written informed consent. Eighteen males and 8 females (mean age = 28.81, S.D.= 7.79) were included with the duration of illness of 3.81 years (S.D.= 3.42). Patients were recruited to meet the DSM-IV criteria for schizophrenia, and did not have history of brain trauma or current substance use disorder. There were no specific medication criteria for inclusion. The demographic characteristics of the patients are described in Table 1.

Table 1 - Demographics of the sample (N=26)
See Full Size >

Malay translation of the BACS

For the Malay adaptation, the two versions of the BACS (A and B) were first translated into Malay by a Malay clinical psychologist. These Malay versions were then back translated into English by a native English speaker who was blind to the original versions. These back-translated versions were then checked and approved by Dr Keefe (original author). Instructions and Verbal Memory Test items were translated and adapted to suit the Malay language. All other the stimulus items were preserved in the Malay adaptation except for the 2 letters used in the verbal fluency, which were changed for “T” and “S” to ‘J’ and ‘G’ which are more often used in the Malay language.

Assessment procedures

The two Malay versions of the BACS were tested on subjects by trained psychologists on two separate days, with duration of 2 weeks between assessments. Subjects were randomly assigned to a sequence of BACS -M versions A and B.

Verbal memory

2.3.1.1. List learning

Fifteen words were presented to patients and they were asked to recall as many words as possible. This procedure is repeated five times. The outcome measure is the total number of words recalled. There are two alternate forms.

Working memory

2.3.2.1. Digit sequencing task

Clusters of numbers in random orders of increasing length are presented to patients. They are required to tell the experimenter the numbers in order, from the lowest to highest. The outcome measure is the total number of correct items.

Motor speed

2.3.3.1. Token motor task

100 plastic tokens were given to patients and they were asked to place the tokens into a container as quickly as possible for 60s. The outcome measure is the total number of tokens placed in the container.

Verbal fluency

2.3.4.1. Category instances.

Patients were given 60 s to name as many words as possible within the category of animals. The outcome measure is the total number of animal names generated.

2.3.4.2. Controlled oral word association test.

In two separate tasks, patients are given 60s to generate as any words as possible that begin with the letter J and G.

Processing speed

2.3.5.1. Symbol coding

Patients wrote numerals 1-9 as matches to symbols on a response sheet as quickly as possible in 90s.

Executive functions

2.3.6.1. Tower of London

Patients look at two pictures simultaneously. Each picture shows three different - coloured balls (red, blue, and green) arranged on three pegs, with the balls in a unique arrangement in each picture. The patients are told about the rules in the task and asked to provide the total number of times the balls in one picture would have to be moved in order to make the arrangement of balls identical to that of the other opposing picture. The outcome measure is the number of trials on which the correct response is provided. There are two alternate forms.

Data analysis

The data analysis was conducted using Statistical Package for Social Sciences (SPSS) version 21.0. Descriptive statistics were used to report the patients in terms of demographic and clinical data. For all subtests and the composite score, test-retest reliability was calculated with intra-class correlations (ICCs) for both sessions. The relationship among the BACS-M measures was determined by calculating Pearson correlations among the scores. A composite score was calculated by averaging all z-scores of the six primary measures from the BACS-M.

Results

Administration time and completion rates

All 26 patients completed the Malay BACS in first and second visit without any interruption. The average duration of BACS in first visit was 44.12 min (S.D.= 8.49) whereby mean duration of BACS in second visit was slightly shorter 39.27 min (S.D.= 9.03).

Description of BACS data

Test-retest reliability

Table 2 lists the mean and standard deviation for all the measures from the BACS. The intra-class correlations (ICC) between performance from test visit 1 and test visit 2 for each measure are also included. The ICC results between the A and B versions of the BACS-M subtests for the entire sample were 0.89, and two out of six scores were greater than 0.80. Intercorrelations between the BACS subtests were strong in the whole sample (Table 2) and did not change when the samples were analyzed separately. The Pearson correlations between the first and second visit for each measure are given in Table 2. All BACS-M primary measures and the composite score were significantly correlated between two visits.

Table 2 - Mean performance and reliability coefficients (ICCs) of BACS-M tests in patients with schizophrenia
See Full Size >

Discussion

The results of the present study indicate that the BACS-M is a reliable tool to evaluate cognitive function in schizophrenia in Malaysia. The BACS-M required a mean of 39.27 min to complete, whereas the original BACS was required less than 35 minutes to be completed. The findings in this study are in line with the results using the original English version of the BACS (Keefe et al., 2004). The BACS-M showed good test-retest reliability ICCs of 0.89.

The results of the present study indicate that the BACS-M is a reliable tool to evaluate cognitive function in schizophrenia in Malaysia. The BACS-M required a mean of 39.27 min to complete, whereas the original BACS was required less than 35 minutes to be completed. The findings in this study are in line with the results using the original English version of the BACS (Keefe et al., 2004). The BACS-M showed good test-retest reliability ICCs of 0.89.

The Pearson correlation between the BACS-composite score and the primary measure indicated high correlations between each individual measure with the exception of the Verbal memory list learning (Verbal fluency). This result is slightly different from those reported in the previous studies. Specifically, in the validation of the French version of the BACS (Bralet et al., 2007), Token Motor Task (Motor speed) was reported to have lower correlation between each individual measures whereas in the Japanese version of the BACS (Kaneda et al., 2007), Symbol Coding (Processing speed) was reported to have lower correlation. In the present study, the Verbal memory list learning was found to be significantly lower. This may due to the nature of the Malay language considering most of the Malay words comprise more than one syllable. This itself may contribute to the difficulties in remembering the required words. Apart from that, the duration between first assessment and the second one is considered to be long, that is two weeks. In comparison to the previous validity studies, this is considered the longest duration which can affect the subjects’ practice effects. With regard to this, Bralet et al. (2007) in his discussion pointed out that the long duration between test 1 and test 2 could have reduced the practice effect amongst patients.

The composite score derived from each BACS-M primary measure was strongly correlated between two assessments, and showed a significant practice effect, as was noted in the original BACS study. Using an alternate form is recommended to reduce the practice effect as when different versions of test were administered, the composite score did not show a statistically significant practice effect. Apart from test-retest reliability, inter rater reliability for this instrument has been shown to be excellent (r=0.9 – 1.0). This suggests that BACS-M can be used by different clinicians or researchers in multicenters.

Conclusion

The present study indicates that the BACS-M is a promising useful tool with good test-retest reliability to assess the major constructs of cognitive function in schizophrenia patients. The Malay version of the BACS can be administered both in clinical and research setting for cognitive enhancement in patients with schizophrenia.

Acknowledgements

This work was supported by a grant Fundamental Research Grant Scheme (FRGS No. 5523878) from The Ministry of Higher Education, Malaysia. We thank the Universiti Putra Malaysia Department of Psychiatry staff nurses for their assistance in the recruitment of the patients at the Kajang Hospital. The author(s) declare that there is no conflict of interest.

References

  • Anselmetti, S., Poletti, S, Ermoli, E., Bechi, M., Cappa, S., Venneri, A., Smeraldi, E., & Cavallaro, R. (2008). The Brief Assessment of Cognition in Schizoprhenia. Normative data for the Italian population. Neurol Sci, 29, 85-92. DOI:

  • Bralet, M. C., Falissard, B., Neveu, X., Lucas-Ross, M., Eskenazi, A. M., & Keefe, R. S. E. (2007). Validation of the French version of the BACS (the brief assessment of cognition in schizophrenia) among 50 French schizophrenic patients. Eur Psychiatry, 22, 365 – 370. DOI:

  • Cogtest plc. Cogtest ™ (2002). Computerized Cognitive Battery for Clinical Trials. Retrieved from http://www.cogtest.com.

  • Green, M. F. (1996). What are the functional consequences of neurocognitive deficits in schizophrenia? Am. J. Psychiatry, 153, 321-330.

  • Green, M. F., Kern, R. S., Braff, D. L., Mintz, J. (2000). Neurocognitive deficits and functional outcome in schizophrenia: Are we measuring the ‘right stuff’? Schizophrenia Bulletin, 26, 119 – 136. DOI:

  • Gold, J.M., Queern, C., Iannone, V.V.M Buchanan, R.W. (1999). Repeatable battery for the assessment of neuropsychological status as a screening test in schizophrenia: I. Sensitivity, reliability, and validity. Am. J. Psychiatry, 156(12), 1944 – 1950.

  • Harvey, P. D, Keefe, R. S. E. (1997). Cognitive impairment in schizophrenia and implications atypical neuroleptic treatment. CNS Spectr, 2, 1-11. DOI:

  • Harvey, P. D., & Keefe, R. S. (2001). Studies of cognitive change in patients with schizophrenia following novel antipsychotic treatment. The American journal of psychiatry, 158(2), 176–184. DOI:

  • Heinrichs, R. W., & Zakzanis, K. K. (1998). Neurocogniive deficit in schizophrenia: a quantitative review of the evidence. Neuropsychology, 12(3), 426 – 445. DOI:

  • Hobart, M. P., Goldberg, R., Bartko, J. J., & Gold, J. M. (1999). Repeatable battery for the assessment of neuropsychological status as a screening test in schizophrenia: II. Convergent/discriminant validity and diagnostic group comparison. Am. J. Psychiatry, 156(12), 1951-1957.

  • Hunter, R., Cmeron, S., Perks, S., Wesnes, K. (1997). The cognitive profile of unmedicated schizophrenic patients in relation to controls. J. Psychopharmacol, 11, A74 (Suppl.).

  • Kaneda, Y., Sumiyoshi T., Keefe, R.S., Ishimoto, Y., Numata, S., Ohmori, T. (2007). Brief Assessment of Cognition in Schizophrenia: validation of the Japanese version. Psychiatry Clin Nuerosc, 61, 602-609. DOI:

  • Keefe, R.S.E. (1995). The contribution ofneuropsychology to psychiatry. Am. J. Psychiatry, 152, 6-15. DOI:

  • Keefe, R. S. E., Goldberg, T. E., Harvey, P. D., Gold, J. M., Poe, M. P., & Coghenour, L. (2004). The Brief Assessment of Cognition in Schizophrenia: reliability, sensibility, and comparison with a standard neurocognitive battery. Schizophrenia Research, 68, 283-297. DOI:

  • Mohamed, S., Paulsen, J. S., O’Leary, D., Arndt, S., & Andreasen, N. (1999). Generalized cognitive deficits in schizophrenia: A study of first- episode patients. Arch. Gen. Psychiatry, 56, 749-754. DOI:

  • Sachs, G., Winklbaur, B., Jagsch, R., & Keefe, R. S. E. (2011). Validation of the German Version of the Brief Assessment of Cognition in Schizophrenia (BACS) – Preliminary Results. European Psychiatry, 26, 74-77. DOI:

  • Saykin, A. J., Gur, R. C., Gur, R. E., Mozley, P. D., Mozley, L. H., Resnick, S. M. Et al. (1991). Neuropsychological function in schizophrenia. Selective impairment in memory and learning. Arch. Gen. Psychiatry, 48, 618-624. DOI:

  • Saykin, A. J., Shtasel, D. L., Gur, R. E. et al. (1994). Neuropsychological deficits in neuroleptic naïve patients with first -episode schizophrenia. Arch. Gen. Psychiatry, 51, 124-131. DOI:

  • Robbins, T. W., James, M., Owen, A. M., Sahakian, B. J., McInnes, L., & Rabbitt, P. M. (1996). A neural systems approach to the cognitive psychology of ageing: studies with CANTAB on a large sample of the normal elderly population. In P. K. Rabbitt (Ed.), Methodology of Frontal and Executive Function (pp. 215-238). Lawrence Erlbaum Associates, Hove.

  • Wilk, C. M., Gold, J. M., Bartko, J. J., Dickerson, F., Fenton, W. S., Knable, M., Randolph, C., & Buchanan, R. W. (2002). Test -retest stability of the repeatable battery for the assessment of neuropsychological status in schizophrenia. Am. J. Psychiatry, 59(5), 838-844. DOI:

Copyright information

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

About this article

Published online: 03.10.2013
Pages: 235-244
Publisher: Cognitive-crcs
In: Volume 5, Issue 2
DOI: 10.15405/ejsbs.66
Online ISSN: 2301-2218
Article Type: Original Research
Cite this article