The Bilingual Verbal Ability Tests (BVAT):
A Breakthrough in Bilingual Assessment - or Is It?
"Communiqué" (National Association of School Psychologists), 27 (6), 26-27.
Bilingual psychological assessment is a growing and controversial field of contemporary school psychology. The complexity of the issue of bilingual assessment includes the lack of thoughtful theoretical underpinning, poor operational (practical) conceptualization of what is to be measured, and nearly empty psychological "arsenals" of tools and procedures. In terms of the latter, it was Sattler's (1992) observation that few, if any, instruments for bilingual assessment meet acceptable psychometric standards (p. 576). "Little progress has been made in terms of developing valid and reliable assessment measures for the assessment of LEP and bilingual children." - lamented Lopez (1997) a few years later (p. 513). Indeed, only a few instruments (mostly for research purposes, e.g. Lambert's "Interlingual Verbal Flexibility Test", 1966) were developed for the specific purpose of assessing bilingualism per se; most measures applied to bilingual students were constructed for monolingual individuals. "This model of assessment uses the monolingual speaker as a yardstick for assessing bilingual competence (Mohanty & Perragaux, 1996, p. 217). However, there is scientific and empirical evidence that the bilingual is more than the sum of two monolinguals in one person and that his/her psychological "profile' indeed displays unique characteristics (Blanc & Hamers, 1989). In each bilingual individual there is a dynamic relationship between competencies of two languages, therefore, cognitive skills and conceptual knowledge that a given child possesses may be more functional in one language than in the other. The cornerstone of an assessment of bilingual students should be the determination of their total language proficiency (Valdez & Figueroa, 1994). Application of language proficiency data to the assessment of cognitive and social skills should take into consideration that the processing of verbal information is affected by the child's level of mastery of the language in which the intelligence test was administered or behavior regulation was attempted. The above considerations, along with Cummins' observation about "common underlying proficiency" and his conceptualization of bilingual language development (Cummins, 1984, 1996), laid down the theoretical background of the test under review, namely: the Bilingual Verbal Ability Tests (BVAT).
BVAT at a glance
The test is designed to help in developing entry and exit criteria in bilingual education, to facilitate appropriate program placement and planning, and to assess a bilingual student's academic readiness.
The BVAT consists of the Comprehensive Manual, the Easel Test Book, different language-specific format test records, a computer floppy disk (either Window or Macintosh format) containing a scoring and report writing program and a training videotape.
The BVAT is comprised of three individually administered tests
The above subtests were drawn from the Woodcock-Johnson-Revised (1989) Cognitive Battery and translated into 15 different languages, presumably the most widely used languages in the USA: Arabic, Chinese (Simplified and Traditional), English, French, German, Haitian-Creole, Hindi, Italian, Japanese, Korean, Polish, Portuguese, Russian, Spanish, Turkish, and Vietnamese.
The best practice for administration of the BVAT is by a trained bilingual examiner. However, when the bilingual examiner is not available, the Manual suggests a training for a Primary/Ancillary Team (pp. 51-59), and defines the role, limits, and scope of functioning of the "ancillary" examiner (who is not a translator, just a person with mastery of the targeted language) insisting that a "primary" examiner remains at the testing site and bear full responsibility for the accuracy of the procedure, as well as scoring and interpretation of the results.
The procedure includes administration of all the subtests in the English language first. Each item failed in English is re-administered in the native language. If the child gets that item correct in his/her native language, it is added to the score for that subtest, The overall subtest score is based on the child's knowledge/reasoning skills using both languages, thus reflecting the very nature of the bilingual ability.
Time is up to 30 minutes and depends on a child's age and language proficiency.
There are two basic options for the BVAT interpretation: age-based or grade-based. In addition to standard scores, the relative proficiency index (the same indicator as the Relative Mastery Index in the WJ-R), and percentile ranks, the BVAT offers "instructional zones" index, five levels of English language proficiency (from Negligible through Very Limited, Limited, Fluent to Advanced), and aptitude/achievement discrepancies in relation to the WJ-R Achievement Tests. (The rationale for using the BVAT as an aptitude measure is based on the assumption that the BVAT score may be interpreted as "...the level of English language ability that would be demonstrated by the bilingual if all of that person's language abilities were available in English" (Manual, p. 3). All scoring is automated through the "Scoring and Reporting Program" software, which is a standard feature of the BVAT kit. The BVAT Comprehensive Manual contains the "Examiner Training and Practice Exercises". A training videotape, prepared by the publisher, accompanies the test.
According to the Comprehensive Manual, normative data for BVAT was obtained during WJ-R COG standardization (1988-89)
Reliability was calculated through the traditional (a split-half procedure corrected for length by the Spearman-Brown formula and resulted in the high .80s for the subtests and in the mid .90 for the clusters) and an innovative "alternate-form alternate-procedure" operation. Following a counterbalanced design (so-called "a two-way study"), the participants were tested in English and the gains were obtained by re-testing in Spanish (a translation). Then, the same participants were tested in Spanish (Bateria, 1996) and the gains were obtained in English (a translation). The Manual (p. 89) reports the median BVAT reliability for the 542 bilingual subjects as .84. Due to the lack of parallel forms in other languages the "two way study" with bilinguals was not carried out in any other language (Manual, pp. 85-86).
Content validity is crucial for a test translated into 15 different languages. The "comparability" of translation was attempted through an 8-step procedure described in the Manual on pp. 68-70. This appears as a very meticulous and thoughtful proceeding; still, native speakers of these languages may find inadequacies due to the enormous difficulty of the translation itself. Some of the 15 languages have deleted items (sometimes up to four items) because of the "untranslatable" nature of these items (Manual, Table 8-5, p.71). It is not clear what effect it may have on the scores obtained if they are based on norms that include all items. Another aspect of the content validity is the issue of "uneven" complexity (from a relatively easy naming task in Picture Vocabulary to a much more difficult verbal reasoning task in Verbal Analogies). The authors tried to minimize this difficulty by introducing the "cluster concept" which is based on the "convergence" of three different aspects of verbal ability: receptive language, expressive language and verbal reasoning. The concurrent validity study was attempted on the English/Spanish bilingual population using as criteria measures eight well known tests of verbal abilities (Manual, p.74, Table 8-10). The correlation coefficient was within the range of .7 to .9 on average. A predictive validity study of the BVAT and a number of achievement tests produced a correlation coefficient ranging from .65 to .85.(Manual, pp.70-79). Construct validity was reported within the .7 to .9 range.
Publication of the BVAT is a significant event in the field of bilingual assessment. The Publisher and the authors should be complimented for their bold effort to measure the total bilingual competency in bilingual individuals. We have to understand, however, that this is only a first step of a long journey: The BVAT measures only a part of the child's cognitive-academic language (e.g., literacy skills remain beyond the scope of the test) and the content of the test is too narrow to make any long-term instructional planning or an educational placement decision based on this test alone. Item-by-item translation in a particular language may not satisfy some bilingual (in this language) professionals due to the regional variations and possibility of different ways of translating certain items. Sometimes the English word has a meaning that corresponds to several words in the targeted language: the synonym/antonym subtest is particularly vulnerable to this situation. More important, however, is the fact that the level of difficulty may change significantly (from easy to difficult and vice versa) as the result of translation. Professionals should use particular caution with the Picture Vocabulary subtest, the weakest in the battery (in terms of its reflection of the cognitive skills and operations) and the most culturally biased. In my experience, some items in this subtest are too culture-specific (e.g. men panning gold) to be passed even by older and educationally advanced bilingual children. Hand-scoring is not available for this test: the Publisher assumes access to computers by all assessment personnel, which is not always the case. The computer generated report available for the BVAT software users should be used as a template only for a qualitative and individualized interpretation of a student's performance. Restrictions related to norming (see above) should also be taken into consideration while interpreting the results of this test.
The BVAT is the first attempt to create a standardized test for assessing the "common underlying proficiency" (Cummins) in utilizing cognitive-academic language. The BVAT goes beyond the existing practices of testing separately mastery of English and the child's first language. Probably for the first time in the history of psychometric measurements it provides a "holistic" overall estimate of a bilingual child's cognitive language mastery - an event long overdue in the field of bilingual assessment (Valdes & Fifueroa, 1997)l. What is new and important, the BVAT includes other languages, not only Spanish. The BVAT is educationally relevant because it measures the cognitive-academic aspect of language proficiency needed for learning in schools. In sum, the BVAT is an effective (although not perfect!) instrument in bilingual assessment.