This study examined the Tejas LEE or El Inventario de Lectura en Español de Tejas (Grade 1) to determine if a relationship existed between reading comprehension in Spanish and the tested skills on the diagnostic assessment. This quantitative research design evaluated the psychometric characteristics of the Tejas LEE and followed customary practices in research for validity and reliability studies. It analyzed the content, concurrent, predictive, construct and consequential validity of the test. The data collected included 700 students in 46 first grade bilingual classrooms from 7 different schools in Kind ISD (a pseudonym), an urban school district in South Texas. The data used for the analysis and the interpretation included beginning of the year (BOY) and end of the year (EOY) results at the item level for each student. Findings produced no evidence to support the claims that performance on sub-skills would result in reading comprehension. Items showing inadequate levels of difficulty (p > .80) were eliminated in order to focus attention on the remaining items, so that some important insight was noted in subsequent testing. For this reason, most BOY items for letter identification, syllable blending and segmenting and phoneme blending and segmenting were eliminated. Regression analyses indicated that a very low variance (6.1%) accounted for reliability for reading comprehension. Of the 59 original items, four (4) items remained. Findings support the need to use valid and reliable diagnostic reading assessments for Spanish reading. The results also support the need to create awareness in all stakeholders as they reflect on the financial, educational and social consequences of using assessment instruments that do not measure what they purport to measure.