Given the growing body of evidence demonstrating the significant implications of health literacy on a myriad of outcomes, researchers continue to incorporate health literacy metrics in studies. With this proliferation in measurement of health literacy in research, it has become increasingly important to understand how various health literacy tools perform in specific populations. Our objective was to compare the performance of two widely used tests, the Short Test of Functional Health Literacy in Adults (S-TOFHLA) and the Newest Vital Sign (NVS) among and between a sample of English and Spanish-speaking patients. Adults (N = 402) ages 50-75 years participating in a trial to promote colorectal cancer screening completed in-person interviews which included both measures of health literacy. In the full sample, the tests were moderately correlated (r = 0.69, p <.0001); however, there was a stronger correlation among those completing the test in Spanish (r = 0.83) as compared with English (r = 0.58, p <.0001). English speakers more often were categorized as having adequate literacy by the S-TOFHLA as compared with the NVS, whereas Spanish speakers scored consistently low on both instruments. These findings indicate that the categorization of participants into levels of literacy is likely to vary, depending on whether the NVS or S-TOFHLA is used for assessment, a factor which researchers should be aware of when selecting literacy assessments.