The purpose of this study was to determine the effect of four different design layouts of the New York State elementary science learning standards on user processing time and preference. Three newly developed layouts contained the same information as the standards core curriculum. In this study, the layout of the core guide is referred to as Book. The layouts of the new documents are referred to as Chart, Map, and Tabloid based on the format used to convey content hierarchy information. Most notably, all the new layouts feature larger page sizes, color, page tabs, and an icon based navigation system (IBNS).

A convenience sample of 48 New York State educators representing three educator types (16 pre-service teachers, 16 in-service teachers, and 16 administrators) participated in the study. After completing timed tasks accurately, participants scored each layout based on preference. Educator type and layout were the independent variables, and process time and user preference were the dependent variables.

A two-factor experimental design with Educator Type as the between variable and with repeated measures on Layout, the within variable, showed a significant difference in process time for Educator Type and Layout. The main effect for Educator Type (F(2, 45) = 8.03, p <.001) was significant with an observed power of .94, and an effect size of .26. The pair-wise comparisons for process time showed that pre-service teachers (p = .02) and administrators (p =.009) completed the assigned tasks more quickly when compared to in-service teachers. The main effect for Layout (F(3, 135) = 4.47, p =.01) was also significant with an observed power of .80, and an effect size of .09. Pair-wise comparisons showed that the newly developed Chart (p = .019) and Map (p = .032) layouts reduced overall process time when compared to the existing state learning standards (Book). The Layout X Educator type interaction was not significant. The same two-factor experimental design on preference, showed the main effect for Layout (F(3, 135) = 28.43, p =.001) was significant. The observed power was 1.0, with an effect size of .39. Pair-wise comparisons for preference scores showed that the Chart (p = .001), Map (p = .001), and Tabloid (p = .001) were preferred over the Book layout. The Layout Type X Educator Type interaction and the main effect for Educator Type were not significant.

This study provides evidence that the newly developed design layouts improve usability (as measured by process time and preference scores) of the New York State elementary science learning standard documents. Features in the new layout design, such as the IBNS, may provide a foundation for a visual language and aid users in navigating standard documents across grade level and subject areas. Implications for the next generation of standard documents are presented.