Welcome to the fourth volume of the International Journal of Process Education! This issue provides results from significant classroom research conducted within the context of Process Education philosophy as well as practical tools for practitioners. The topics include creativity, faculty professional development, and course development with a focus on instruments for measuring and achieving quality teaching and learning.

• From the Editorial Board

  We begin the journal with four articles that discuss educational tools developed for measuring and achieving quality teaching and learning. Utschig, Newton, and Bryan discuss an instrument to measure the attributes represented in the profiles of a quality learner and a quality engineer. This work extends the usefulness of the profiles beyond a simple vision or goal, helping students by defining the critical characteristics they need to develop in order to become excellent learners and engineers. Burke, Ouellette, Miller, Leise, and Utschig examine the reliability of a newly developed version of the Academy of Process Educators’ Writing Rubric. The new rubric is designed to measure both fundamental writing skills as well as depth of disciplinary knowledge. The research seeks to determine whether the design of this rubric allows for measurement by raters across different disciplines as well as for different writing purposes. A rubric is also the subject the work of McCormack, Williams, Noren, Beyerlein, Cordon, and Morgan. These authors discuss a general purpose quantitative problem-solving rubric intended to promote reflective practice of analytical problem solving and increase solution documentation while standardizing accreditation data across courses and programs. Finally, Beyerlein, Burke, and Hintze provide a set of three concept maps that conveys important collections customized for instructional staff, rather than learners, to help them better visualize their personal teaching and learning practices as well as promote a supportive culture.

  continue reading...

• Measuring Skills across the Profile of a Quality Learner and of a Quality Engineer

  by Tristan T. Utschig, Sunni M. Newton, and Jeffrey S. Bryan

  Abstract

  We have adapted two previously published profiles to create instruments which measure the attributes represented in the profiles of a quality learner and of a quality engineer on a scale of 1-5. This work is important because it extends the usefulness of the profiles beyond a simple vision or goal state. First, we confirm that the instruments have high face and content validity. Then, we calculate internal reliability coefficients for each of the attributes based on data from approximately 200 students in two different engineering courses. Strong reliability (alpha > 0.7) was found for 12 of the 13 attributes. Next, we compute ranges, averages, and standard deviations among responses for each attribute and find reasonable discrimination among the population tested. We then test the criterion-related reliability of the instrument by correlating ratings for attributes to course and individual assignment grades. Six of the 13 attributes for these two profiles were found to have statistically significant correlations with student grades in the course where the instrument was employed, and numerous significant correlations were found among individual assignments with particular attributes. The attribute of an “achiever” in engineering was most strongly correlated with course and assignment grades. These results imply that quantitative data about student perceptions of skill across the profiles can now be collected and used for program, course, or activity design in order to better achieve learning outcomes and produce high quality graduates. In addition these two instruments can help define for students what critical characteristics they need to develop in order to become excellent learners and engineers. Further, we note that the identified attributes are qualities desirable in many fields. In particular, the instrument for quality engineers, though designed with engineers in mind, is applicable to many fields, needing only minor adjustments to suit the specific needs of the user.

  continue reading...

• Measuring Writing as a Representation of Disciplinary Knowledge

  by Kathleen Burke, Judith Ouellette, Wendy Miller, Cy Leise, and Tris Utschig

  Abstract

  Most students are exposed to different writing genres as well as to grammar and other writing-specific skills during their required composition course(s). Discipline-specific writing skills are often not taught, rather they are assumed to be prerequisite knowledge by disciplinary instructors. Rubrics are often used by instructors to measure these skills; however, few rubrics integrate measurement of both general and discipline-specific skills. One such rubric has been designed by the Academy of Process Educators. The present research examines the empirical reliability and validity of this rubric. It was hypothesized that consistency of ratings would be achieved for three different types of writing by four raters from different disciplines when read in a counterbalanced pattern. The Written Communication Value Rubric from the Association of American Colleges and Universities (AAC&U, 2012) was also utilized to make possible the evaluation of concurrent validity of the two rubrics. Different statistical methods were utilized to analyze the ratings. The research methodology utilized provides a model for empirically exploring the reliability and validity of other rubrics. Overall, our findings indicate that individually raters were consistent on both rubrics, but the inter-rater reliability was not strong. The ratings were less dispersed on the Academy rubric than on the AAC&U rubric but “concurrent” validity, that is significant positive correlations between the rubrics, were clearly present. Some of the implications of the study include the importance of encouraging careful rater and inter-rater training, using counter-balanced sequences to avoid bias from repetition, ensuring that writing assignments reflect clear performance criteria in regards to rubric performance areas, and assuring that writing pre-requisites are carefully integrated throughout the program’s curriculum and assessment strategy.

  continue reading...

• Development and Usability of a Generic Quantitative Problem-Solving Rubric for Student Learning

  by Jay McCormack, Barbara Williams, Ken Noren, Steve Beyerlein, Dan Cordon, and Jim Morgan

  Abstract

  An interdisciplinary group of faculty within colleges of engineering at the University of Idaho and Texas A&M University has developed a general purpose quantitative problem-solving rubric for students and young professionals. The rubric is intended to promote reflective practice of analytical problem solving, encourage more comprehensive solution documentation, provide formative assessment in conjunction with selected homework assignments, and enable collection of accreditation data across courses and programs. This article describes the development of an assessment package (problem-solving rubric combined with implementation tips and instructor resources) that is broadly applicable across multiple disciplines. The validity and usability of the assessment package is evaluated through a small-scale survey of faculty who teach mid-program courses involving multiple, calculation-based homework assignments. Based on survey input, the rubric was revised to incorporate better customization and the assessment package was expanded to include an example of scored student work as well as a facilitation plan for introducing students to the rubric.

  continue reading...

• Concept Maps for Linking Aspects in the Transformation of Education

  by Steven Beyerlein, Kathleen Burke, and Denna Hintze

  Abstract

  In previous IJPE scholarship, fourteen aspects were identified as mediators of quality in teaching/learning in higher education. These were derived from more than ten years of teaching institute observations, peer coaching sessions, and classroom self-assessments. The aspects highlight challenges that can be embraced to enrich educational experience, or ignored, to perpetuate the status quo. As a group, these aspects are overwhelming without a framework for addressing them within a larger instructional system. This work offers a set of three concept maps that cluster the aspects based on natural affinity as well as natural order of implementation. Each aspect was placed under one of four broad categories: epistemology of teaching/learning, course design for learning, facilitation for learning, and assessment for learning. Separate maps were then constructed using the aspects as well as other elements of classroom instruction to illustrate differences between traditional, transitional, and transformational teaching/learning environments. Once the maps were created, educator ways of being and common teaching and learning toolkits became obvious from each map. The set of concept maps is customized for instructors, rather than learners, to help them better visualize their personal teaching/learning practices and their local teaching/learning culture on the continuum from traditional to transformational environments. These new resources are expected to lay the foundation for a next-generation Faculty Guidebook resource as well as tools to measure educator performance and activities that promote professional growth within the three different teaching/learning environments.

  continue reading...

• Impact of Faculty Development on Classroom Accessibility as Measured Using a Classroom Observation Instrument

  by Tristan T. Utschig, Nathan W. Moon, Aaron Bozzorg, Robert L. Todd

  Abstract

  SciTrain University is a U.S. Department of Education-sponsored program to improve the accessibility of science, technology, engineering, and mathematics (STEM) education for students with disabilities. This paper builds on previous work describing more general issues regarding the efficacy of faculty in creating productive learning environments. Specifically, this new research explores how faculty who are exposed to principles of universal design for learning (UDL) in STEM courses change their observable behaviors in the classroom over time. Through persistent, targeted, personalized attention, these faculty gained experience and expertise in addressing the learning needs of all students, and in particular those with disabilities. We first very briefly highlight some of the main features of SciTrain University, including workshops and web course-modules used for faculty development. Then, we discuss the impact this development has produced in the classroom for a set of 18 longitudinal faculty participants as seen through the lens of a classroom observation instrument developed to assess classroom accessibility and inclusion for learners. Results indicate that accessibility scores increase, on average, about 5-7% per year. We then conclude by offering some recommendations for process educators, based on our research findings, that address ways to improve performance in making one’s classrooms more accessible and inclusive.

  continue reading...

• Enhancing a First-Year Success Course through Process Education

  by Rebecca Jones and Kimberly Kilgore

  Abstract

  Learning-to-Learn Camps have traditionally been offered to students who have been unsuccessful in an academic program. St. Louis College of Pharmacy (STLCOP) decided to use the same Process Education® concepts of the Learning-to-Learn Camps in its first-year success course for high-achieving freshman entering into its highly competitive pharmacy program. Assessments indicate that the first-year students consider the course to be engaging and that it increases their academic success. This paper will discuss which aspects of the course have been most valuable to students, how student feedback has changed over three years of implementation, and what changes STLCOP plans to make to the course in future years.

  continue reading...

• Importance of Psychomotor Development for Innovation and Creativity

  by Mohamed El-Sayed and Jacqueline El-Sayed

  Abstract

  Innovation and creativity have been on the top of desired learning outcomes for over a century. Many research efforts have been dedicated to creativity and creative behavior in education. Most of these research efforts have demonstrated and emphasized the importance of the development of the cognitive and affective domains in fostering creativity in educational settings. Minimum effort, however, has been directed toward the role that psychomotor domain skills play in the development of creativity. This paper discusses the importance of the development of the psychomotor domain in fostering creativity. To demonstrate this, the creative realization process is studied. The creative realization domains are redefined to identify the virtual and perceptual realities, in addition to physical reality, as valid domains of the realization process. Bloom’s learning domains and educational objectives are also discussed and aligned with the objectives of the realization process. When we consider the different reality domains, the importance of the psychomotor domain becomes clear for educational disciplines that are not limited to perceptual domain realization. To illustrate the concepts presented in the paper, several examples are included.

  continue reading...