Within the engineering education community, one of the more powerful mechanisms known to stimulate conversations about improvements in engineering education is to ask hard questions about

1. What students should be learning and
2. How well are they performing.

Recognizing that the Force Concept Inventory developed by Hestenes and Halloun has encouraged physicists to re-examine what students are learning in their introductory physics courses and how these courses might be restructured and taught differently, faculty members across the FC have been developing and sharing assessment instruments targeted at areas of interest to engineering educators. Work began on four concept inventory assessment instruments in October 2000 while work on three others began in October 2001. In addition, work continues on instruments designed to explore students' perceptions and attitudes and on a self-assessment instrument for teams.

Signals and Systems Concept Inventory

Linear signals and systems is a core subject in the undergraduate electrical and computer engineering curriculum. The Signals and Systems Concept Inventory (SSCI) is a twenty-five-question, multiple-choice exam designed to assess students' understanding of fundamental concepts in this subject. There are separate versions of the SSCI exam for continuous-time and discrete-time material. John Buck (University of Massachusetts Dartmouth) and Kathleen Wage (George Mason University) are the principal developers of the SSCI. More information is available on the SSCI website, which includes password-protected versions of the instruments that are available for distribution.

The continuous-time SSCI (CT-SSCI) is currently at version 2.0, and faculty members at University of Massachusetts Dartmouth, George Mason University, the U.S. Air Force Academy, and the U.S. Naval Academy are doing pre/post testing surveys in undergraduate signals and systems classes. Preliminary statistical analysis of the results obtained to date has been initiated to determine if scores can be correlated with gender, race, grade-point average, and common prerequisite courses. The analyses will look for race or gender biases and also will try to characterize which prior academic measures are the best predictors of exam performance and normalized gain.

The discrete-time SSCI (DT-SSCI) is in "beta" and undergoing preliminary testing at University of Massachusetts Dartmouth, George Mason University, and Massachusetts Institute of Technology. Based on the results of this testing, the DT-SSCI will be revised this summer for broader testing beginning in the fall of 2002.

In fall 2001, 97 students at GMU, UMD, and the USNA took the CT-SSCI as a pre-test and a post-test. Motivated by Hake's survey of the FCI (Hake, 1998), the normalized gain was computed for each student, as well as normalized gains for each course (based on the average pre-test and post-test scores for each course). Normalized gain represents the fraction of the available improvement in score that was obtained during the course. In analyzing the FCI, Hake showed that normalized gain is a stable performance measure for courses that have similar pedagogical formats, regardless of variations in student background or instructor experience. Analysis of the 97 SSCI exams revealed a normalized gain between pre- and post- test scores of 0.24 ± 0.08 which is consistent with the results Hake reported for other concept inventory studies of traditional lecture courses.

Conference Proceedings Papers

1. Wage, K.E., and Buck, J.R., "Development of the Signals and Systems Concept Inventory (SSCI) Assessment Instrument," Proceedings, 2001 Frontiers in Education Conference, Reno, NV, October 2001.
2. Wage, K.E., Buck, J.R., Welch, T.B., and Wright, C.H.G. (2002), "The Continuous-time Signals and Systems Concept Inventory," Proceedings, International Conference on Acoustics Speech and Signal Processing, Orlando, FL, May 2002
3. Wage, K.E., Buck, J.R., Welch, T.B., and Wright, C.H.G. (2002), "The Signals and Systems Concept Inventory," Proceedings, American Society of Engineering Education Annual Conference, Montreal, Quebec, June 2002.

Thermodynamics Concept Inventory

The Thermodynamics Concept Inventory (TCI) was developed by Donovan Evans (Arizona State University), Clark Midkiff (University of Alabama), and Thomas Litzinger (Pennsylvania State University) and has been modified through two semesters of testing. The thirty-three-question inventory requires approximately thirty minutes to complete. The first complete cycle of pre- and post-testing of the concept inventory will conclude at the end of the spring 2002 semester. Faculty members at the University of Alabama, Pennsylvania State University, Baylor University and San Jose State University have agreed to test the instrument in spring 2002. Faculty members at the Franklin W. Olin College of Engineering in Massachusetts and the Colorado School of Mines have evaluated or considered using the instrument. Highlights from early testing are that beginning students often fail to recognize or exploit situations of constant volume or constant pressure, and that students have a rudimentary grasp of First Law of Thermodynamics concepts but perform poorly when Second Law concepts are tested.

Thermodynamics concept testing was studied by a team at the University of Wisconsin Madison. The evaluation team consisted of Sandy Klein, Jay Martin, and John Mitchell. This team made slight modifications to the original and then gave the modified version to two thermodynamics classes (taught by Klein and Sanders) in the first two weeks of class. The TCI was given as a homework assignment, collected, and graded. The TCI was neither returned nor discussed. Evaluation of the pretest results follows.

Students took less than an hour to complete the inventory. Their performance shows that they do bring some knowledge of thermodynamics into the course. However, results also indicate that they do not fully grasp the concepts of energy balances and entropy (Second Law of Thermodynamics) at the beginning of the course. These results are not surprising. Student performance on the thermodynamics of chemical reactions was poor, however, and this result was surprising, since they have worked on these concepts in previous chemistry courses.

Conference Proceedings Papers

1. Midkiff, K.C., Litzinger, T.A., and Evans, D.L., "Development of Thermodynamics Concept Inventory Instruments," Proceedings, Frontiers in Education Conference, Reno, NV, October 2001.

Electromagnetics Concept Inventory

The Electromagnetics Concept Inventory (EMCI) is best suited for junior-level undergraduate electromagnetics (EM) courses in electrical engineering departments. It can be applied in a variety of undergraduate and graduate EM-related courses in engineering and physics departments. EMCI Version 1.0 contains three exams:

• EMCI-Fields (electro and magnetostatic, and time-varying EM fields),
• EMCI-Waves (uniform plane waves, transmission lines, waveguides, and antennas), and
• EMCI-Fields & Waves (a combination of the first two exams).

Branislav Notaros at the Colorado State University, the principal developer of the EMCI, will chair the session on electromagnetics education at the 2002 IEEE Antennas and Propagation Society International Symposium, to be held 16-21 June 2002 in San Antonio, Texas.

Conference Proceedings Papers

1. Notaros, B.M., "Concept Inventory Assessment Instruments for Electromagnetics Education," Proceedings, IEEE Antennas and Propagation Society International Symposium, San Antonio, Texas, 16-21 June 2002.

Strength of Materials Concept Inventory

The Strength of Materials Concept Inventory (SoMCI) assesses students' mastery of fundamental concepts in a sophomore mechanics of solids or strength-of-materials course. Concepts examined include stress, strain and deformations due to axial, bending, and torsional loads; failure of ductile and brittle materials, stress transformation, and axial buckling. Follow-on courses such as structural analysis (in civil engineering) or machine design (in mechanical engineering) build directly upon these concepts. Many students typically do not master some of the more abstract strength of materials concepts until completion of these follow-on courses. The principal developers are Jim Richardson (University of Alabama [UA]) and Jim Morgan (Texas A&M University [TAMU]). Testing began at UA and TAMU in summer 2001, and the SoMCI was available to other schools beginning in spring 2002.

Conference Proceedings Papers

1. Richardson, J., and Morgan, J., "Development of an Engineering Strength of Material Concept Inventory Assessment Instrument," Proceedings, Frontiers in Education Conference, Reno, Nevada, 10-13 October 2001.

Student Perceptions and Attitudes
Student perceptions and attitudes regarding educational experiences have been shown to contribute significantly to the retention of students in undergraduate science, math, engineering, and technology (SMET) programs. For example, studies conducted at Texas A&M suggested that female students as a group had lower retention, despite higher incoming preparation and higher performance, when compared to male students. Findings of this nature have also been reported elsewhere, supporting the assumption that student perceptions do help drive persistence and may be based upon issues independent of a student's academic preparation and grade performance.

Student persistence in SMET disciplines is a matter of national concern in the context of contemporary life, which is increasingly affected by scientific, mathematical, and technological innovations. Hence, generating and sustaining positive attitudes and appropriate perceptions about SMET disciplines and about learning experiences in SMET programs has become a matter of great importance among academicians. The measurement of student perceptions and attitudes has therefore become a focus of research in a variety of academic programs.

Self Assessment Instrument for Teams

The assessment team at University of Massachusetts Dartmouth revised and tested the FC team process check (TPC) and developed a team knowledge test (TKT), both of which can be administered either on paper or on-line.
The TPC provides a self-report assessment of a team's functioning and is intended to be administered periodically during the life of a team. The measure has demonstrated good internal consistency and has also yielded two meaningful factors. The first factor appears to assess what may be termed the team's sense of agency or ability to get the job done well. The second factor is an affiliative or interpersonal factor, including communication and conflict resolution. The TPC was found to be correlated with faculty ratings of the teams, and partial support was found for its capacity to predict outcome as measured by team project scores.

The TKT is the first draft of a measure intended to assess individual team members' general knowledge of team issues and concepts. The development team has used it as a pre/post measure of team knowledge over a semester's team experience as well as before and after the use of the team training material that was posted on the Web site and was assigned by faculty members as part of their courses. Additional revision will be required to improve the scale and to establish its reliability and validity. The group also developed two Web sites, a faculty guide and a student team training site, to facilitate the assessments that were performed and the studies that were conducted. Ultimately, these Web sites will be incorporated into an ongoing continuous improvement process of team assessment and training. The team has also begun to disseminate the tools and the assessment process by reaching out to several other campuses within the FC to examine the materials and consider their use.

Conference Proceedings Papers

1. Powers, T.A., Sims-Knight, J., Topciu, R.A., Haden, S.C., "Assessing Team Functioning in Engineering Education," Proceedings, American Society of Engineering Education Annual Conference, Montreal, Quebec, June 2002
2. Sims-Knight, J., Upchurch, R.L., Powers, T.A., Haden, S.C. and Topciu, R.A.., "Teams in Software Engineering Education." submitted to the 2002 Frontiers In Engineering Conference

Materials Concepts Inventory

Stephen Krause (Arizona State University) and Richard Griffin (Texas A&M University) have been developing a Materials Concepts Inventory (MCI) during the past six months. Graduate and undergraduate students majoring in materials have been employed in developing questions and identifying misconceptions. Undergraduate engineering students from many disciplines who are taking an Introductory Materials Science and Engineering course have answered both open-ended and multiple-choice questions to probe their understanding of fundamental materials concepts. They have also been interviewed to gain insight on their background so gaps in knowledge and the origin of misconceptions can be identified. Basic concepts utilized in developing the questions include bonding, band structure, crystal structure, defects, atomic motion, microstructure, solutions, simple thermodynamics, deformation, and the nature of metals, polymers, ceramics, and semiconductors. The MCI will be field tested this summer and fall to refine, enhance, expand, and further understand the nature of misconceptions and interventions in engineering materials.

Some surprising results have emerged on students' preparation coming into the class. A few examples are that less than 10% of the students had heard of simple band theory to explain electrical and electronic properties of materials, less than 10% realized that biological and botanical materials (e.g., plants, animals, humans) have much material composed of the same long-chain structure of molecules as synthetic polymers have, and less than 20% had a knowledge or understanding of simple concepts of solubility that are important in the topic of phase diagrams (e.g., if excess saturated salt sits at the bottom of a glass of salt-saturated water, the students thought that adding more salt would increase the concentration of salt in the solution-it does not). Appropriate interventions using active learning techniques are being developed to address misconceptions and gaps in background. Also, discussions will be initiated with introductory physics and chemistry instructors to see if gaps in background can be addressed in those courses.

Fluid Mechanics Concept Inventory

Ty Newell (University of Illinois), Jay Martin (University of Wisconsin), and John Mitchell (University of Wisconsin) met in February 2002 at the University of Wisconsin to develop a taxonomy of fluid mechanics concepts as the first step in developing the Fluid Mechanics Concept Inventory (FMCI). Taxonomy has three broad categories: basic concepts, fundamental fluid relationships, and special cases that may be of special interest to particular disciplines and could form the basis for optional sections of the instrument. Basic concepts encompassed four areas: properties, boundaries, dimensional analysis, and similarity. Fundamental fluid relationships included two areas: continuity and momentum. Under continuity three sub areas were outlined: steady/unsteady fluid flow, compressible/incompressible fluid flow, and the dimensionality of the physical situation to be analyzed. For momentum the following outline was developed

2.Viscous flow (variation of terms 1, 2, 3, and 4)