Introduction and Invitation

Constructing resources for assessment and instruction related to the eleven student outcomes contained in Criterion 3 of the ABET Engineering Criteria requires contributions across the entire engineering community. If you have one or more resources (for example, helpful papers, survey forms, assessment materials, instructional materials) for assessment and/or instructional related to outcome c click here. Please indicate whether and how you would like your contribution to be acknowledged. Thanks for contributing the growing understanding of how we might help engineering students develop knowledge and skills that they will draw upon throughout their careers.

Learning Objectives

The first step in selecting assessment and instructional approaches for a learning outcome is to formulate learning objectives that support the outcome. Learning objectives describe expectations associated with the outcome in terms of expected and observable performances. Several researchers have already constructed learning objectives and these may provide worthwhile starting points for others.

A team of researchers (Larry Shuman, Mary E. Besterfield-Sacre, Harvey Wolfe, Cynthia J. Atman, Jack McGourty, Ronald L. Miller, Barbara M. Olds, and Gloria M. Rogers) working a NSF-supported project, Engineering Education: Assessment Methodologies and Curricula Innovation, used Bloom's Taxonomy to develop and organize a set of learning objectives for outcome 3c (an ability to design a system, component, or process to meet desired needs within realistic constraints such as economic, environmental, social, political, ethical, health and safety, manufacturability, and sustainability) [1]. They constructed 13 different outcome elements and then, for each outcome element, developed learning objectives for each of the six levels in Bloom's taxonomy.

• Need Recognition: Identify stated and unstated wants and needs that motivate the design effort; convert them into a needs statement.
• Problem Definition: Determine design objectives and functional requirements based on needs statement, identify constraints on the design problem, and establish criteria for acceptability and desirability of solutions.
• Strategic Planning: Develop a design strategy, which includes an overall plan of attack, decomposition of design problem into subtasks, prioritization of subtasks, establishment of timetables and milestones by which progress may be evaluated.
• Tactical Control and Management: Guidance of course of action during design and in response to changing conditions.
• Information Gathering: Gather information about the design problem, including the need for a solution, user needs and expectations, relevant engineering fundamentals and technology, and feedback from users.
• Generate Ideas: Transform functional objectives/requirements into candidate solutions.
• Feasibility: Evaluate feasibility of alternatives or proposed solutions by considering stated constraints as well as implied constraints such as manufacturability, cost, compatibility.
• Evaluation: Objectively determine relative merit of feasible alternatives or proposed solutions by comparing expected or actual performance to evaluation criteria.
• Selection / Decision: Selection of most feasible and suitable concept among design alternatives.
• Implementation: Creating an instance of physical products and processes for purpose of testing or production
• Communication: Exchange of information with others, utilizing appropriate formats.
• Documentation: Produce usable documents of record regarding the design process and design state, including decision history and criteria, project plan and progress, intermediate design states, finished product and use of product.
• Iteration: Utilize strategies to inform design decisions which may contribute to a change in a design state (e.g., the problem definition, problem solutions, or design process plan).

A multi-institution project, Transferable Integrated Design Engineering Education (TIDEE), has worked with 30 capstone (senior) design course instructors of various disciplines and 10 engineers in business and industry to help TIDEE personnel construct an engineer profile with 12 attributes and 45 key actions that describe characteristics of high performing engineers [2]. The 12 attributes, some of which might be more closely related to other ABET program outcomes, are listed below. The 45 key actions can be found on the TIDEE web site.

• Technically Competent Competent in knowledge and tools of engineering
• Business Aligned Conducts engineering in a business environment
  Customer/Quality Focused
• Idea Generator Finds and creates useful ideas
• Decision Maker Makes sound decisions
• Solution Integrator Produces engineering products, processes, plans, and/or systems
• Teamworker Builds and maintains effective collaboration
• Leader Initiates and facilitates achievement
• Communicator Exchanges information to meet needs of stakeholders
• Results Oriented Proactively completes assignments
• Change Manager Pursues strategic personal development Invests in self-assessment, planning, and learning for ongoing professional growth
• Principle Centered Acts from professional and ethical principles
  

Assessment Approaches

In a report from the National Research Council, Knowing What Students Know: The Science and Design of Educational Assessment [3], assessment, once expectations have been constructed, rests on three pillars: cognition, observation, and interpretation. Following this recommendation, the present section has subsections for each of the three pillars and then offers suggestions on assessment approaches for outcome c, an ability to design a system, component, or process to meet desired needs.

Theories of Cognition

Schön's Reflective Practitioner Theory

Schön [4]

Under construction (25 Feb 2005)

Theories of Observation

Under construction (25 Feb 2005)

Theories of Interpretation

Under construction (25 Feb 2005)

Potential Assessment Resources

Under construction (25 Feb 2005)

Instructional Approaches

Under construction (18 Jan 2005)

References for Further Information

1. Learning Outcomes/Attributes, ABET c—Design a System, Component, or Process, accessed 18 January 2005
2. Davis, D., Trevisan, M., Daniels, P., Gentili, K. Atman, C., Adams, R., McLean, D., Beyerlein, S. (2003). A Model for Transferable Integrated Design Engineering Education. Proceedings, World Federation of Engineering Conference
3. National Research Council. (2001). Knowing What Students Know: The Science and Design of Educational Assessment. Committee on the Foundations of Assessment, James W. Pellegrino, Naomi Chudowsky, and Robert Glaser, editors, Board on Testing and Assessment, Center for Education, National Research Council.
4. Schön, D.A. (1993) The reflective practitioner: how professionals think in action. New York: Basic Books

Resources

Transferable Integrated Design Engineering Education (TIDEE)

The Transferable Integrated Design Engineering Education (TIDEE) project focuses on developing and assessing students' capabilities in engineering design. The end goal is to produce qualified and successful engineers for future generations. In order to achieve its goal, design education must be transferable, integrated holistically throughout the student's education, educate students and document the learning of content and skills important to engineering practice. The project has constructed:

• Definitions of student learning outcomes for engineering design, based on broad constituency input;
• A framework for organizing and implementing engineering design instruction that develops students’ capabilities in the use of the engineering design process, teamwork, communication, and higher-level professional skills required in engineering practice; and
• An assessment and evaluation system that supports classroom assessments for improved student learning and evaluations of student achievement for program assessment and accreditation.

Sims-Knight, J.E., Upchurch, R.L., Pendergrass, N., Meressi, T., Fortier, P., Tchimev, P., VonderHeide, R., and Page, M. (2004). Using Concept Maps to Assess Design Process Knowledge. Proceedings, Frontiers in Education Conference

Abstract: If engineering educators are to incorporate assessment of student learning outcomes into their curricula, they need assessments that are reliable, valid and feasible within the time constraints of coursework. We are engaged in an NSF supported project to develop such measures for design skill. This paper describes our exploration of the use of student-generated concept maps to assess students understanding of how various aspects of the design process go together. Students in three senior-level engineering courses constructed concept maps of the design process. The resulting maps could be reliably sorted into patterns that presumably represent distinctly different ways of understanding the process. In addition, subpatterns of the concept maps were used to assess specific units of knowledge (e. g., the relation between feasibility, on the one hand, and requirements and preliminary design, on the other). These two components comprise an easily created report that provides detailed and useful pointers toward course and curricular improvement.

Sims-Knight, J.E., Upchurch, R.L., Pendergrass, N., Meressi, T., and Fortier, P. (2003). Assessing Design by Design: Progress Report 1, Proceedings, Frontiers in Education Conference

Abstract: This paper is a first progress report of a National Science Foundation Assessment of Student Achievement project designed to develop and test assessments of the ABET student learning outcome of “an ability to design a system, component or process to meet desired needs.” The overall strategy is to use the research on the nature and acquisition of expertise to guide the choice of assessment tools and then to embed those assessments in a course-based continuous improvement loop to evaluate which work best to improve those learning outcomes. So far, the project, which began in June 2002, has yielded prototypes of four assessments and a precursor to a fifth. One of those tools, a multiple-choice test of declarative understanding of design process principles, has been refined and the current version (the third of four versions) exhibits both reliability and validity.

CDIO Standards

In January 2004, the CDIO Initiative adopted 12 standards that describe CDIO programs. These guiding principles were developed in response to program leaders, alumni, and industrial partners who wanted to know how they would recognize CDIO programs and their graduates. As a result, these CDIO Standards define the distinguishing features of a CDIO program, serve as guidelines for educational program reform and evaluation, create benchmarks and goals with worldwide application, and provide a framework for continuous improvement.

Wood, K., Jensen, J., Bezdek, J., and Otto, K.N. (2001). Reverse Engineering and Redesign: Courses to Incrementally and Systematically Teach
Design. Journal of Engineering Education, 90:3, 363–374

Abstract: A variety of design-process and design-methods courses exist in engineering education. The primary objective of such courses is to teach engineering design fundamentals utilizing repeatable design techniques. By so doing, students obtain (1) tools they may employ during their education, (2) design experiences to understand the “big picture” of engineering, and (3) proven methods to attack open-ended problems. While these skills are worthwhile, especially as design courses are moved earlier in curricula, many students report that design methods are typically taught at a high-level and in a compartmentalized fashion. Often, the students’ courses do not include opportunities to obtain incremental concrete experiences with the methods. Nor do such courses allow for suitable observation and reflection as the methods are executed. In this paper, we describe a new approach for teaching design methods that addresses these issues. This approach incorporates hands-on experiences through the use of “reverse-engineering” projects. As the fundamentals of design techniques are presented, students immediately apply the methods to actual, existing products. They are able to hold these products physically in their hands, dissect them, perform experiments on their components, and evolve them into new successful creations. Based on this reverse-engineering concept, we have developed and tested new courses at The University of Texas, MIT, and the United States Air Force Academy. In the body of this paper, we present the structure of these courses, an example of our teaching approach, and an evaluation of the results.

The Computer Engineering and Electrical Engineering Programs at the Unviersity of California, Santa Clara describe specifically how they assess and evaluate the performance of their graduates with respect to outcome c, an ability to design a system, component, or process to meet desired needs.

Adams, R.S., Turns, J., Atman, C.J. (2003). Educating Effective Engineering Designers: The Role of Reflective Practice. Design Studies, 24:3, 275-294

Abstract: Educating effective engineering designers is an important goal. Exploring the extent to which this goal is being met hinges on our ability to characterise what contributes to effectiveness and to map students’ performance against such standards. In previous work, we used verbal protocol analysis to analyse differences in the design processes of freshmen and seniors, the effects of interventions on student design processes, and process factors that contribute to product quality. In this paper, we utilise Schön’s reflective practitioner theory to discuss our empirical results in the context of educating reflective practitioners. Such an approach may provide implications for enhancing engineering education.