Individual Research

It seems to be taken for granted that research should be primarily a single-person effort. Even those projects with large teams seems to be broken up so that each contributor provides one component of the whole project and contributes very little to other people’s components. This may be a misunderstanding of how research is actually performed, but it seems to be the case.

Why do we do this? Why don’t researchers work in teams of two on any given topic? It seems that we could do better if there were teams each working on the same problem. Free market economies have demonstrated that multiple solutions from multiple sources tend to yield a better overall answer to any given problem. In research, this is done on a larger scale between different researchers at different universities, but on a smaller scale, what about putting two people on the same task in the same research group? I think that students could help prevent each other from getting stuck.

It might be tough to support two students on the same project, however. Additionally, it might be tougher to give the appropriate amount of credit to each participant (which is a big deal in research). I think I’ll ask around in grad school.

Lets Switch our Focus to Learning Rather than Achievement

We’ve always been told that getting an ‘A’ in school is important.  Our parents will often give us wide smiles and perhaps some money or a nice dinner out.  Rarely does anyone ask, “What did you learn in that class?”  Is that praise for an ‘A’ helping us learn?  According to a psychology study, focusing on performance can result in road blocks for our young learners.

According to Claudia Mueller and Carol Dweck, children focusing on their performance tend to avoid tasks if it might be hard and result in failure.  After performing a set of problems that they were told they scored high on, 5th graders were given choices indicating what kinds of problems they wanted to do next. These choices colored their attitudes towards learning.  The first three choices, “problems that aren’t too hard, so I don’t get many wrong,” “problems that are pretty easy, so I’ll do well,” and “problems that I’m pretty good at, so I can show that I’m smart,” are geared toward attitudes focused on demonstrating their abilities.  The last choice, “problems that I’ll learn a lot from, even if I won’t look so smart,” demonstrated that their main concern was for the development of their ability.  What determined their choice?

In the study, 5th graders were given three types of feedback after being told their score: “you must be smart at these problems,” “you must have worked hard on these problems” and the last group was only told their score, with no additional feedback (the control group).  Children who were told that they must have been smart chose attitudes geared toward demonstrating abilities, while those who were told that they exerted a lot of effort chose the attitude focused on their development.

Children with attitudes geared toward demonstrating their abilities, in turn, tended to want to avoid activities that have the prospect of failure, even when they know that they might gain a valuable learning experience.  To avoid being seen as incompetent now is more important than becoming competent later.  This can be so severe that children who strive to perform well will often freeze when faced with failure and perform worse then they would otherwise. To them, their ability seemed like a stable trait, something they could not change, and, therefore, things that test that ability are worth avoiding when faced with failure.

However, children that focused on their development tended to see failure as a step along a road towards learning.  They attributed their failure to not having put enough work into it yet and that if they worked hard enough, they would get it eventually.  To them, their ability was variable depending on how hard and much they worked on it, instead of stagnant.  They believed this so much that, when tested for their persistence and fun in the problem sets, they were eager to do more after failing a second and harder problem set, unlike their demonstrating-ability counterparts.

This study provides support for praise for effort only after a child has done well.  What do you say in times of failure?  I’ve heard many people say, “well, at least you tried your best.”  The problem I see with this is that the statement ends there.  It does not imply any continual effort.  It only indicates past effort but does not suggest that more is needed.  Perhaps a better way to say it is, “I’m sure if we spent some more time on it, you’ll get it!”  What do you think?

Mueller, Claudia & Dweck, Carol. “Praise for Intelligence Can Undermine Children’s Motivation and Performance.” Journal of Personality and Social Psychology. 1998, Vol. 75, No. 1, 33-52

Paper: Academic Disciplines and Undergraduate Interdisciplinary Education…

William H. Newell “Academic Disciplines and Undergraduate Interdisciplinary Education: Lessons from the School of Interdisciplinary Studies at Miami University, Ohio.” European Journal of Education, Vol. 27, No. 3 (1992), pp. 211-221

  • Key questions: How much disciplinary background is necessary before beginning interdisciplinary work? What role should disciplines play in interdisciplinary courses? Can interdisciplinary classes adequately prepare students for advanced disciplinary work?
  • Miami University’s School of Interdisciplinary Studies the premier undergraduate interdisciplinary program with 300 students 14 full time faculty/staff
  • The School of Interdisciplinary Studies consists of a core curriculum of interdisciplinary courses in humanities, arts, sciences, and technology culminating in a year-long interdisciplinary group project
  • Almost without exception, interdisciplinary curricula across the nation have no disciplinary prerequisites
  • Different operational definitions have arisen out of pragmatic discussions over what to cover in each topic, rather than through intentional philosophical discussions
  • As a result, courses may draw concepts (e.g. discourse communities from lit, marginal utility from econ, etc), theories (e.g. plate tectonics from geo), facts (e.g. Avagadro’s number), or methods (e.g. laboratory experiments in science)
  • An interdisciplinary course can provide illustrative readings written from the perspectives of different disciplines which can enable interdisciplinary thought without prerequisites
  • Faculty teaching interdisciplinary courses are typically expected to have traditional postgraduate training (focused primarily on one discipline) and wide-ranging interests
  • Two issues in interdisciplinary courses: conceptualizing and design an interdisciplinary course, and set up sufficient opportunities for staff to learn about other disciplines from colleagues
  • The most common structural device for promoting staff insight into other disciplines is the weekly staff seminar, where staff members discuss common readings from different discipline perspectives
  • Assessment of interdisciplinary course in this article by Field and Lee
  • The Miami University English Dept. accepts the interdisciplinary writing-across-the-curriculum as the only alternative to freshman English, and faculty members have said that they would prefer all freshman learn composition
  • A study of Miami SIS students found that approximately half of the Interdisciplinary Studies seniors had higher GPAs out of the program and half had lower GPAs outside
  • Additionally, a higher proportion of SIS graduates went on to study for a PhD, and graduates achieved an average percentile score of 85.7% on the LSAT, 79.3% on the GRE verbal section, and 69.2% on the GRE quantitative
  • Self-reported outcomes, percentage of students agreeing with the statements in the ISI program (vs. Miami U.-wide sample, and other highly selective liberal arts colleges sample norms):
      -Often/very often worked with staff on a research project: 20% (4%, 10%)
      -Very often thought about practical applications: 63% (32%, 35%)
      -Very often revised a paper twice or more: 24% (18%, 19%)
      -Very often engaged in outside discussion which referred to class material: 88% (50%, 70%)
      -Often/very often changed an opinion after discussion: 40% (25%, 23%)
      -Read more than 20 assigned books in the last year: 49% (12%, 40%)
      -Read more than 10 non-assigned books in the last year: 24% (33%, 29%)
  • The student responses do not measure the quality of the patterns, but they do tend to lead to the development of intellectual skills and habits prized by traditional education
  • In a 1983 survey of ISI graduates: 83% rated career preparation and 91% rated academic satisfiction in the highest two categories
  • In a 1986/’87 survey of ISI alumni: 84% compared the school favorably to other colleges (vs 36% nationwide norm), 76% said it helped them define and solve problems (vs. a norm of 44%), 85% rated writing skills acquisition very highly (vs. 40%), 51% speaking (vs. 35%), and 78% working independently (vs. 54%)
  • Interdisciplinary learning seems more likely to promote what is termed “strong sense critical thinking,” or becoming critically self-reflective, and “multi-logically,” or the ability to think fairly and reconstruct the strongest arguments from opposing points of view

___________

Paper: Benefiting from Mistakes: The Impact of Guided Errors on Learning, Performance, and Self-Efficacy

S. J. Lorenzet, E. Salas, S. I. Tannenbaum. “Benefiting from Mistakes: The Impact of Guided Errors on Learning, Performance, and Self-Efficacy.” Human Resource Development Quarterly, vol. 16, no. 3, 2005. doi:10.1002/hrdq.1141

  • Ninety undergraduate students with no previous experience received either training that guided them to commit common errors or alternatively training that sought to prevent errors from occurring
  • A typology for manipulating errors and a new way of using errors in training are presented
  • Findings revealed superior performance (accuracy and speed) and self-efficacy associated with using guided errors during training
  • Classic reinforcement theories suggest that training should be structured so errors are minimized. The logic behind this “error-avoidance” approach was that errors take away from “on-task” time and therefore reduce the amount of learned information.
  • The logic behind the use of errors is that they serve an informational function and thus provide feedback, giving trainees an opportunity to see the consequences of their mistakes, learn corrective strategies, and take corrective steps
  • There are four approaches to error occurrence: avoiding, allowing, inducing, and guiding errors, all summarized below:
  • The second component of using errors in training is either allowing trainees to work through the errors unaided by the trainers or any other support mechanism or supporting the trainees through trainer intervention or computer-based assistance
  • The dominant model of training evaluation in the research literature for nearly the past half century has been Kirkpatrick’s four levels of evaluation: trainee reactions, learning, behavior, and results
  • Several new evaluation frameworks have recently emerged emphasizing evaluation issues such as learning, performance, change, perception, and return on investment
  • The emergence of these new models may at least in part be a result of research highlighting the limitations of Kirkpatrick’s evaluation model, including the implication that results are the best measure and the practice of many cases studies relying on just one level of training evaluation
  • On the basis of taxonomies developed in other disciplines, Kraiger and colleagues (1993) proposed three categories of learning outcomes: cognitive, skill-based, and affective. In the present study,
    these categories are represented as cognitive learning, performance, and self-efficacy
  • Guided errors should be more effective than error-free training in giving trainees coping strategies when they do encounter mistakes, may aid in the construction of a mental model and remind learners to avoid prior mistakes, and aid in discovering relevant information or generalizing skills acquired during training to new problems
  • Some of the rationale for advocating error-free training has come from concern over negative affective reactions associated with mistakes, which may lead to reduced self-efficacy and lower performance outcomes
  • Trainees who experience guided errors should be less likely to attribute mistakes to internal causes and thus may not experience a reduction in self efficacy
  • Successful performance during error-free training may actually have some negative effects, including overestimation of skill level and a subsequent drop in self-efficacy after a poorer performance than expected
  • Error-free trainees were given “click-by-click” instructions and were led through training without errors, and guided-error trainees were given the same click-by-click instructions but were led into mistakes and then shown how to fix them
  • Fixing mistakes simply included identifying mistakes and performing the operations correctly, not teaching additional software capabilities
  • A potential concern was that guided-error training would take longer than error-free training, which could account for post-training performance differences, but this was controlled for by ensuring that the type took approximately the same amount of time
  • Results did not reveal statistically significant differences between guided error trainees and error-free trainees with regard to self-efficacy immediately following training
  • Analysis revealed higher post-performance self-efficacy for guided-error trainees compared to error-free trainees
  • Trainees who received guided errors performed more accurately and performed faster than those who received error-free training
  • The self-efficacy of guided-error trainees remains virtually unchanged after performance (M=75.34 before, 77.09 after), while error-free trainees show a substantial drop in self-efficacy (M=77.09 before, 63.60 after)

______________

This research, demonstrating that guiding trainees through errors and their solutions leads to better outcomes than guiding through the correct procedure only, seems both plausible from a personal perspective and immediately useful in an instructional context. While the study does not address the origin of the difference in outcomes from the two training methods, it appears that there are two likely candidate causes for the lower performance in the error-free approach: lack of knowledge on how to fix errors and fear of making errors in the first place. More to come in an upcoming blog post…

Paper: Viewpoint: An Industry View of Engineering Design Education

L M Nicolai. “Viewpoint: An Industry View of Engineering Design Education.” International Journal Engineering Education Vol. 14, No. 1, p. 7-13, 1998

Key points:

  • What industry needs is clear: engineering graduates with a better design experience. American engineering schools respond to this need by producing great scientists but mediocre engineers
  • American industries place the highest value on engineering design in their product development; timing and quality are essential, and both are dependent on the design
  • The solution to this problem involves a major change in the attitudes and priorities within the engineering departments and a minor restructuring of the curriculum
  • An attitude change is required that will make design faculty equal to analytical faculty. This should involve changing the reward/promotion system to be more compatible with the design faculty’s situation
  • An Arizona State University College of Engineering task force (composed of students, faculty and industry representatives) study revealed that the unanimous number one attribute desired for a newly graduated engineer was the ability to identify and define a problem, develop and evaluate alternative solutions, and effect one or more designs to solve the problem
  • Several engineering schools have had great success in teaching engineering science by introducing the course material from a design approach rather than the traditional analytical approach
  • Need a senior year capstone design course and more open-ended problems inserted into the engineering science courses with frequent and spirited discussions of the design process and

_____________

Table 2 is a perfect illustration of the contrast between the problems we work in school and the problems in both industry and academic research. (I am a senior engineering student, and I have worked two positions in both industry and academia so far.) Though the types of problems in industry and academia do differ, they share some key characteristics. First, they are both open-ended. The specifics are different; broadly, in industry the open-ended piece comes in the conceptual design and selection of components and characteristics while in academia it comes in the form of assumptions and models used. Second, they both require mastery of the fundamentals as a stepping stone but always need several more steps afterwards.

The methods of preparing students in school for this type of work, however, focuses exclusively on the stepping stone fundamentals. I do not see how this is the best way to educate either engineers separately, scientists separately, or both together. It does the engineers a disservice by promoting neither practice in applying the knowledge nor design experience. It does scientists a disservice by covering so many topics in such shallow detail and by . Wouldn’t everyone benefit if classes took the form of something like 50% fundamentals, 25% in-depth study of a specific effect learned during the fundamentals section, and 25% design experience? It would require serious thought about exactly what information is important in each class, but that important information would be learned very well while gaining the critical engineering and research experience.

Table 3 is a fascinating portrait of a different educational style, but could that design approach work as the main or only method of learning? Would it be possible to implement a project-based curriculum all the way down from elementary school up through graduate school? Also, could it be retooled to work with interdisciplinary groups? Obviously, the focus might have to be expanded from thermodynamics to energy, for example, to be workable for interdisciplinary study. These are the issues in education that fascinate me, and I’ll be working through them over the next several months. More to come!

Paper: An Industry View of Engineering Education

K M Black. “An Industry View of Engineering Education.” Journal of Engineering Education, January 1994, pp 26-28.

Key points:

  • We [engineers] prided ourselves on performing to specs tighter than anyone else could meet. But no one asked if the customer wanted or needed that extra performance—nor whether he would pay more for it
  • We’re now putting tremendous emphasis on total quality management (TQM), continuous process improvement (CPI) and cycle time reductions
  • Today’s engineer must also have: effective communication skills, a thorough understanding of current design tools (software basics, CAD, simulation, etc.), the expectation that in every project he or she will succeed the first time, a sense of the total business equation
  • One often hears that universities have a responsibility to “broaden” a student—to graduate a “fully-rounded” individual. Frankly, I believe the curriculum too often is overloaded
  • Some of the social studies, philosophy, English literature, and even history and art are personal interests, and engineers, who by their very nature are curious, will pursue those subjects that interest them—after graduation
  • Research capability is important to our nation’s total technology base, provides value-added for the engineering curriculum, and is vital in bringing dollar and faculty resources to the school, but relevant and effective teaching is critical for those who aspire to be engineers
  • Rockwell, for example, has about 15,000 scientists and engineers. We get most of our research scientists from research schools, and most of our engineers from teaching schools or the teaching functions of schools that pursue both endeavors. For every research scientist, the company has 15 or 16 engineers, and engineering is the function most in need of change

___________

Kent Black provides a concise summary of the problem with engineering education. His main points: exclusive focus on theory when only a small fraction of practicing engineers are researchers, very little practice with the tools (numerical analysis, especially), bloated curricula, and little understanding of the business side of engineering. He gives a few solutions, such as better communication between industry and engineering schools, but leaves most of that function to others.

Let me comment on his assessment from the perspective of a senior undergraduate engineering student.

First, too much focus on research and theory. My school (Virginia Tech) has done a better job of this than most, it seems. The two freshman engineering courses, for all their problems, both included excellent, substantial engineering design projects and software primers (Inventor and LabVIEW). There is a lot of support for undergraduate research; my department will pay for a student’s first 60 hours of work in anticipation that the professor will begin paying afterward. There is also a lot of support for summer internships, free access to software packages, relatively open access to certain machine shops, and so on. However, the rest of the curriculum is very heavily theory based (even in the more practically oriented engineering majors like mechanical engineering).

Second, numerical methods. His points on numerical analysis are completely accurate. In my current job at a small R&D company, two of the three mechanical engineers know or are learning some form of numerical analysis (Finite element analysis, finite difference, etc). They plan to budget me some time on a project where I can learn FEA soon. Indeed, another technique I recently came across, topology optimization-an awesome method of geometric optimization, requires FEA as an intermediate step. The numerical tools are obviously critical. My major requires two classes on numerical analysis: numerical methods, and intro to finite elements. It sounds great, but, unfortunately, both classes are exclusively the theory behind them. The theory is obviously worth knowing, but it should be a split of maybe one month of theory and three months of practice with the software tools on large-scale. An idea for how to teach the practice part:  each student in the class of one hundred performs numerical analysis on a part of a huge design, the students combine their results and tweak their design for the next semester’s class to analyze and optimize. Students doing *real* engineering with real engineering tools would be awesome.

Third, bloated curricula. We cannot learn in school all that we will need to know as practicing engineers or researchers or whatever else. How exactly the curricula could be slimmed I do not know. I shall write more on this later.

Fourth, business sense. I actually disagree with his point here. Businesses are better positioned to teach the business of engineering and science. I do fully understand the benefit of learning business fundamentals while in school (I took a technology entrepreneurship/business management class), but they should be one of the “personal interests” that black relegates to a student’s personal or after-graduation time. They can be learned better elsewhere.