Expert Problem Solving

I've been reading Alan Shoenfeld's article Learning to Think Mathematically: Problem Solving, Metacognition, and Sense-Making in Mathematics, where he summarizes an interesting study about novice and expert problem-solvers.

Experts Don't Get Bogged Down

The study videotaped pairs of college students attempting novel problems and coded how much time they spent in different kinds of problem-solving activity. The following was typical of novice problem solvers:

"The students read the problem, quickly chose an approach to it, and pursued that approach. They kept working on it, despite clear evidence that they were not making progress, for the full twenty minutes allocated for the problem session. At the end of the twenty minutes they were asked how that approach would have helped them to solve the original problem. They couldn't say."

They did the same for pairs of math professors, and here is what they found:

From the article:

"The first thing to note is that the mathematician spent more than half of his allotted time trying to make sense of the problem. Rather than committing himself to any one particular direction, he did a significant amount of analyzing and (structured) exploring -- not spending time in unstructured exploration or moving into implementation until he was sure he was working in the right direction. Second, each of the small inverted triangles in Figure 4 represents an explicit comment on the state of his problem solution, for example "Hmm. I don't know exactly where to start here" (followed by two minutes of analyzing the problem) or "OK. All I need to be able to do is [a particular technique] and I'm done" (followed by the straightforward implementation of his problem solution) ....

... as he worked through the problem the mathematician generated enough potential wild goose chases to keep an army of problem solvers busy. But he didn't get deflected by them. By monitoring his solution with care -- pursuing interesting leads, and abandoning paths that didn't seem to bear fruit -- he managed to solve the problem, while the vast majority of students did not."

Teaching Self-Monitoring

A second study attempted to teach students this type of self-monitoring (though perhaps 'habituate' would be a better term). Shoenfeld had students spend approximately 1/3rd of their class time working on novel problems in groups of 3-4. His role was to be a roving consultant, and to press the students with the following 3 questions:

1. What (exactly) are you doing? (Can you describe it precisely?)
2. Why are you doing it? (How does it fit into the solution?)
3. How does it help you? (What will you do with the outcome when you obtain it?)

Initially, students can't answer his questions, because their way of working doesn't include thinking about them. Once they learn that he won't stop asking them, however, they begin to think about them ahead of time, before he asks. By the end of the semester, he reports, this type of thinking has become habitual.

Here is one activity graph for a pair of students who had completed this course.

He concludes:

"The point here is not that the students managed to solve the problem, for to a significant degree solving non-standard problems is a matter of luck and prior knowledge. The point is that, by virtue of good self-regulation, the students gave themselves the opportunity to solve the problem." [emphasis added]

This is like speed dating for math (if you'll indulge me the analogy for 2 paragraphs). Considering more candidate-strategies earlier gives you more chances of finding the right one for you. The difference is that speed dating prescribes a (short) fixed amount of time with each candidate. Problem solving would be more like speed dating if the speed-daters were left to decide for themselves how much time to spend with each person before moving to the next.

So how do you know when to move on? It's easy if you've run out of things to talk about, or if you don't know what to do next with your current strategy. But some people (and strategies) are seductive--it may take a long time to realize they're not taking you where you want to go. Savy and experienced daters, as well as mathematicians, learn to recognize signs of an interaction that seems promising or troubling. And this kind of subtle judgement is something that only comes with reflection on experience, I think.

What Now?

Thinking about this has suggested a few things I'm curious to try.

1). Brainstorm our options first. I might precede some problems with group- or class-wide brainstorming to get a short list of approaches that make sense to try.

2). Heuristics tell you what to try and what to look for. When I was teaching heuristics as skills, I focused on what each heuristic recommends that you try to do. Shoenfeld's questions make it obvious what this leaves out. Learning a problem-solving heuristic also means learning what to look for as you try, so you'll know whether the approach is working or not. It might also involve learning what you might do next with the kind of information you're likely to get. When first teaching students each heuristic through guided examples, I should make all three parts explicit--not just the first, as I had been doing.

3). Practice talking about all three parts. When a student was stuck, my first question was always "what could you try?" If they could give a coherent answer, I would often leave them to try. Instead, I should ask students all three questions: 1). What can you try? 2). What will tell you if it's working? (If you're looking for a pattern, can you quickly write a few types of pattern you might expect from prior experience?) 3). What then? (try to describe the pattern? connect its structure to the problem-statement? falsify it with test cases? represent it algebraically or graphically?).

Until students start to get a feel for how to answer the questions, this will probably have to be on a small-group or individual basis. Once students gain experience, I might stop all problem-solving work at 5 minutes, then at 10 minutes, then at 15, and have students write concise answers to these questions (no more than 1 minute each question).

4). Practice forced switching. Knowing how to make a mental bookmark of your current approach is immensely useful. It's required for high-level task-switching, but also to simply listen to and understand someone else's ideas. Being forced out of one's comfort zone is also sometimes required to experience the benefits of a new approach. Students might work for 5 minutes down one path, summarize their progress, then work down another path from scratch. This would be good practice for #3, and also let them know what it feels like to mentally start over, which is something that happens all the time in problem solving.

Those "getting to know you" handouts

Every teacher I know gives a "getting to know you" questionnaire at the beginning of the year, including me! Cleaning up all my papers in June, I'm always surprised at just how much students really told me about themselves in that first week, and how I wished I'd used that information better somehow. What that in mind, here are a few of my ideas about What to ask and How to use the answers.

1. Everyone's got something

For the past 3 years I've asked students the following unexceptional questions:

a). Describe yourself in 3 words: ___________, ___________, ____________
b). Describe school in 3 words: ___________, ___________, _____________
c). What's something you're really good at?
d). What's something you wish you were really good at?
e). What do you want to do "when you grow up?" (or what do you definitely NOT want to do?).

I've often been surprised by their interesting answers. "I can swim a technically perfect butteryfly", "I wish I were better at controlling my will", "I want to be a physical anthropologist (!)", "I know how to breed giant tortoises".

I make an overhead for each question with an anonymous jumble of answers. Even though most juniors and seniors recognize each other by the time I get them, they don't really know each other. I find that sharing the responses helps creates a really positive atmosphere of excitement and curiosity about each other. Also, the parents at back to school night love seeing the answers as well.

2. All I had to do was ask

One year on the back of my expectations form I also asked:

a). Which of these expectations do you think you'll have the most trouble with?
b). What can I do to help you succeed?

Common responses include "always doing the homework", "not being tardy", but also things like "I have trouble asking for help when I need it" and "I have trouble working in groups". These provide great heads' up for all kinds of things. The answers to part (b) are typically less useful, but sometimes can be a real help. Every now and then you'll get a student with some really bad habits who actually asks you to be really tough on them, which is a great point to come back to with them later.

3. Matchmaking

Something I haven't tried but would like to is asking the students something like this:

(a) Circle things you're pretty good at
(b) Underline things you still struggle with

The list could include specific math topics, more general skills, and "soft skills" like working with others, explaining your thinking verbally or in writing, asking good questions, etc.

Aside from providing a very rough pre-assessment of knowledge, this information could be used for creating working pairs, or surreptitiously choosing warm-up problems to boost the confidence/status of particular individuals.

Another nice question for designing working groups might be to ask about working style. Something like:

(c) Rate how much each statement applies to you:
I like to work by myself.
I like to explain my thinking out loud.
I like to have someone explain their thinking to me.
I like to try things myself, but have someone close-bye to help if I get stuck.
etc.

Clearly there's a lot of ways to get at this, and I don't particularly like this version, but that's the general idea.

4. Stealing ideas through the students

One year I asked:

(a) What are things that teachers n your past have done that really helped you learn?

I've gotten all kinds of great suggestions from this one. The opposite question of what to avoid is usually less helpful as students say things like "don't be boring".

Something that has worked for me, however, is asking students at the end of the first week

(b) What about the class is "working for you" (helping you learn)
(c) What about the class isn't working for you? What could I change?

Last year, students told me that I should: "slow down", "do more examples more slowly", and "give us more chances to do examples at the board." Obviously not all suggestions will be consistent with your goals or values, but the students are telling you something that needs paying attention to. Maybe their suggestions reveal that they don't understand what you expect of them in the same way you do.

On my end-of-the-year suggestion form for students, I noticed that some of their suggestions were the same as the ones they gave me in that 1st week (ouch!). I tried, but clearly it wasn't enough. Next year I'm thinking of re-phrasing some student suggestions I want to work at and having them all rate my success every few days so I can tell if I'm getting better.

5. Actually doing something

All that said, the main challenge I have isn't asking the questions but actually doing something with the data. Without a specific plan (typically including dates) it's too easy to file the answers away until June cleaning. Two other practical issues are the sheer time involved in compiling some of the data, and finding a useful and accessible format to put it in for when I need it. That's all a little much for the denouement of a post, so I'll leave those details until next year when I actually try to implement some of these again.