Saturday, July 4, 2009, 5:56AM ET - U.S. Markets Closed.

Charles Wheelan, Ph.D. The Naked Economist

Charles Wheelan, Ph.D., The Naked Economist

Want Good Schools? First, Define 'Good'

by Charles Wheelan, Ph.D.

Very Good (505 Ratings)
3.837618/5
Posted on Tuesday, July 31, 2007, 12:00AM

The country may be sweltering in the August heat, but that means autumn isn't far off. Vacations are winding down. The NFL has started training camp. Mattress stores everywhere are gearing up for their Labor Day sales.

And soon the kids will be back in school. This seems like a reasonable time to begin a series of columns on what we know about the economics of school reform -- or, more accurately in some cases, how surprisingly little we know.

Something We Didn't Learn in School

We know a lot of things about educational outcomes -- for instance, that schooling matters now more than ever before. The wage gap between college graduates and high school graduates is large and growing. And students who end up at the bottom of the skill spectrum, such as high school dropouts, have seen their paychecks shrink now that a personal computer or a guy in India with a phone can do the same work for less.

And we know that American students aren't doing particularly well by international standards. In eighth grade science, for example, U.S. students trail their peers in Singapore, China, Korea, Japan, Hong Kong, Hungary, and Estonia.

But school itself -- what happens inside the buildings between the first and last bells -- is still largely a black hole. We have a very limited understanding of what inputs -- teachers, books, curricula, memorization, reading, extracurricular activities -- map into the outcomes that we care about, whether it's test scores, or earnings, or just not dropping out of high school.

An Error in Logic

It's actually more frustrating than that. Here's the shocking secret of school reform research: We don't really even know which schools are good schools.

That seems impossible. Surely the local newspaper has just released the list of the "50 Best Schools" or "20 Communities with Great School Districts." And indeed, the students in those places are likely to be doing very well.

But that doesn't necessarily mean the schools are any good.

Why? Well, see if you can spot the flaw in this logic: "The pediatrics unit of the hospital is doing a far better job than the intensive care unit because fewer of the patients are dying."

Hint: The patients in the intensive care unit start out sicker. And that's exactly what happens with schools. Family background has a profound influence on student outcomes. Highly educated parents produce children who do well in school, for lots of fairly obvious reasons: Smart parents are more likely to have smart kids (though it's no sure thing), and they spend extensive time and money doing things that promote the academic success of their children.

What 'Good' and 'Bad' Really Mean

More than a decade ago, I wrote a magazine story on a very famous high school located in an affluent Chicago suburb. I remember an education researcher telling me at the time, "If you took these kids and put them in a closet in their freshman year, and then let them out three years later, they'd still do well on the SAT."

(Just to be clear, this was a hypothetical example. No students were actually put in a closet. Their parents would have gone ballistic.)

Sadly, the opposite is true in other communities: Students begin kindergarten woefully unprepared to learn, and they receive little support while they're in school. Because American communities tend to be highly segregated by income, we have a lot of schools with a disproportionate number of privileged or disadvantaged students.

We all know that -- and yet we routinely describe schools as "good" or "bad" based on things that have more to do with who walks in the front door than with what happens inside the building.

Measuring with a Broken Yardstick

Think of it this way: If a golf pro gives Tiger Woods a lesson, and a different golf pro gives me a lesson, can we conclude that Tiger's teacher is better than mine because Tiger beats me by three shots in a match after our lessons?

That's usually how newspapers and real estate agents pick the "best schools." The true measure of quality -- with golf pros or elementary schools -- is how much value they add. In education, that turns out to be extremely difficult to calculate.

You may be thinking that we could probably fix this with a simple pre- and post-test -- let's measure what kids know when they walk in the school door, and then measure how much they know when they leave, one or four or eight years later. The best schools will be the ones with the students who show the most improvement.

Not exactly. Gifted students don't merely begin at a higher level, they also learn at a faster rate. So, to stick with our athletic example, suppose that neither Tiger Woods nor Michael Moore has ever played tennis before. If we give them each a tennis coach, can we evaluate the quality of those coaches based on the subsequent outcome of a tennis match between Tiger Woods and Michael Moore? (Pause for a moment and consider what a great pay-per-view event that would be.)

Evaluate the Evaluation

So schools with high test scores may or may not be doing a great job; perhaps their students are capable of much more. And conversely, some schools with middling or poor test scores may be doing a terrific job educating students who would otherwise be failing abjectly.

Obviously, we can spot the outliers -- the school in the middle of Detroit that manages to send 95 percent of its students to college, say. If we give researchers enough time and enough data, they can try to answer the school-quality question using statistical techniques that take account of what kind of students are walking through the front door.

But even then the results are often equivocal. The bottom line is that it's hard to evaluate school quality, which is why it's even harder to make schools better.

The First Challenge

We're trying to encourage and replicate success without being able to tell with any degree of certainty which schools are succeeding, then. Imagine a pharmaceutical company trying to evaluate new cancer treatments without being able to determine which patients are getting better.

So that's the first big education challenge -- developing a more sophisticated way to identify "good schools." Only then will we be able to create more of them.

Rate This story

Very Good (505 Ratings)
4/5
Sign-in to rate!

122 Comments

Showing comments 1-5 of 122Next >>
Sort: first to last
  • sajugj - Friday, February 13, 2009, 7:44PM ET  Report Abuse

    • Overall: 5/5

    I have been searching the internet for an article like this and I am pleased to have discovered it at last. As a student that attended the Bronx H.S of Science, NYU and City College, I discovered that schools with lesser prestigious names had different opportunities. As a high school math teacher, I felt that it was unfair that my ability to teach was based more on my ability to teach students that had no interest in learning than those that truly wished to excel. As a parent, I discovered that the local elementary school in the Bronx had much more extensive Parent-Teacher workshops than in the prestigious Riverdale elementary school. Unfortunatlely, my daughter has to commute to Riverdale because the local school does not have a program for the gifted and talented program. The best education is to not to only attend a school with a prestigious name, but to go to diffferent schools. This article is a refreshing break from reading the publications that constantly rate schools as the "best".

  • pierpaolo_savio - Tuesday, November 13, 2007, 9:42AM ET  Report Abuse

    • Overall: 4/5

    pls try to give hints for a solution in addition to raising the problem, thank you!

  • Yahoo! Finance User - Saturday, September 22, 2007, 12:40AM ET  Report Abuse

    • Overall: 2/5

    Blather. Need more concise ending; the last section is weak.

  • meterramadre - Monday, August 27, 2007, 11:17PM ET  Report Abuse

    • Overall: 3/5

    Good analogies. Incredibly complex problem. If you feel interested in working toward solutions, I would strongly encourage you to volunteer at a local school (fingerprinting &background check aren't as big a deal as you might think) and see what the teachers work with every day. I'm a volunteering parent and I see the whole gamut of students come to our public schools. Interested parents are, of course, an important part of the equation. But some kids just won't ever have that. Another adult or teen--a mentor--could make all the difference. Maybe you!

  • happyness_2b - Saturday, August 25, 2007, 11:11PM ET  Report Abuse

    • Overall: 4/5

    Agreed.Especially in the light of US students being one of the poorest in quantitative abilities internationally - you have to take the quality of schools with a pinch of salt.

Showing comments 1-5 of 122Next >>
The columns, articles, message board posts and any other features provided on Yahoo! Finance are provided for personal finance and investment information and are not to be construed as investment advice. Under no circumstances does the information in this content represent a recommendation to buy, sell or hold any security. The views and opinions expressed in an article or column are the author's own and not necessarily those of Yahoo! and there is no implied endorsement by Yahoo! of any advice or trading strategy.

An accessible and entertaining introduction to economics for lay readers, now available in paperback.

View more about Charles Wheelan.

The Chicago Tribune described Naked Economics as "clear, concise, informative and (gasp) witty."

Order Naked Economics today. Average customer review on Amazon.com: 4.7 out of 5 stars.

More from Yahoo! Sources

  • CNN Money
  • Consumer Reports
  • Kiplinger
  • The Motley Fool
  • Business Week
  • Wall Street Journal

Sponsored Links

Earn From 2.35% to 2.65%
With GE Capital Corporation. Not An Offer Of Securities For Sale.
www.geinterestplus.com
Buy Stocks - $4 Fee at ShareBuilder
No account or investment minimums. No inactivity fees. Start today.
www.sharebuilder.com
Traders Wanted
Learn how you can begin trading forex with GFT. Free Practice Account.
www.GFTforex.com
personal finance quote
Make Real $375 A Day At St Louis. Requires: Just A Computer.
TheDallasChronicle.com/jobs
How to Reduce Your Debt
Personal Budgets, Managing Money. Improve Your Personal Finances.
MainStreet.com/budgeting
Free 2009 Credit Report and All 3 Scores
Free 3-bureau Credit Report – includes Transunion, Equifax, Experian.
FreeCreditReportsInstantly.com

Historical chart data and daily updates provided by Commodity Systems, Inc. (CSI). International historical chart data and daily updates provided by Morningstar, Inc. Fundamental company data provided by Capital IQ. Quotes and other information supplied by independent providers identified on the Yahoo! Finance partner page. Quotes are updated automatically, but will be turned off after 25 minutes of inactivity. Quotes are delayed at least 15 minutes. Real-Time continuous streaming quotes are available through our premium service. You may turn streaming quotes on or off. All information provided "as is" for informational purposes only, not intended for trading purposes or advice. Neither Yahoo! nor any of independent providers is liable for any informational errors, incompleteness, or delays, or for any actions taken in reliance on information contained herein. By accessing the Yahoo! site, you agree not to redistribute the information found therein.

Yahoo! Answers is provided for informational purposes only, and no Q&A is intended for trading or investing purposes. Yahoo! shall not be responsible or liable for the accuracy, usefulness or availability of any Q&A information, and shall not be responsible or liable for any trading or investment decisions based on such information. View Complete Answers Disclaimer.