The short answer is: half, U-5 probably tells you everything you need to know, and women are going to play the most interesting role as it evolves. Now for the question and longer answer....
The average labor force participation rate went from an average of 66% in 2007 to a 2011 average of 64.1%. Last month it was 63.6%. As a reminder, the labor force is the employed and the unemployed (those without a job who are actively looking for one) added together. When this number decreases it means that there are less people working, though it doesn't increase the unemployment rate (because, by definition, those leaving the labor force are no longer looking for a job). Let's try to get our arms around the latest econoblogosphere debate: how much is the decrease in labor force participation a type of shadow unemployment?
To recap, there's a handful of longer-term trends to watch in the economy. When Ben Bernanke was asked about labor force participation at his most recent press conference, he responded that labor force participation was dropping because the economy was (my bold) "no longer getting increased participation from women... society ages and also, for other reasons, male participation has been declining over time." However a lot of it "represent cyclical factors, much of it is young people, for example, who presumably are not out of the labor force indefinitely, but given the, uh, weak job market, they are going to school or doing something else, rather than, than working."
But how to get a good estimate of what is cyclical - related to the economic downturn - and what is structural and the result of longer-term trends - what would have happened without the Great Recession? First off, what's the largest number possible? Evan Soltas (a new blogging superstar you should be reading) takes the labor force participation rate of 2007 and projects it to now, and finds 5.8 million people missing. This would give us an unemployment rate of around 11.4 percent, but would also exclude the longer-term trends. Greg Ip, looking at CBO numbers, finds 5 million people missing.
At the other end of the spectrum are those who would think that the unemployment rate is capturing all we need to know. If someone really wants a job, they would look for one, and there's nothing interesting policy-wise in this information. At 8.1% unemployment there's still plenty of slack in the labor market. (There's an unemployment crisis at 8.1% unemployment!) The answer of the "true" unemployment rate should be somewhere in the middle.
Chicago, Kansas City
the current LFPR [Labor Force Participation Rate] is roughly 1 percentage point lower than our estimated trend rate (the LFPR consistent with the contemporaneous composition of the work force and an economy growing at its potential)....As of late 2011, the actual LFPR for 16–79 year olds is 1.1 percentage points below trend LFPR...Indeed, over the 2008–11 period, we find that only one-quarter of the 1.8 percentage point decline in actual LFPR for 16–79 year olds can be attributed to demographic factors.
Since so much of the cylical elements of the labor force participation is driven by female labor choices, those will be key in understanding how this evolves. Catherine Rampell wrote last December about how young women dropping out of the labor force "are not dropping out forever; instead, these young women seem to be postponing their working lives to get more education." We could see a wave of much more highly educated women enter the labor force further down the road. And the New York Fed's blog argued that "a key factor for future aggregate labor force participation is the behavior of married women," and whether or not they look to re-enter the labor force. In general, and likely for men, as both the Kansas City paper and Ryan Avent note, many of these workers are going into disability.
Overall I agree with what Ryan Avent argues here. If we were hitting constraints, we'd see job openings and prices, especially labor costs, shooting upwards, which we do not see. I'm not sure what policy lessons people are drawing from these missing workers, but they amplify the case that expansionary policies, from fiscal to monetary to debt workouts, are necessary and urgent.