“It’s crystal clear that the economic recovery is in doubt—you have an economy that is in need of organic growth and you’re not getting it from these reports,” said Todd Schoenberger, managing principal at The BlackBay Group. “Participation rate dropped and average working hours are down—if you’re lucky enough to have a job, your spending power is still down.”

Oil prices dropped sharply to settle near $98 a barrel , pressured by the jobs report. Southwestern Energy was the biggest laggard in the sector after the firm posted disappointing earnings.

Despite the disappointing report, traders say the jobs number was not weak enough to force the Fed's hand on a new round of stimulus, which has been a positive catalyst for stocks.

"Stock market bulls should be forewarned: The bottom-line from this report is that the momentum in the labor market is slowing, but not enough to bring the Fed off the sidelines for an additional round of asset purchases," according to a BofA Merrill Lynch research note.

Apple dragged on the Nasdaq as the iPad maker fell below its 50-day moving average of $585 a share. The iPad maker has slumped nearly 6 percent for the week, and almost 12 percent from its all-time high of $644 a share.