Buffett Questions Performance as S&P 500 Beats Berkshire

NEW YORK (TheStreet) - For the first time, Warren Buffett appears concerned he will underperform the S&P 500 when it comes to his favorite way to peg the performance of his investing conglomerate, Berkshire Hathaway(BRK.A) .

In Berkshire Hathaway's annual letter to shareholders, Buffett outlined why he is worried a rising stock market will put the firm's performance below that of the S&P 500 over a five-year stretch.

Such a scenario would be the first in Berkshire's history, indicating that even the 'Oracle of Omaha' is having trouble keeping up with rising markets.

Still, the lead to Buffett's self-deprecating annual letter may simply be a way for him to put a humble spin on a year when Berkshire's book value increased by $24.1 billion and the company's profit to shareholders nearly reached $15 billion, amid strong performance from new subsidiaries like Burlington Northern Santa Fe, Lubrizol and MidAmerican Energy and a solid year of investment gains from large holdings of Coca-Cola(KO) , Wells Fargo (WFC) and American Express(AXP) .

Berkshire's over 14% stock gain in 2012 also beat the S&P 500, a performance most investors would take as proof of a strong year.

Buffett, in contrast, said in the letter to investors that he fears Berkshire's five-year book value growth rate may underperform the S&P 500 for the first time ever. In 2012, Berkshire's book value per share increased 14.4%, below gains for the S&P 500 in excess of 15%.

The legendary value investor attributed his feared underperformance to rising markets, which may reflect expectations of continued economic recovery.

"Our relative performance, however, is almost certain to be better when the market is down or flat. In years when the market is particularly strong, expect us to fall short," said Buffett.

To that end, Buffett noted subpar book value gains were a result of Berkshire's operating subsidiaries. Meanwhile, the firm's investing portfolio significantly outperformed major stock indices. "They left me in the dust," Buffett wrote in an intentionally small-sized font, referring to Berkshire's two portfolio managers Todd Combs and Ted Weschler, who have taken an increasing role in common stock investments.