Gold prices are at an all-time high—but experts like Warren Buffett don’t always recommend

But even with a favorable outlook, gold should play a very different role in your portfolio than stocks or bonds, investing experts say.

Because it tends to move in different ways than more traditional investments, gold may be an appropriate way to diversify for some investors — but don’t make it a major building block of your portfolio. Billionaire investor and Berkshire Hathaway chairman Warren Buffett is known to avoid it for a reason.

Different investors cite different reasons for owning gold. For one, it has a reputation to maintain or increase in value during periods of inflation, though that track record is spotty. For another, it’s considered a store of value should paper money become significantly devalued — after all, gold has been considered currency for millennia.

It’s also generally expected to hold up in so-called “risk off” markets, when investors tend to flee from riskier fare, like stocks, into perceived safe-haven assets, including gold and bonds. That means investors tend to pick up more gold in the lead-up to and during recessions and bear markets.

That makes the recent uptrend in gold a little bit strange, says Ford O’Neill, co-portfolio manager at the Fidelity Strategic Real Return Fund, a mutual fund strategy focused on shielding investors from inflation risk.

“It’s been anything but [a risk off market] since October of last year,” he says. “I would argue we’ve had what I would call an ‘everything rally,’ where obviously quite a few assets have done quite well.”

Essentially, he says, gold is doing well because investors are boosting the price of just about everything, from stocks to bonds to cryptocurrency.

In addition to a rising tide, a weakening U.S. dollar and falling bond rates have boosted gold prices of late, says Hayes. At lower rates, bonds and cash accounts “have less of a competitive advantage” over gold, he tells CNBC Make It.

And with the Federal Reserve projected to begin cutting interest rates this year, the outlook for gold is growing rosier. The lower interest rates go, the lower the opportunity cost for investors to hold gold, which pays no interest.

“We continue to be bullish on gold,” says Hayes.

If you want to add gold to your portfolio, the easiest way is to buy an ETF which tracks the price of the yellow stuff. Doing so allows you to track gold’s performance relative to the rest of your portfolio and keeps you from having to shell out big bucks to own physical gold.

But whether you hold it in your brokerage account or stash coins and bars in your safe, gold is an asset that doesn’t produce anything. That’s why the world’s most famous long-term investor never touches the stuff.

In his 2011 letter to shareholders, Buffett pointed out that for the price of acquiring all the world’s gold, an investor could buy all of the cropland in the U.S. with enough money leftover to buy ExxonMobil 16 times over. Come back a century later, and one of those options will have delivered a bounty of crops and ample dividends. The other would still be a large quantity of gold.

Over the past 15 years, an ETF tracking the spot price of gold has returned an annualized 5.5% compared with a 15.3% return in the S&P 500.

As for inflation, gold’s record is a mixed bag. Despite steady inflation since 1988, gold has submitted a negative return in 18 calendar years, including 2021 and 2022 — years with particularly high inflation.

Some investors like to hold a small allocation of gold because it provides peace of mind when other assets are in decline.

“When everything else is going down the tubes, gold is the one thing that’s likely going to do well,” William Bernstein, the author of “The Four Pillars of Investing,” recently told CNBC. “Home insurance also has a high return when you have a fire.”

But over the long term, you’re better off with assets that will grow and deliver returns at a compounding rate. Take it from Buffett.

“True, gold has some industrial and decorative utility, but the demand for these purposes is both limited and incapable of soaking up new production,” Buffett wrote in 2011. “Meanwhile, if you own one ounce of gold for an eternity, you will still own one ounce at its end.”

 Want to make extra money outside of your day job? Sign up for CNBC’s new online course How to Earn Passive Income Online to learn about common passive income streams, tips to get started and real-life success stories. Register today and save 50% with discount code EARLYBIRD.

Plus, sign up for CNBC Make It’s newsletter to get tips and tricks for success at work, with money and in life.