Analysts said one factor that could help calm bank shareholders' worries about bonds is that the market is not as volatile as it might look.
"Bond volatility may have risen, but it is not outside the normal range: 30-day volatility is just under a standard deviation above the long-term average and 60-day volatility is just over," said Nicholas Smith, Japan strategist at the brokerage CLSA in Tokyo.
Smith said the fact that Japanese yields were rising was in sync with a rise in bond yields in other major debt markets and so was not unusual.
The trigger for Thursday's spike in yields followed an overnight rise in U.S. Treasury yields as bond investors fretted about an early end to the Federal Reserve's quantitative easing program following comments from Fed Chief Ben Bernanke.
(Read More: Wagging the Dog: Why the Fed Fears Wall Street )
"I'm not particularly worried about the rise in JGB yields - so what if we are at 90 basis points? Bond yields around the world are rising," Smith told CNBC's "Cash Flow."
The rise in yields also meant the bonds could become attractive to institutional investors such as pensions funds. Their buying, together with buying from the central bank, could help slowdown the rise in yields, analysts said.
Tai Hui, chief Asia-Pacific strategist at JP Morgan Funds said that he expected the 10-year JGB yield to rise to around 1.1-1.3 percent over the next 12 months - implying a rise of roughly 20 basis points from current levels.
- An earlier version of this story said that Japanese banks held 40 trillion yen ($390 billion) worth of these bonds. This version corrects that to say some of the biggest banks hold this number.
More From CNBC