You have hit on the key - it is refining capacity.
If design begins on a new complex today it will be 5 years before it is fully on-stream. That assumes it doesn't get delayed for years by ecolawsuits.
Misguided regulations also have a huge effect on fuel supply. For example, instead of developing a performance specification for gasoline and letting the people in that business decide how to meet it there are a multitude of mandated recepies that refineries must adhere to. This creats spot shortages of specific blends and hampers refiners' capabilities to get the best economies of scale out of their existing facilities.
Everyone tends to think of peak oil as a classically shaped bell curve. In fact, I would expect the top to be quite mis-shaped because as price rises more expensive reserves will be produced and more expensive technology will squeeze more out of existing reserves.
Much of the worlds production is from fields that are horribly mismanaged. There seems to be a tendancy for countries to nationalize and kick big oil out, and with that goes the best practices in resevoir maintenance, facilities maintenance, economics, etc. I would propose that if Shell, XOM, et al were allowed to operate some of the major fields (and if economics dictated) production could be doubled in 5 years in places like Iran, Saudi and Venezuela.