Where can I get information about GPU floating point precision? Is it IEEE-compliant?
Current NVIDIA graphics hardware provides 32-bit (s23e8) floating point arithmetic that is very similar to the arithmetic specified by the IEEE 754 standard, but not quite the same. The storage format is the same (see above), but the arithmetic might produce slightly different results. For example, on NVIDIA hardware, some rounding is done slightly differently, and denormals are typically flushed to zero.
ATI X1k series GPUs also provide 32-bit floating point arithmetic, older ATI hardware does all its floating point arithmetic at 24-bit precision (s15e8), even though it stores values in the IEEE standard 32-bit format.
Both NVIDIA and ATI provide a "half-precision" (http://en.wikipedia.org/wiki/Half_precision) 16-bit (s10e5) floating point storage format; some NVIDIA GPUs can perform half-precision arithmetic more quickly than single-precision arithmetic.
No GPU currently provides double-precision storage or double-precision arithmetic natively in hardware. There are several ongoing efforts to emulate double precision through a single-double approach (doubling the mantissa) and CPU-GPU interplay, but apart from a very preliminary special case paper about iterative solvers (http://www.mathematik.uni-dortmund.de/~goeddeke/pubs/asim2005.html), nothing has been published about double precision on the GPU.
For a look at the exact precision of various 32-bit GPU floating point operations (and an example program to test your own hardware), refer to Karl Hillesland's whitepaper GPU Floating-Point Paranoia (http://www.cs.unc.edu/~ibr/projects/paranoia/).
Excellent find, P9ing. This is exactly what I was referring to when I said that GPU's aren't suitable for real floating point intensive calculations. They're taking shortcuts all over the place to get their speed. It may not matter much in a video game graphics environment about that precision, but it does if you're making movie graphics or doing anything remotely scientific. Hell, it doesn't even have to be scientific, even the precision available here isn't suitable for regular financial calculations. They're using not just single-precision here, but they're even shortcutting that to 3/4-precision or even 1/2-precision!