SQL set floating point precision SQL set floating point precision sql sql

SQL set floating point precision

In TSQL, you can specify two different sizes for float, 24 or 53. This will set the precision to 7 or 15 digits respectively.

If all you want to do is truncate to a set number of decimal places, you can use ROUND, ie:

ROUND(AVG(CAST(e.employee_level as float)), 3)

As a general rule, you can't specify the number of digits after the decimal point for a floating-point number. Floating point data types store the closest floating-point approximation to any given value. The closest floating-point approximation is unlikely to have the number of digits you want. Although you might be able to suppress every digit after the third one, that will only change the appearance of the value, not the value itself.

Integers are a different story. An integer--stored, converted, or cast to a floating-point data type--will be stored exactly over a large range. Floating-point data types don't have to store any fractional units for integers.

I'd suggest, though that the best practice for you is to

  • avoid casting integers to floating-point if you don't need fractional units, or
  • cast integers to decimal or numeric if you do need fractional units, or
  • handle display issues entirely in application code.