I'm guessing that you have the --color=always option to ls set, either through an alias, functions or the LS_COLORS environment variable and ls is sending color directives to a non-terminal (that is, your pipe to less).
--color=always
ls
LS_COLORS
less
Use less -R or set the LESS environment variable to -R.
less -R
LESS
-R
What you're seeing are ANSI escape sequences for setting colors. Run ls --color=no.
ls --color=no