configure "--prefix" option for cross compiling

Yes you are right, --prefix is the path for working environment. Just use --prefix=/usr. You can check in which folder path make install command will install your binary by installing in DESTDIR. For example if you use --prefix=/usr and make install DESTDIR=/home/me/arm/build/target_fs, then the binaries will be installed in the folder /home/me/arm/build/target_fs/usr. And If you just run make install, then the binary will be installed in your prefix i.e. in "/usr".

As you are cross-compiling, I think it doesn't matter which prefix you use, because anyways you will be installing in DESTDIR and then copying the binary files manually to your target.

As you may found:

--prefix=dirname Specify the toplevel installation directory. This is the recommended way to install the tools into a directory other than the default. The toplevel installation directory defaults to /usr/local.

As far as I understand you are trying to compile a compiler for some target.

In this case prefix will specify directory when the compiler will be installed after make install command on build machine. After it you can take a compiler there.

. Should I use --prefix=/home/me/arm/build/target_fs/usr or just --prefix=/usr and then make install DESTDIR=/home/me/arm/build/target_fs?

In your case prefix command have no sense. Because you are copy binaries by hands.

Also you can find all other info on GCC official site:https://gcc.gnu.org/install/finalinstall.html