Why use armeabi-v7a code over armeabi code?

Depends on what your native code does, but v7a has support for hardware floating point operations, which makes a huge difference. armeabi will work fine on all devices, but will be a lot slower, and won't take advantage of newer devices' CPU capabilities. Do take some benchmarks for your particular application, but removing the armeabi-v7a binaries is generally not a good idea. If you need to reduce size, you might want to have two separate apks for older (armeabi) and newer (armeabi-v7a) devices.

EABI = Embedded Application Binary Interface. It is such specifications to which an executable must conform in order to execute in a specific execution environment. It also specifies various aspects of compilation and linkage required for interoperation between toolchains used for the ARM Architecture. In this context when we speak about armeabi we speak about ARM architecture and GNU/Linux OS. Android follows the little-endian ARM GNU/Linux ABI.

armeabi application will run on ARMv5 (e.g. ARM9) and ARMv6 (e.g. ARM11). You may use Floating Point hardware if you build your application using proper GCC options like -mfpu=vfpv3 -mfloat-abi=softfp which tells compiler to generate floating point instructions for VFP hardware and enables the soft-float calling conventions. armeabi doesn't support hard-float calling conventions (it means FP registers are not used to contain arguments for a function), but FP operations in HW are still supported.

armeabi-v7a application will run on Cortex A# devices like Cortex A8, A9, and A15. It supports multi-core processors and it supports -mfloat-abi=hard. So, if you build your application using -mfloat-abi=hard, many of your function calls will be faster.

Instead of having a fat APK file, I would like to use just the armeabi files and remove the armeabi-v7a folder.

The opposite is a much better strategy. If you have minSdkVersion to 14 and upload your apk to the play store, you'll notice you'll support the same number of devices whether you support armeabi or not. Therefore, there are no devices with Android 4 or higher which would benefit from armeabi at all.

This is probably why the Android NDK doesn't even support armeabi anymore as per revision r17b. [source]