Architecture of a secure application that encrypts data in the database
This is a customer's requirement.
Customer's requirements must be projected onto the real world.
If your application can read some business data, and if the aggresor takes control of your application, then the agressor can read that business data.
Assymetric cryptography won't do magic.
The place to store the private key is with the client, and decrypt it only on the client. The data would need to be at no time decrypted, read or passed through the server in unencrypted form.
I am not sure about the database options, but it's worth to have a look at Oracle Advanced Security(OAS). But, the key is not stored inside the database but in Oracle wallet (OS managed), as far as I can see, compromising this is hard.
OAS supports encryption at tablespace level and at column level. All these, it claims to be performing with no overhead.