Having to run Flutter Clean and remove framework everytime I switch building from Physical iOS and Simulator
Xcode 11.4 changed the way frameworks are linked and embedded. Flutter v1.15.3 and later will automatically migrate your Xcode project.
In the meantime, to fix it manually, follow the following instructions: https://flutter.dev/docs/development/ios-project-migration