Turns out you can remove the module babel-plugin-emotion and use macros provided by emotion.
Use the following import
import styled from "@emotion/styled/macro";
That's it and you are good to go.