@FindBy doesn't care if the element isn't there when I call initElements
You don't. The PageFactory does lazy initialization, and that's how it's designed.
Consider a Page Object where certain of your elements don't exist on the page until some action is taken. Since Page Objects are intended to encapsulate business logic and not just the elements on the page, this is a perfectly logical scenario. In that case, your initElements() call would fail on page object initialization every single time, and not give you the chance to call the business logic method that would cause the element to appear.
It's possible that the PageFactory will not work for you if this is a requirement for your test framework. In that case, you'd do well to construct your own implementation.