Edge case with ref forwarding doesn't throw an error when needed

[kripod]

Description

Please see my tweet for further details.

Reproduction

Sandbox with all the information required: https://codesandbox.io/s/typescript-as-prop-with-react-polymorphic-box-vr269?file=/src/App.tsx