Skip LpaiPartitionFallbackSupport when compiler_specs is None (forward fix for D108707519) (#20324)

Skip LpaiPartitionFallbackSupport when compiler_specs is None (forward fix for D108707519) (#20324) #4848