change(web): track right-hand split index for input source of tokenized transforms 🚂

[jahorton]

After much examination, I believe that in order to identify use of subsets of user input that converge to a common prefix for future extensions, we need to track and reference the right-hand index used to split input transforms when such splits occur. This will also assist with identification of cases where a previously-split input is to be remerged; the earlier component's "split" index (.end) will match the later component's .start index for such cases.

To clarify the requirements for considering subsets of user input to converge:

1. The represented text for the token (and SearchSpace) must have the same codepoint length.
2. The underlying InputSegment properties for each input must cover exactly the same range.
   1. They should share the exact same set of .transitionId entries.
   2. For each .transitionId entry, the .start and .end values must match precisely.

To restate this more plainly:

1. They should represent the same total amount of text - as future delete-left operations must have the same net effect, regardless of how large the total deletion effect is.
2. They should represent exactly the same keystrokes, and the same portions thereof if only parts of certain keystrokes apply.
   • Applying a suggestion based on a tokenization (and its SearchSpace) directly replaces this range of the input; this range should be clearly defined with consistent boundaries.
   • If this process only replaces part of a keystroke, knowing this fact may prove useful for other adjustments to be made... such as increasing the suggestion's range of coverage (due to affecting multiple tokens).

Build-bot: skip build:web
Test-bot: skip

[keymanapp-test-bot]

User Test Results

Test specification and instructions

User tests are not required

Test Artifacts

• Web
  • KeymanWeb Test Home - build : all tests passed (no artifacts on BuildLevel "build")