New leveraging command features

[wadimw]

This PR adds the following features:

• new leveraging mode based on the string ID (name)
• option to force preserve original status after leveraging
• option to avoid overwriting existing translations during leveraging

Leveraging by name

Added a new leveraging mode NAME which matches based on the string ID (name) rather than its source content (as EXACT would).

This is useful in cases where some strings have slight modifications to their content, but are functionally equivalent, e.g. spacing differences in ICU plural syntax like leveraging from

name	element_count
source	{count, plural, other {# elements}}
target	{count, plural, one {# element}, few {# elementy}, many {# elementów}, other {# elementów}}

to

name	element_count
source	{ count, plural, other {# elements} }
target	{ count, plural, one {# element}, few {# elementy}, many {# elementów}, other {# elementów}}

⚠️ There are no safety checks to verify similarity between sources - only the string ID is compared.

Force preserve original status

Currently, when leveraging, status of the leveraged string is sometimes downgraded to NEEDS_TRANSLATION so that they can be later verified by linguists - this is determined based on the match precision and uniqueness. A match is 'unique' when exactly one candidate text unit matched (no ambiguity). A match is 'high-precision' when matched on both name and content (or full MD5, i.e. name + content + comment). By default, Mojito preserves status of the leveraged string only when the match is both unique and high-precision. Matches on name-only or content-only are always downgraded (even if unique).

In some cases (like when migrating a handful of strings to another project with a different naming scheme), it's desirable to preserve the original status of the target string.

Added --preserve-status option that can be used to control this status downgrade safeguard. This option accepts the following values:

Mode	Risk	Description
PRECISION (default)	Low	(unchanged) Preserve status only when the match is both unique and high-precision. Matches on name-only or content-only are always downgraded, even if unique.
UNIQUE	Medium	Preserve status for any unique match, regardless of its precision. Useful when e.g. migrating between naming schemes.
ALL	High	Always preserve the original status, even for ambiguous matches. Match will be arbitrarily picked from the available candidates.

Avoid overwriting existing translations

Currently, when leveraging, Mojito overwrites all units in the target repository regardless of their status (unless filtered out via name regex).

More often than not, it's desirable to only leverage a subset of strings rather than all available matches, e.g. only fill strings that don't have any translations yet, or only overwrite with translations that have a higher status.

Added --overwrite-mode option that can be used to control this behavior. This option accepts the following values:

Mode	Behavior
ALL (default)	(unchanged) Always overwrite, regardless of the current status.
HIGHER_STATUS	Overwrite only when the candidate's status is higher (e.g. APPROVED overwrites REVIEW_NEEDED).
HIGHER_OR_EQUAL_STATUS	Same as HIGHER_STATUS but also overwrites when statuses are equal.
FOR_TRANSLATION	Leverage into strings with status TRANSLATION_NEEDED or no translation at all.
NONE	Never overwrite; only leverage into locales that have no translation at all.

[ehoogerbeets]

What does "unique" mean in this case. I'll probably find out more when I read the rest of this PR, but some just looking at the help like I am now is going to wonder what that means. Does that mean the text, the id, and the asset, or does it mean just the text? Does it mean "exact" or does it mean "if there aren't multiple source units that match the target"... also what happens if it is not unique? Or if all of the source matches have the same status?

[wadimw]

TODO: improve description, consider rename and/or flip condition (so that the flag is something like "downgrade status" with default "based on leverager precision" medium risk "only downgrade for non-unique strings" and high risk "never")

[wadimw]

Updated

[ehoogerbeets]

Uh... a little confusing. Are you saying that the leveraging doesn't happen if the target's current translation for a locale doesn't have one of these statuses? What does happen to it then? So, conversely, the leveraging only happens when the current translation of the unit for the target locale has one of the given statuses. Do we need this? I thought we only leveraged for units that are untranslated or have translation needed status? Do we need to overwrite the current translations that have review needed or approved status?

What I was advocating before was a single command-line flag:

--set-status=[COPY,NEEDS_TRANSLATION,NEEDS_REVIEW,APPROVED]

where COPY means "copy the status from the source unit" and the other ones mean "override the status with the given status". I imagined that translations would only be copied from units with needs translation or approved status to units with no translation or needs translation status. If the target already has "needs review" or "approved" status, then we wouldn't need to overwrite that. I understood leveraging to mean "copy missing translations from some other place that does have translations". Is that not the case?

[wadimw]

usecases for filtering by status:

1. fill-only - only paste translations where there are none at all
2. upgrade-only - overwrite if you have translation with a higher status (needs translation -> needs review, needs review -> approved etc)
3. upgrade-or-override-same - overwrite if you have translation with a higher or same status (replace existing translations even though they may have equal status)
4. always-override - flag in all caps that this is the default leveraging mode which is a bit unsafe

[wadimw]

Updated

[ehoogerbeets]

I can't really see any problems with this, but I'm not that familiar with how mojito works anyways. The problem is that I can't test this on my own. Is it deployed to mojito-dev? Maybe we have to merge it first, then deploy to mojito-dev and test it out with "real"-ish tests, and then when we think it works, we deploy to prod mojito.