Wrap the deduplication engine in a load of extra logic to support things like
- conditional lookups (only lookup if field X exists - note there's an undesirable conditional logic built into the dedup service currently .. i tihnk it will discard any objects that don't have a field, even it finalize is set...)
- field name changes (eg lookup hash in current bucket as md5sum in another bucket)
- a library of automatic operations (eg tag with X, insert object in Y, filter if not present, etc)
Wrap the deduplication engine in a load of extra logic to support things like