Partially saturated pixels flagged as DO_NOT_USE but not as SATURATED?

[schlafly]

I'm concerned by the kind of flagging happening in this line:

stcal/src/stcal/saturation/saturation.py

Line 176 in f7c6d19

flagarray = (partial_sat * dnu).astype(np.uint32)

I understand what is happening to be:

• saturated resultants are flagged as saturated, but not as do not use
• partially saturated resultants (i.e., some of the reads are saturated, others aren't) are flagged as DO_NOT_USE, but not as SATURATED.

I think the pipeline should be flagging both of these as SATURATED and neither of them as DO_NOT_USE. Do people feel differently?

For pixels we've talked about having special behavior surrounding partially saturated ramps:

• all groups saturated -> DO_NOT_USE & SATURATED
• some but not all groups saturated -> SATURATED, not DO_NOT_USE
  which I'm partial to, but I don't think is relevant here.

[braingram]

@schlafly would you make an AL ticket for this and ping @tapastro ?

[schlafly]

https://jira.stsci.edu/browse/AL-1035

[tapastro]

On first read, I agree with your assessment. I'll come back to this to make sure I understand intent, but I'd be happy see a PR tested that moves the flag assigned here from DNU -> SATURATED.

[schlafly]

This was done intentionally as part of #321 , but I don't understand the rationale there yet.

[schlafly]

@drlaw1558 , you mention in JP-3805 that you intend a better longer term fix. I just consider it surprising that the saturation step has the behavior that it mostly flags saturated pixels but flags DO_NOT_USE and not SATURATED for a small number of pixels. IMO this isn't particularly clean; these DO_NOT_USE-only groups don't really have anything saying why they got masked, making it hard to track down the cause. I think it's more natural for them to be flagged SATURATED and possibly introduce a new flag if we need to distinguish between partial and full saturation of an individual read (though I would have said we do not need to make that distinction).

Are there alternatives we should discuss to resolve JP-3805 by another approach, or should we at least better document this behavior in the saturation flagging?