Annotations in vertically padded table cells are misplaced, or raise IndexError

[jimbofreedman]

Steps to reproduce

No URL is involved; the input is a self-contained snippet. Two cases, both on inscriptis 2.7.3.

a) valign=bottom — crash

from inscriptis import get_annotated_text
from inscriptis.model.config import ParserConfig

html = ('<table><tr>'
        '<td style="vertical-align:bottom">a1<br><b>a2</b></td>'
        '<td>b1<br>b2<br>b3<br>b4</td>'
        '</tr></table>')

get_annotated_text(html, ParserConfig(annotation_rules={'b': ['b']}))

b) valign=middle — silently wrong offsets

from inscriptis import get_annotated_text
from inscriptis.model.config import ParserConfig

html = ('<table><tr>'
        '<td style="vertical-align:middle">'
        '<span>x1</span><br><span>x2</span><br><span>x3</span></td>'
        '<td>L1<br>L2<br>L3<br>L4<br>L5<br>L6<br>L7<br>L8<br>L9</td>'
        '</tr></table>')

result = get_annotated_text(html, ParserConfig(annotation_rules={'span': ['s']}))
for start, end, label in result['label']:
    print((start, end), repr(result['text'][start:end]))

In both cases the annotated cell is shorter than its row, so height.setter pads it with empty lines.

Expected behaviour

a) Returns normally, with the b annotation slicing to 'a2'.

b) Each of the three span annotations slices to its own content:

(21, 23) 'x1'
(28, 30) 'x2'
(35, 37) 'x3'

Observed behaviour

a) Raises:

  File "src/inscriptis/model/table.py", line 143, in get_annotations
    annotation_lines[no + self.vertical_padding].append(a)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
IndexError: list index out of range

b) No exception, but the second and third annotations point at the bottom padding lines rather than the content:

text  = '    L1\n    L2\n    L3\nx1  L4\nx2  L5\nx3  L6\n    L7\n    L8\n    L9\n'
label = [(21, 24, 's'), (36, 39, 's'), (45, 48, 's')]

(21, 24) 'x1 '
(36, 39) '3  '     <- expected 'x2'
(45, 48) ' L7'     <- expected 'x3'; text from the *other* column

valign=top is unaffected, since vertical_padding is then 0.

Additional information

The multi-line branch of TableCell.get_annotations treats the loop variable no as both a content-line index and a padded-line index.

height.setter pads short cells with empty lines, and width.setter then records line_width from the padded blocks — so line_width carries one zero-length entry per padding line. line_break_pos accumulates over that padded list, which makes no a padded index; self.vertical_padding is then added on top of it:

line_break_pos = list(accumulate(self.line_width))
...
for no, line_break in enumerate(line_break_pos):
    if a.start <= (line_break + no):
        annotation_lines[no + self.vertical_padding].append(a)

Annotation start offsets reference the pre-padding cell content, so no needs to be a content-line index 0..rows-1 for the + vertical_padding shift to be correct. Because it isn't, the top padding is counted twice — overflowing annotation_lines for valign=bottom, and landing on bottom padding lines for valign=middle.

I have a fix and regression tests ready and will open a PR referencing this issue.

Not the same as #92

Worth stating explicitly, since it looks adjacent: this is not the bug in #92. That report's snippet has two equal-height columns, so no padding is applied and its output is unchanged by the fix — #92 is the annotation-span overlap you described in that thread, which needs an output-format change. This one only appears once the columns have different heights.