mdsip: GetMany batch containing a query that succeeds with a NULL result crashes the server worker (NULL deref in getManyObj)

[dgarnier]

Affiliation
OpenStar Technologies (working against OpenStar's own copy of the LDX archive)

Version(s) Affected
Server Version: Alpha 7.160.1 (alpha_release-7-160-1). The faulty code path is unchanged on current alpha HEAD (726e945); the code references below are pinned to the release tag.
Client: mdsthin 1.7.0 (the client only observes the crash; the fault is server-side and client-agnostic — the same TDI expressions are sent by the MDSplus python Connection.getMany())

Platform(s)
Client: macOS. Server: Linux (OpenStar mdsip server hosting a copy of the LDX archive).

Installation Method(s)
Not relevant to the code path below (reproducible from source).

Describe the bug
If a GetMany batch contains a query that succeeds but evaluates to a NULL/empty result, the mdsip server worker handling the connection dies mid-reply (the reply never arrives; the TCP connection is left half-open). In one instance the whole mdsip service was wedged afterwards and stopped accepting new logins until restarted.

Queries that fail inside a GetMany are handled correctly (an "error" entry comes back). It is specifically a successful evaluation yielding a null result that kills the worker. The same expression sent through a plain get() harmlessly returns nil/None.

Concrete example: a node whose record is a Range with null begin/end, e.g. \LDX::TOP.HARDWARE:DCLOCK_1.CHANNEL_1:CLOCK in tree ldx, shot 100805020, whose record is * : * : 5000E-9. Then

DATA(BEGIN_OF(GETNCI(\LDX::TOP.HARDWARE:DCLOCK_1.CHANNEL_1:CLOCK,"RECORD")))

evaluates successfully to nothing — fine via get(), fatal inside GetMany.

To Reproduce

import mdsthin  # same behavior expected with MDSplus.Connection

c = mdsthin.Connection('SERVER')
c.openTree('ldx', 100805020)  # any LDX shot with the DCLOCK_1 hardware should do

EXPR = 'DATA(BEGIN_OF(GETNCI(\\LDX::TOP.HARDWARE:DCLOCK_1.CHANNEL_1:CLOCK,"RECORD")))'

# 1. Plain get: returns None, no problem
print(c.get(EXPR))

# 2. Same expression inside GetMany: server worker dies, no reply ever arrives
gm = c.getMany()
gm.append('begin', EXPR)
gm.execute()   # hangs forever / connection dead

Any node whose record is a Range/expression with a missing part should do; presumably any expression that evaluates successfully to a NULL descriptor (e.g. DATA(*)) triggers it too, though we only verified against tree nodes like the one above.

Expected behavior
The result dictionary should carry a nil "value" for that query (so the client's GetMany.get(name) returns None, matching plain get()), or at worst an "error" entry — never a worker crash.

Where it crashes (from reading alpha_release-7-160-1; identical on current alpha @ 726e945 apart from a two-line offset in mdsdataobjects.cpp)

GetManyExecute (tdi/remote/GetManyExecute.fun) → GetManyExecute() in mdsobjects/cpp/mdsdata.c#L914-L926 → getManyObj() in mdsobjects/cpp/mdsipobjects.cpp#L122-L253:

1. For each query, getManyObj calls execute(expr) / executeWithArgs(...) (mdsipobjects.cpp#L238).
2. executeWithArgs (mdsdataobjects.cpp#L945-L981) calls compData->data(), which evaluates with a success status but an empty XD when the result is nil (mdsdataobjects.cpp#L409-L421).
3. convertFromDsc then returns NULL (mdsdata.c#L264-L269) — so execute() returns a NULL Data* without throwing.
4. Back in getManyObj, the NULL is stored as the answer: answDict->setItem(valueKey.get(), currAnsw) (mdsipobjects.cpp#L240-L241).
5. Dictionary::setItem unconditionally calls data->incRefCount() (include/mdsobjects.h#L2908-L2929) — NULL dereference, SIGSEGV in the worker.

Failing queries never reach that line — the MdsException is caught and turned into an "error" entry (mdsipobjects.cpp#L243-L248) — which is exactly the asymmetry we observe (failures fine, successful-null fatal).

Possible fixes: guard currAnsw == NULL in getManyObj (store nothing, or an explicit nil, under "value"), and/or make Apd::setItem/appendDesc NULL-safe.

Additional context

• Observed while bulk-archiving shots from OpenStar's copy of the LDX tree (batched metadata reads). Worked around client-side by keeping nullable expressions out of GetMany batches.
• A client-side consequence of the worker dying mid-reply: mdsthin clients spin at 100% CPU on the half-open socket — reported separately as Connection recv loops spin forever at 100% CPU when the peer closes the socket (recv_into() == 0 never checked) mdsthin#8 (recv loop doesn't detect EOF).
• Possibly separate, mentioned for completeness: a GetMany whose successful result dictionary is very large (a segmented photodiode signal, ~58 MB data + ~115 MB timebase in one reply) also killed the worker. If desired we can file that separately once we can characterize it better.
• Claude (Anthropic's AI assistant) helped identify the crash path in the source and document this report.

[mwinkel-dev]

Hi @dgarnier -- Thanks for the bug report.