HERE document mystery: bug or no bug

[jghub]

not sure whether this could be an issue, so I first ask here. consider a script file holding

[[ -n isloaded ]] && return
function mydoc {
cat <<-HERE | nroff -man
.\"nroff -man source document (some 100 lines)
HERE
}
# many more function definitions
isloaded=1

so this defines assorted functions when sourced for first time, but returns immediately if file is sourced 2nd time. the file is under development and changing frequently right now.
the problem I see is that after sourcing it for 1st time, mydoc for some time correctly shows the manpage (the content of the HERE document) but at some point, sometimes, it stops doing that (possibly after me having edited the file, can't say for sure).
issuing typeset -f at this point shows the function body with an empty here document like so:

cat <<-HERE | nroff -man
HERE

Q1: why/how can here doc content vanish just so from the function definition?

if I do at that point source myfile for a second time and repeat typeset -f mydoc the function is back to normal content (full here doc part of function definition).

Q2: why does the here doc content reappear despite script returning in line 1 and definitely not defining mydoc a second time (or so I would think)?

[JohnoKing]

I managed to reproduce this behavior on the dev branch, but it seems horribly inconsistent. In my case the function lost its heredoc after being run thrice, then I had to source the file twice to get the function's heredoc to return to normal. It seems like a bug to me (maybe memory corruption?).

EDIT: Now it won't reproduce at all. 99% certain it's a bug. ASan isn't throwing any errors though, so maybe it isn't a memory bug.

[jghub]

yes, I forgot to mention this: seen with Version AJM 93u+m/1.1.0-alpha+a9b0e3a5 2026-02-09

[jghub]

in the meantime I asked gemini (and I have no idea whether the reply is pure halluzination):

When ksh93 defines a function containing a Here-Doc, it doesn't always store that Here-Doc as a simple string in the environment. To save memory, ksh93 often keeps a pointer to the original script file or a temporary memory buffer (a "shmem" or "vnode" structure) where the Here-Doc content lives. This is a known, long-standing quirk/bug specifically associated with ksh93. It relates to ksh93's "virtual node" (vnode) and "file-mapping" optimizations. The problem is almost certainly related to the file being modified, moved, or accessed by another process after the initial sourcing, but ksh93 is uniquely sensitive to this because of how it handles "inert" files.

does that make any sense to you? "long-standing quirk" at least sounds as if it has been reported in the past and the report was picked up by the LLM (whether factually correct: different question...).

[JohnoKing]

Here documents utilize tmp files via SFIO's sftmp:

ksh/src/cmd/ksh93/sh/io.c

Lines 1581 to 1596 in 71aff4e

	/*
	* Create a tmp file for the here-document
	*/
	static int io_heredoc(struct ionod *iop, const char *name, int traceon)
	{
	Sfio_t *infile = 0, *outfile, *tmp;
	int fd;
	Sfoff_t off;
	if(!(iop->iofile&IOSTRG) && (!sh.heredocs || iop->iosize==0))
	return sh_open(e_devnull,O_RDONLY);
	/* create an unnamed temporary file */
	if(!(outfile=sftmp(0)))
	{
	errormsg(SH_DICT,ERROR_system(1),e_tmpcreate);
	UNREACHABLE();
	}

I don't think it does that to 'save memory'. I also fail to see why that temp file would be modified, since it's not a pointer to the original script file on the disc (so mere modifications to the original file on the disc shouldn't affect what ksh is retrieving from RAM, /dev/shm or /tmp).

Perhaps the AI is confusing ksh's here document handling with how the 1.0 branch handles typeset -f (1.0 uses the file on the disc, whilst the dev branch uses sh_deparse() to generate the script using the parser tree in memory).