Stream mode: Phase 4 finale#126
Open
merv1n34k wants to merge 4 commits into
Open
Conversation
Contributor
Author
|
I have also fixed the issue with errors on stream with pipe test - this is due to race conditions (quite noticeable on ubuntu) - I switched to FIFO like in the last 2 tests, now it should be robust. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hey!
I suppose this is the last piece of the stream mode functionality. With this PR (and 3 others) we essentially have a system, where both editor and engine have full push/pull capabilities. This in turn let us create a smart approach for fetching and updating engine requests for tex project (possibly even quite large ones with complex nested inputs). What is more interesting this approach completely eliminates the issue from PR 3, when engine restart was required.
Now, if the editor can detect and provide file(s) with
registercommand to the driver and the engine simply pauses and continue uponopencommand. Like in this scheme:(register "a.tex") (register "b.tex")-- marks both as promisedQ_OPRD "a.tex"-- driver can't find it, file is promised, stores query, doesn't answer yet(request-file "a.tex")-- engine blocks on socket(open "a.tex" "...")-- driver storesedit_data, skips rollbackA_OPEN, continuesQ_OPRD "b.tex"-- same as step 2(request-file "b.tex")-- engine blocks(open "b.tex" "...")-- same as step 4Additionally, I have added timeout for
Makefiletest targets to fix the issue with long running tests, cap on 120s is generous.Report
Problem
Phase 3's
request-fileis non-blocking: the engine fails the missing file immediately withA_PASS, TeX errors out, and the engine terminates. When the editor later provides the file, the engine must fully restart. Every missing file costs one restart.Solution
(register "path")lets the editor pre-declare files it will provide. When the engine hitsQ_OPRDfor a registered-but-not-yet-provided file, the driver doesn't answer - it stores the query, emitsrequest-file, and the engine blocks on the socket. When the editor sendsopen, the driver resolves the stored query in the sameengine_stepcall. The engine continues as if the file was always there. Zero restarts.How it works
Details
The editor sends
(register "ch1.tex"), which setspromised = trueon the file entry. When the engine later asks forch1.texviaQ_OPRDandlookup_pathfails, the driversees e->promisedand instead of answeringA_PASS, stores the query in adeferredstruct and breaks without responding. The engine blocks.The driver emits
(request-file "ch1.tex"). The editor responds with(open "ch1.tex" "content...").interpret_openstoresedit_databut skipsnotify_file_changesfor promised files - calling it would trigger a rollback that kills the blocked engine.On the next
engine_step, the deferred check at the top seesdeferred.activeandedit_datapresent. It resetsseen = -1, replaysanswer_querywith the stored query, and flushes.Q_OPRDnow findsedit_dataand respondsA_OPEN. The engine resumes.What changed
Details
state.h-- addedbool promisedtofileentry_teditor.h/editor.c-- parse(register "path")asEDIT_REGISTERengine_tex.c--deferredstruct ontex_engine, defer logic inQ_OPRDhandler, resolution check at top ofengine_step, cleared on rollback and preparemain.c—interpret_registersetspromised = true,interpret_openskipsnotify_file_changesfor promised files receiving first content,EDIT_REGISTERcase in command dispatchEDITOR-PROTOCOL.md— documented registerP.S. I have built this PR on top of #125 , when we will finish with it I'll rebase this one to make a clean git history.