Skip to content

Linux SIGSEGV in idLexer::CreatePunctuationTable caused by signed char indexing #24

Description

@bobmoser76

Linux SIGSEGV in idLexer::CreatePunctuationTable caused by signed char indexing

Description

OpenPREY consistently crashes during startup on CachyOS Linux while initializing declarations.

I tracked the crash with GDB to src/idlib/Lexer.cpp, specifically idLexer::CreatePunctuationTable().

Environment

  • CachyOS x86_64
  • GCC 16.2.1
  • Meson 1.12.0
  • Ninja 1.13.2
  • OpenPREY main branch
  • PREY 2006 Linux installation

Crash

Startup reaches:

----- Initializing Decls -----
Parsing Guides...
Found 0 guides...

and then crashes with SIGSEGV:

Thread 1 "OpenPrey-client" received signal SIGSEGV, Segmentation fault.

idLexer::CreatePunctuationTable(
    this=0x7fffffffc9c0,
    punctuations=default_punctuations
)
at ../src/idlib/Lexer.cpp:550

Backtrace:

#0  idLexer::CreatePunctuationTable()
    at ../src/idlib/Lexer.cpp:550
#1  idLexer::SetPunctuations()
    at ../src/idlib/Lexer.cpp:665
#2  idLexer::idLexer()
    at ../src/idlib/Lexer.cpp:2172
#3  idDeclFile::LoadAndParse()
    at ../src/framework/DeclManager.cpp:786
#4  idDeclManagerLocal::RegisterDeclFolder()
    at ../src/framework/DeclManager.cpp:1525
#5  idDeclManagerLocal::Init()
#6  idCommonLocal::InitGame()
#7  idCommonLocal::Init()
#8  main()

Cause

At the crash, GDB showed:

i = 52

newp->p = "�"

(unsigned char)newp->p[0] = 0xef
(int)newp->p[0] = -17

The code at line 550 uses:

idLexer::punctuationtable[(unsigned int) newp->p[0]]

On this platform, char is signed. Therefore the byte 0xef is interpreted as -17 before being converted to unsigned int.

Instead of indexing the 256-entry table with:

239

it effectively attempts to index it with:

4294967279

resulting in the SIGSEGV.

There are three occurrences of this expression in CreatePunctuationTable():

Lexer.cpp:550
Lexer.cpp:558
Lexer.cpp:570

Tested fix

Changing all three from:

(unsigned int) newp->p[0]

to:

(unsigned char) newp->p[0]

fixes the crash.

For example:

for (n = idLexer::punctuationtable[(unsigned char) newp->p[0]];
     n >= 0;
     n = idLexer::nextpunctuation[n]) {

and the two assignments later in the function likewise use:

idLexer::punctuationtable[(unsigned char) newp->p[0]] = i;

Result

After rebuilding OpenPREY with those three changes, PREY successfully gets past declaration initialization and launches into the game.

The fix was tested on the same CachyOS/GCC 16.2.1 system that consistently reproduced the crash before the change.

I can submit the three-line change as a PR if desired.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions