Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 28 additions & 30 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,14 @@ Welcome to the Vipr codebase! This document provides a comprehensive guide for f

## 1. System Overview

Vipr is a statically-typed, block-scoped, ahead-of-time (AOT) compiled programming language. The compiler is written in Rust and uses a multi-pass structure that translates Vipr source code into C code, which is then compiled into native machine code using `gcc`.
Vipr (v0.2.0) is a statically-typed, indentation-scoped, ahead-of-time (AOT) compiled programming language. The compiler is written in Rust and uses a multi-pass structure that translates Vipr source code into C code, which is then compiled into native machine code using `gcc`.

### Compilation Pipeline

```mermaid
graph TD
A[Vipr Source Code .vipr] --> B[Lexer]
B -->|Token Stream| C[Parser]
B -->|Token Stream w/ Indents| C[Parser]
C -->|Abstract Syntax Tree AST| D[Semantic Analyzer]
D -->|Validated AST| E[Code Generator]
E -->|C Source Code| F[Native Compiler GCC]
Expand All @@ -27,29 +27,27 @@ graph TD
Here is the organization of the codebase:

* **[README.md]**: User-facing Vipr Language Guide, describing variables, control flow, functions, types, and loops.
* **[docs/Vipr.md]**: Official Vipr Programming Language Specification v0.1, containing the formal EBNF grammar.
* **[docs/VIPR2.md]**: Official Vipr Programming Language Specification v0.2.0, containing the formal EBNF grammar.
* **[Cargo.toml]**: Cargo configuration. Notice that there are no external dependencies; the compiler is implemented using only the Rust standard library.
* **`src/`**: The Rust source code of the compiler.
* **[src/main.rs]**: Orchestrates the compilation phases: arguments parsing, reading files, lexical analysis, parsing, semantic checking, C code generation, GCC invocation, and cleaning up.
* **`src/lexer/`**: Lexical analysis.
* **[src/lexer/token.rs]**: Defines the [`Token`] enum representing Vipr's lexicon.
* **[src/lexer/mod.rs]**: The [`Lexer`] struct that tokenizes raw bytes, handles whitespace/comments, and processes escape sequences.
* **[src/lexer/token.rs]**: Defines the [`Token`] enum representing Vipr's lexicon. Includes structural tokens (`Newline`, `Indent`, `Dedent`).
* **[src/lexer/mod.rs]**: The [`Lexer`] struct that tokenizes raw bytes, manages an indentation stack for Python-like scoping, and handles escape sequences.
* **`src/parser/`**: Syntax analysis.
* **[src/parser/ast.rs]**: Definition of AST nodes ([`Program`], [`Decl`], [`Stmt`], [`Expr`], and [`Literal`]).
* **[src/parser/mod.rs]**: The recursive-descent [`Parser`] with precedence climbing for operators.
* **[src/parser/mod.rs]**: The recursive-descent [`Parser`] that strictly enforces block structures via `Indent`/`Dedent` matching and lookaheads for statement disambiguation.
* **`src/semantic/`**: Static semantics and type checking.
* **[src/semantic/symbol_table.rs]**: The block-scoped [`SymbolTable`] tracking variable and function signatures using a stack of hash maps.
* **[src/semantic/mod.rs]**: The [`SemanticAnalyzer`] that performs symbol resolution, type mismatch verification, main function requirement checks, and constant reassignment checks.
* **[src/semantic/mod.rs]**: The [`SemanticAnalyzer`] that performs a two-pass symbol resolution, type mismatch verification, main function requirement checks, and constant reassignment checks.
* **`src/codegen/`**: Code generation.
* **[src/codegen/mod.rs]**: The [`CodeGenerator`] which generates C source code from the checked AST, translating primitive types and loops, and injecting compiler macros for I/O functions.
* **`src/utils/`**: Placeholder directory.
* **[src/utils/mod.rs]** and **[src/utils/error.rs]**: Both are currently empty files ready for future shared compiler utility functions or error reporting systems.
* **[src/codegen/mod.rs]**: The [`CodeGenerator`] which generates C source code from the checked AST. It translates primitive types, safely evaluates multiple-assignments into temporary variables to prevent side effects, and maps Vipr structures into valid C11 syntax.

---

## 3. Vipr Language Specification Details

Vipr has a strict and clean feature set:
Vipr v0.2.0 has a strict, clean, and whitespace-sensitive feature set:

### Primitive Types
* `int`: 32-bit signed integer (mapped to C `int32_t`).
Expand All @@ -60,34 +58,34 @@ Vipr has a strict and clean feature set:
* `void`: Empty return type.

### Grammar & Structural Quirks
1. **Mandatory Semicolons**: Every statement (except block statements themselves) must end with a semicolon `;`.
2. **Variable Declarations**: No type inference (`int x = 10;`, not `x := 10;` or `var x = 10;`).
3. **No Multiple Declarations**: Only one variable declaration is allowed per line.
4. **Strict Scope**: A block `{}` defines its own scope. Inner declarations can shadow outer declarations, but cannot be accessed outside the block.
1. **Python-style Indentation**: Blocks are defined by colons `:` and strict indentation rules. The Lexer pushes and pops `Indent` and `Dedent` tokens natively based on spaces (tabs are illegal).
2. **Variable Declarations**: `let x: int = 10` for variables, `const y: float = 3.1` for immutable constants. No type inference.
3. **Multiple Declarations & Assignments**: Vipr allows `let a, b: int = 1, 2`. Reassignment `a, b = expr1, expr2` evaluates the right-hand expressions into temporary memory *before* binding them to the left side, guaranteeing safe swaps like `a, b = b, a`.
4. **Strict Scope**: An indented block defines its own scope. Inner declarations can shadow outer declarations, but cannot be accessed outside the block.
5. **No Increment/Decrement Operators**: The `++` and `--` operators do not exist. Increments must be written explicitly as `i = i + 1`.
6. **Conditionals**: Conditions in `if` and `while` must evaluate exactly to the `bool` type. No implicit truthiness (e.g. `if (1)` is a compile error).
7. **The `main` Function**: Execution starts at the `main` function. It must be declared as either returning `void` (implicitly when omitted) or `int`.
6. **Conditionals**: Conditions in `if`, `elif`, and `while` must evaluate exactly to the `bool` type. No implicit truthiness (e.g. `if 1:` is a compile error).
7. **The `main` Function**: Execution starts at the `main` function. It must be declared as returning `void` (`def main() -> void:`). The Code Generator automatically maps this to the required `int main(void)` for GCC.

---

## 4. Compiler Internals

### The I/O Subsystem (`print` and `scan`)
In Vipr, `print(expression)` and `scan(variable)` are built-in functions. They do not exist in the user symbol table but are handled specially:
### The I/O Subsystem (`print` and `input`)
In Vipr, `print(expr1, expr2)` and `input(var1, var2)` are built-in macros. They do not exist in the user symbol table but are handled specially:
1. **Semantic Analysis**:
- `print` expects exactly one argument of any non-void type.
- `scan` expects exactly one variable identifier that is mutable (not a constant).
- `print` accepts variable arguments of any non-void type.
- `input` accepts variable arguments, requiring them to be mutable identifiers (not constants).
2. **Code Generation**:
The generator includes a C boilerplate utilizing C11 `_Generic` macros:
- `vipr_print(x)` resolves the format specifier at compile-time: `%d\n` for `int`, `%f\n` for `float`, `%c\n` for `char`, `%s\n` for `const char*` or `bool` (printing `"true"` or `"false"`).
- `vipr_scan(x)` similarly resolves `%d`, `%f`, `%c`, or `%s` depending on the variable's type.
The generator resolves types dynamically using the AST symbol tables and generates standard C library calls natively:
- `print(x, y)` translates to multiple discrete `printf()` calls sequentially based on the type of the AST expression (`%d` for `int`, `%f` for `float`, `%s` for string, etc.).
- `input(a, b)` calculates a unified format string (e.g. `"%d %f"`) and injects it into a single `scanf("%d %f", &a, &b)` call.

### Compilation Mechanics (`src/main.rs`)
When you invoke the compiler on a source file `test.vipr`:
1. It parses arguments, reads the file, and runs the tokenization -> parsing -> type-checking.
2. If successful, it writes C code into `temp_vipr_output.c`.
3. It spawns the command `gcc temp_vipr_output.c -o test` to compile it natively.
4. Finally, it cleans up and deletes `temp_vipr_output.c`.
4. Finally, it cleans up and silently deletes `temp_vipr_output.c`.

---

Expand All @@ -98,20 +96,20 @@ To compile a Vipr program:
```bash
cargo run -- <file.vipr>
```
For example, if you have a file `hello.vipr`, this produces a compiled executable `hello` in the current directory.
For example, if you have a file `test.vipr`, this produces a compiled executable `test` in the current directory.

### Running Tests
Unit tests are defined in almost every module's `tests` submodule. Run them with:
```bash
cargo test
```
The codebase enforces `unwrap()` isolation—it is forbidden in core compiler mechanics except for `.expect("ICE: ...")` clauses indicating unrecoverable semantic validation failures.

---

## 6. Gotchas & Tips for Future Work

* **GCC Dependency**: The compiler invokes `gcc` on the host system to perform the final native compilation step. Ensure `gcc` is installed and in the user's `PATH`.
* **C11 Requirement**: The generated code uses C11 `_Generic` macros. The host compiler `gcc` must support C11 standards.
* **Lexer Limits**: The lexer reads numbers and converts them directly via `.parse()`. Very large integers or floats that overflow standard bounds are handled as parsing errors.
* **Return Statements**: Non-void functions are validated to ensure all paths return the expected type, though currently the compiler doesn't do control-flow reachability analysis on whether a return is guaranteed at the end of every branch.
* **Compiler Utilities**: The `src/utils` module is currently empty. If you need to implement unified compiler error spans or span-tracking source mappings, this is the place to start.
* **C11 Requirement**: The host compiler `gcc` must support C11 standards.
* **Lexer & Parser Coordination**: The lexer tracks parenthesis depth `paren_depth`. When `paren_depth > 0`, it intentionally swallows newlines and skips indentation mapping so multi-line expressions inside parens parse cleanly.
* **Return Statements**: Non-void functions are strictly validated via a conservative recursive tree check (`guarantees_return` in `src/semantic/mod.rs`) to ensure all `if`/`elif`/`else` control paths yield the expected type.
44 changes: 37 additions & 7 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,42 @@
# Changelog

All notable changes to this project are documented in this file.
All notable changes to the Vipr programming language and compiler will be documented in this file.

## [v0.2.0] - The Indentation Update

VIPR v0.2.0 is a massive syntax overhaul. We have pivoted away from the C-style brackets and semicolons of v0.1 in favor of a cleaner, Python-inspired, whitespace-sensitive syntax, while retaining our strict static typing and AOT C-compilation pipeline.

## [0.1.0] - 2026-07-17
### Added
- Initial release of Vipr compiler
- Static typing and Python-like syntax support
- Basic CLI for compiling Vipr programs
- **`let` and `const` keywords**: Variables and constants are now explicitly declared using `let` and `const`.
- **Type Annotations**: Types are now declared after a colon, e.g., `let age: int = 25`.
- **Multiple Assignments**: You can now declare and assign multiple variables on a single line: `let a, b: int = 1, 2`.
- **Safe Swaps**: Reassignments safely evaluate the entire right-hand side before mutating the left-hand side, enabling swaps like `a, b = b, a` without temporary variables.
- **Keyword Operators**: Added `and`, `or`, and `not` to replace `&&`, `||`, and `!`.
- **`elif` Keyword**: Replaced `else if` for conditional chaining.
- **Range-based For Loops**: Introduced `for i: int in range(start, end, step):` to replace the verbose C-style `for` loops.
- **Function Returns**: Added the `-> type:` arrow syntax for declaring function return types.
- **Multi-argument I/O**: `print(a, b)` and `input(a, b)` now accept a variable number of comma-separated arguments.

### Changed
- **Whitespace Scoping**: Removed curly braces `{}`. Code blocks are now strictly defined by a colon `:` and indentation levels.
- **Semicolons Removed**: Statements are now terminated strictly by newlines. No more trailing semicolons.
- **`def` over `func`**: Functions are now declared with the `def` keyword.
- **Parentheses in Conditions**: Parentheses are no longer required around the conditions in `if`, `elif`, and `while` statements.
- **`input` over `scan`**: The built-in console reader is now named `input` instead of `scan`.

### Under the Hood (Compiler Architecture)
- **Lexer Statefulness**: The lexer now maintains an internal indentation stack and parenthesis depth counter to cleanly emit `Indent`, `Dedent`, and `Newline` tokens for the parser.
- **Lookahead Disambiguation**: The parser now utilizes parenthesis-aware lookaheads to seamlessly differentiate assignments from standard expressions without relying on semicolons.
- **Codegen Optimization**: The C transpiler no longer relies on complex C11 `_Generic` macros for I/O. `CodeGenerator` now strictly type-checks AST nodes on the fly and generates standard `printf` and `scanf` format strings natively (`%d`, `%f`, `%s`).
- **Temporary Variable Evaluation**: The compiler backend automatically calculates `_tempX` placeholder variables to sandbox multiple-assignment right-hand-sides and loop ranges (`start`, `end`, `step`), guaranteeing zero side-effect bleed during execution.

### Requirements
- To run Vipr binary, GCC needs to be installed on the system
---

## [v0.1.0] - Initial Release [DEPRECATED]

### Added
- Initial compiler release written in Rust.
- C-style bracket `{}` block scoping and mandatory semicolons.
- Built-in primitives: `int`, `float`, `bool`, `char`, `string`, `void`.
- Basic control flow (`if`, `else if`, `else`, `while`, C-style `for`).
- Compile-time macro resolution for `print` and `scan` using C11 `_Generic`.
Loading
Loading