Skip to content

Neutron Test Suite

Overview

Comprehensive test suite for the Neutron programming language covering all major features, bug fixes, and regression tests.

Status: ✅ All 53 tests passing (100% success rate)
Last Updated: November 27, 2025

Test Organization

The test suite is organized into categorized directories for easy navigation and maintenance:

tests/
├── fixes/         (19 tests) - Bug fix regression tests (NEUT-001 to NEUT-020)
├── core/          (13 tests) - Basic language features
├── operators/     (3 tests)  - Operator behavior
├── control-flow/  (7 tests)  - Control structures (if/for/match/break)
├── functions/     (2 tests)  - Functions and lambdas
├── classes/       (3 tests)  - Object-oriented programming
└── modules/       (6 tests)  - Built-in module system

Running Tests

Linux / macOS

python3 run_tests.py

Windows (Visual Studio)

# Build first
cmake -B build -S .
cmake --build build --config Release

# Run tests
python run_tests.py

Windows (MSYS2 - Alternative)

python3 run_tests.py

Output Format

The test runner provides: - ✅ Color-coded output (green for pass, red for fail) - 📁 Directory-based organization - 📊 Per-directory summaries - 🔍 Automatic failure details (stdout/stderr from failed tests) - 📈 Final summary with pass/fail counts

Example output: 

╔════════════════════════════════╗
║  Neutron Test Suite v2.0       ║
╚════════════════════════════════╝

Testing: fixes
  ✓ neut-001
  ✓ neut-002
  ...
  Summary: 18 passed, 0 failed

Testing: core
  ✓ test_variables
  ✓ test_comments
  ...
  Summary: 11 passed, 0 failed

════ FINAL SUMMARY ═══
Total: 49
Passed: 49
Failed: 0

🎉 All tests passed! 🎉

Test Categories

1. Bug Fix Regression Tests (tests/fixes/) - 19 tests

These tests verify that previously discovered bugs remain fixed:

neut-001.nt - Uninitialized Variable Access ✅

  • Tests proper error handling for undefined variables
  • Verifies RuntimeError is thrown with descriptive message

neut-002.nt - BANG_EQUAL Operator Implementation ✅

  • Tests != operator functionality
  • Verifies inequality comparisons work correctly

neut-003.nt - Stack Overflow Prevention ✅

  • Tests deeply nested expressions
  • Verifies no stack overflow with complex calculations

neut-004.nt - Memory Management and GC ✅

  • Tests garbage collection with large data structures
  • Verifies no memory leaks with repeated operations

neut-005.nt - Exception Handling ✅

  • Tests try/catch/finally blocks
  • Verifies exception propagation and cleanup

neut-006.nt - Symbol Operators ✅

  • Tests workarounds for &&, ||, ++, -- operators
  • Verifies and, or, and manual increment/decrement work

neut-007.nt - Array Bounds Checking ✅

  • Tests out-of-bounds array access
  • Verifies proper error messages for invalid indices

neut-008.nt - Module Loading Performance ✅

  • Tests module caching mechanism
  • Verifies repeated imports use cache

neut-009.nt - Array Operations Type Safety ✅

  • Tests type checking in array operations
  • Verifies proper handling of mixed types

neut-010.nt - String Interpolation ✅

  • Tests ${variable} and ${expression} syntax
  • Verifies interpolation with various data types

neut-011.nt - Comparison Operators ✅

  • Tests all comparison operators: <, >, <=, >=, ==, !=
  • Verifies correct boolean results

neut-012.nt - Operator Precedence ✅

  • Tests correct evaluation order of operators
  • Verifies parentheses override default precedence

neut-013.nt - Memory Leak in Native Functions ✅

  • Tests C API memory management
  • Verifies proper cleanup after native function calls

neut-014.nt - Unsafe Static Cast in Parser ✅

  • Tests safe type casting in parser
  • Verifies no undefined behavior with type conversions

neut-015.nt - Constant Pool Overflow ✅

  • Tests error handling for >255 constants
  • Verifies compiler throws appropriate error

neut-016.nt - Jump Offset Overflow ✅

  • Tests error handling for large jump distances
  • Verifies compiler detects overflow conditions

neut-017.nt - Unchecked Dynamic Cast in VM ✅

  • Tests safe downcasting in virtual machine
  • Verifies proper type validation

neut-018.nt - Missing Bounds Check in READ_CONSTANT ✅

  • Tests constant pool bounds checking
  • Verifies no out-of-bounds access in bytecode execution

2. Core Language Features (tests/core/) - 13 tests

2. Core Language Features (tests/core/) - 11 tests

1. test_variables.nt - Variables and Data Types ✅

  • Variable declaration (with/without initial value)
  • Number types (integer, float, negative)
  • Boolean types (true, false)
  • String types
  • Nil type
  • Dynamic typing
  • Variable reassignment

2. test_comments.nt - Comments ✅

  • Single-line comments (//)
  • Comments with code
  • Inline comments

3. test_cross_platform.nt - Cross-Platform Features ✅

  • Platform detection (Windows, macOS, Linux, BSD)
  • Architecture detection (x86_64, arm64, etc.)
  • File operations across platforms
  • Environment variables
  • Directory operations

4. test_error_handling.nt - Error Handling System ✅

  • Variable access (should work)
  • Function calls (should work)
  • Division (should work)
  • Array access (should work)
  • Numeric operations (should work)

5. test_exceptions.nt - Exception Handling ✅

  • Basic try-catch functionality
  • Try blocks without exceptions
  • Finally blocks execution
  • Exception types (numbers, booleans, strings)
  • Nested try-catch blocks
  • Exception propagation handling

6. test_interpolation.nt - String Interpolation ✅

  • Variable interpolation with ${var}
  • Expression interpolation ${expr}
  • Multiple interpolations in one string
  • Interpolation with literals

7. test_no_match.nt - Match Without Default ✅

  • Match statements without default case
  • Fallthrough behavior

8. test_string_interpolation.nt - String Interpolation Tests ✅

  • Basic variable interpolation
  • Number interpolation
  • Expression interpolation
  • Multiple interpolations
  • Complex expression interpolation
  • Nested expression interpolation
  • Multi-variable interpolation

9. test_truthiness.nt - Truthiness Rules ✅

  • nil is falsy
  • false is falsy
  • 0 is truthy (unlike JavaScript)
  • Empty string "" is truthy
  • Negative numbers are truthy
  • All other values are truthy

10. test_types.nt - Type Safety and Annotations ✅

  • Basic type annotations (int, string, float, bool)
  • Untyped variables (backward compatibility)
  • Reassignment of typed variables
  • Type annotations with arrays
  • Type annotations with objects
  • Any type annotation
  • Type annotations in loops
  • Multiple typed declarations
  • Uninitialized typed variables

11. test_command_line_args.nt - Command Line Arguments ✅

  • sys.args() array access
  • Script path retrieval
  • Argument parsing

12. test_string_methods.nt - String Methods ✅

  • String.length
  • String.contains()
  • String.split()

13. test_multiple_vars.nt - Multiple Variable Declarations ✅

  • Single line multiple declarations
  • Typed multiple declarations

3. Operators (tests/operators/) - 3 tests

1. test_operators.nt - Basic Operators ✅

  • Arithmetic operators (+, -, *, /)
  • Comparison operators (==, !=, <, >)
  • Logical operators (and, or)
  • String concatenation
  • Operator precedence
  • Parentheses for precedence

2. test_equality.nt - Equality Operations ✅

  • Equality comparisons
  • Type coercion in comparisons
  • Null/nil equality

3. test_modulo.nt - Modulo Operator ✅

  • Basic modulo operations
  • Even/odd checking
  • Modulo with larger numbers
  • Modulo equals zero (divisibility)
  • Modulo in loops for pattern generation

4. Control Flow (tests/control-flow/) - 7 tests

1. test_control_flow.nt - Basic Control Flow ✅

  • Simple if statements
  • If-else statements
  • Nested if statements
  • While loops
  • While loops with break
  • Complex conditions (and, or)

2. test_if_else.nt - If-Else Chains ✅

  • Else-if chains
  • Grade calculation systems
  • Nested if-else structures
  • Complex conditional logic

3. test_if.nt - Simple If Statements ✅

  • Basic if conditions
  • Boolean expressions
  • Comparison in conditions

4. test_elif.nt - ELIF Statements ✅

  • Multi-level conditional logic
  • String comparisons
  • Logical combinations
  • Complex elif chains

5. test_for_loops.nt - For Loops ✅

  • Basic for loop syntax
  • For loops with different step values
  • Nested for loops
  • Break and continue in for loops
  • Loop counters and patterns

6. test_break_continue.nt - Loop Control ✅

  • Break in while loop
  • Continue in while loop
  • Break with complex conditions
  • Continue with conditional skipping

7. test_match.nt - Match Statements ✅

  • Basic match with numbers
  • Match with expressions
  • Match with strings
  • Match with block statements
  • Match without default
  • Match with variables
  • Nested match
  • Match with booleans
  • Match with variable assignment

5. Functions (tests/functions/) - 2 tests

5. Functions (tests/functions/) - 2 tests

9. test_functions.nt - Functions ✅

  • Simple function definition and calling
  • Functions with return values
  • Multiple parameters
  • Function calling function
  • Conditional return statements

10. test_lambda_comprehensive.nt - Lambda Functions ✅

  • Basic lambda with parameters
  • Lambda with no parameters
  • Lambdas in arrays
  • Lambdas as function arguments
  • Immediately invoked lambdas
  • Multiple lambdas with different operations
  • Nested lambdas

6. Classes and OOP (tests/classes/) - 3 tests

11. test_classes.nt - Classes and OOP ✅

  • Class definition and instantiation
  • Instance methods
  • The this keyword
  • Instance fields/properties
  • Property assignment in methods
  • Multiple instances with independent state
  • Methods with calculations

12. test_objects.nt - Objects and JSON ✅

  • Object literal creation
  • JSON stringify
  • JSON parse
  • JSON with arrays

13. test_init_syntax.nt - Class Init Syntax ✅

  • init() constructor without fun keyword
  • Backward compatibility with fun init()

7. Built-in Modules (tests/modules/) - 6 tests

13. test_arrays_module.nt - Arrays Module ✅

  • Basic operations (new, push, pop)
  • Access and modification (at, set)
  • Removal operations (remove, clear)
  • Contains and index_of
  • Sorting and reversing
  • Range operations
  • Cloning
  • Join operations
  • Flat operations
  • Slice operations
  • Shuffle operations
  • Fill operations
  • Remove operations
  • to_string operations

14. test_async_module.nt - Async Module ✅

  • async.sleep() - Delay execution without blocking
  • async.run() - Execute functions asynchronously
  • async.await() - Wait for async operations to complete
  • async.timer() - Schedule delayed execution
  • async.promise() - Create promise-like objects
  • Combined async operations
  • Integration with other modules

15. test_http_module.nt - HTTP Module ✅

  • HTTP GET requests
  • HTTP POST requests
  • HTTP PUT requests
  • HTTP DELETE requests
  • HTTP HEAD requests
  • HTTP PATCH requests
  • Response object structure (status, body, headers)

16. test_math_module.nt - Math Module ✅

  • math.add()
  • math.subtract()
  • math.multiply()
  • math.divide()
  • Math functions in calculations

17. test_sys_module.nt - Sys Module ✅

  • File write/read operations
  • File existence checking
  • Current working directory
  • System information
  • Command line arguments
  • File removal

18. test_time_module.nt - Time Module ✅

  • time.now() - Current timestamp
  • time.format() - Date/time formatting
  • time.sleep() - Delay execution

Test Results

Status: ✅ ALL TESTS PASSING (53/53) - 100% SUCCESS RATE

════ FINAL SUMMARY ═══
Total: 49
Passed: 49
Failed: 0

🎉 All tests passed! 🎉

Last Test Run: November 27, 2025

Coverage

The test suite provides comprehensive coverage of:

✅ Core Language (100%)

  • All basic data types (numbers, booleans, strings, nil)
  • Variables and dynamic typing
  • All operators (arithmetic, comparison, logical, modulo)
  • Control flow (if/else, if-else-if chains, while loops, for loops)
  • Loop control (break, continue in both for and while)
  • Functions (definition, calling, return, parameters)
  • Classes and OOP (instantiation, methods, this, fields)
  • Comments
  • String interpolation with ${...} syntax
  • Truthiness rules (nil/false falsy, everything else truthy)
  • Type annotations (int, string, float, bool, any)

✅ Built-in Modules (100%)

  • sys module - File I/O, directory ops, environment, command args
  • math module - Basic arithmetic operations
  • time module - Timestamps, formatting, delays
  • http module - GET, POST, PUT, DELETE, HEAD, PATCH requests
  • arrays module - Array manipulation operations
  • async module - Asynchronous operations and promises

✅ Platform Support (100%)

  • Cross-platform features (Windows, macOS, Linux, BSD support)
  • Platform/architecture detection
  • Command line argument handling

✅ Bug Fixes & Regression Tests (100%)

  • 18 regression tests covering all major bug fixes
  • Memory leak prevention
  • Bounds checking
  • Type safety
  • Operator implementations
  • Exception handling
  • Performance optimizations

Bug Fix Coverage (NEUT-001 to NEUT-018)

All documented bugs in FIXES.md have corresponding regression tests:

Bug ID Issue Test File Status
NEUT-001 Uninitialized Variable Access neut-001.nt ✅
NEUT-002 BANG_EQUAL Operator neut-002.nt ✅
NEUT-003 Stack Overflow Prevention neut-003.nt ✅
NEUT-004 Memory Management/GC neut-004.nt ✅
NEUT-005 Exception Handling neut-005.nt ✅
NEUT-006 Symbol Operators neut-006.nt ✅
NEUT-007 Array Bounds Checking neut-007.nt ✅
NEUT-008 Module Loading Cache neut-008.nt ✅
NEUT-009 Array Type Safety neut-009.nt ✅
NEUT-010 String Interpolation neut-010.nt ✅
NEUT-011 Comparison Operators neut-011.nt ✅
NEUT-012 Operator Precedence neut-012.nt ✅
NEUT-013 Memory Leak in Native Functions neut-013.nt ✅
NEUT-014 Unsafe Static Cast neut-014.nt ✅
NEUT-015 Constant Pool Overflow neut-015.nt ✅
NEUT-016 Jump Offset Overflow neut-016.nt ✅
NEUT-017 Unchecked Dynamic Cast neut-017.nt ✅
NEUT-018 Missing Bounds Check neut-018.nt ✅

Bug Fix Version: All bugs identified in v1.1.3-beta, fixed in v1.2.1

Implementation Status

✅ Fully Implemented and Tested

  • Classes: Complete OOP support with methods, this, instance fields
  • For Loops: Full for-loop support with init/condition/increment
  • String Interpolation: ${variable} and ${expression} in strings
  • HTTP Module: All 6 HTTP methods with response objects
  • Time Module: Current time, formatting, sleep functionality
  • Exception Handling: try/catch/finally with proper cleanup
  • Type Annotations: int, string, float, bool, any types
  • Arrays Module: Comprehensive array manipulation
  • Async Module: Asynchronous operations and promises
  • All Core Features: Variables, operators, control flow, functions

⚠️ Known Limitations

  • Symbol operators: &&, ||, ++, -- not implemented (use and, or, x+1, x-1)
  • Array callbacks: Some methods with callbacks not fully implemented

Notes

Build System

Tests work with both build systems: - ✅ CMake (primary, recommended) - ✅ Makefile (legacy, still supported)

Platform Support

All tests pass on: - ✅ Linux (Ubuntu, Debian, Fedora, Arch, Alpine) - ✅ macOS (Intel and Apple Silicon) - ✅ Windows (Visual Studio, MSYS2, MinGW) - ✅ BSD variants (FreeBSD, OpenBSD, NetBSD)

Test Runner Features

  • 📁 Directory-based organization - Easy to find and categorize tests
  • 🎨 Color-coded output - Green for pass, red for fail
  • 📊 Per-directory summaries - See results by category
  • 🔍 Automatic failure details - Stderr/stdout displayed for failed tests
  • 📈 Final summary - Total pass/fail counts

Running Individual Tests

You can run tests by directory or individual file:

# Run all tests
./neutron tests/fixes/neut-001.nt

# Run specific test
./neutron tests/core/test_variables.nt

# Run all tests in a directory (manually)
for file in tests/fixes/*.nt; do ./neutron "$file"; done

Adding New Tests

  1. Choose the appropriate directory:
  2. tests/fixes/ - Bug fix regression tests
  3. tests/core/ - Basic language features
  4. tests/operators/ - Operator behavior
  5. tests/control-flow/ - Control structures
  6. tests/functions/ - Function-related tests
  7. tests/classes/ - OOP features

  8. tests/modules/ - Built-in modules

  9. Create a new file with descriptive name: test_feature.nt

  10. Follow the testing pattern: 

    say("=== Test Name ===");

// Test case 1
var result = some_operation();
if (result == expected) {
    say("✓ Test case 1 passed");
}

// Test case 2
// ...

say("=== All tests passed ===");

  11. Run the test suite to verify: python3 run_tests.py

  12. Tests are automatically discovered by the test runner

Integration

Tests are included in: - run_tests.py - Main test runner with directory-based organization - create_release.sh - Tests directory included in releases - README.md - Testing instructions - FIXES.md - Bug tracking with test references

Continuous Testing

Run tests after: - Code changes - Building the project - Before creating releases - After platform updates - Before committing changes - After fixing bugs (ensure regression test exists)

Test Maintenance

When fixing a bug:

  1. Document the bug in FIXES.md with NEUT-XXX identifier
  2. Create regression test in tests/fixes/neut-XXX.nt
  3. Fix the bug in source code
  4. Verify the test passes
  5. Update this documentation

Test organization rules:

  • Bug fixes → tests/fixes/ with NEUT-XXX naming
  • New features → Appropriate category directory
  • One test per bug for easy identification
  • Descriptive output with ✓ for passed checks

Version History

  • v2.0 (November 2, 2025) - Complete reorganization into 7 directories, 49 tests
  • v1.0 (October 28, 2025) - Initial flat structure with 30 tests

Total Test Coverage: 53/53 tests passing (100%)
Regression Coverage: 19/19 bug fixes tested (100%)
Module Coverage: 6/6 modules tested (100%)