diff --git a/gc_plan.md b/gc_plan.md index 2e163902..6f9d4bb5 100644 --- a/gc_plan.md +++ b/gc_plan.md @@ -1,602 +1,403 @@ -# GC Refactoring Plan: Cheney Copying Collector +# Plan: Complete Copying GC Implementation ## Overview -Replace the current reference-counting GC with a simple two-space Cheney copying collector. -This fundamentally simplifies the system: no more `JSGCObjectHeader`, no ref counts, -no cycle detection - just bump allocation and copying live objects when memory fills up. +Remove reference counting (DupValue/FreeValue) entirely and complete the Cheney copying garbage collector. Each JSContext will use bump allocation from a heap block, and when out of memory, request a new heap from JSRuntime's buddy allocator and copy live objects to the new heap. -## Architecture +## Target Architecture (from docs/memory.md) -``` -JSRuntime (256 MB pool) - ├── Buddy allocator for block management - ├── Class definitions (shared across contexts) - └── JSContext #1 (actor) - ├── Current block (64KB initially) - ├── heap_base: start of block - ├── heap_free: bump pointer (total used = heap_free - heap_base) - ├── Text interning table (per-context, rebuilt on GC) - └── On memory pressure: request new block, copy live objects, return old block -``` +### Object Types (simplified from current): -**Key principle**: Each JSContext (actor) owns its own memory. Nothing is stored in JSRuntime -except the buddy allocator and class definitions. There is no runtime-level string arena. +**Type 0 - Array**: `{ header, length, elements[] }` +**Type 1 - Blob**: `{ header, length, bits[] }` +**Type 2 - Text**: `{ header, length_or_hash, packed_chars[] }` +**Type 3 - Record**: `{ header, prototype, length, key_value_pairs[] }` +**Type 4 - Function**: `{ header, code_ptr, outer_frame_ptr }` - 3 words only, always stone +**Type 5 - Frame**: `{ header, function_ptr, caller_ptr, ret_addr, args[], closure_vars[], local_vars[], temps[] }` +**Type 6 - Code**: Lives in immutable memory only, never copied +**Type 7 - Forward**: Object has moved; cap56 contains new address -## Memory Model (from docs/memory.md) +### Key Design Points: +- **JSFunction** is just a pointer to code and a pointer to the frame that created it (3 words) +- **Closure variables live in frames** - when a function returns, its frame is "reduced" to just the closure variables +- **Code objects are immutable** - stored in stone memory, never copied during GC +- **Frame reduction**: When a function returns, `caller` is set to zero, signaling the frame can be shrunk -### Object Header (objhdr_t - 64 bits) -``` -[56 bits: capacity] [1 bit: R flag] [3 bits: reserved] [1 bit: stone] [3 bits: type] -``` +## Current State (needs refactoring) -All heap objects start with just `objhdr_t`. No `JSGCObjectHeader`, no ref counts. +1. **Partial Cheney GC exists** at `source/quickjs.c:1844-2030`: `ctx_gc`, `gc_copy_value`, `gc_scan_object` +2. **744 calls to JS_DupValue/JS_FreeValue** scattered throughout (currently undefined, causing compilation errors) +3. **Current JSFunction** is bloated (has kind, name, union of cfunc/bytecode/bound) - needs simplification +4. **Current JSVarRef** is a separate object - should be eliminated, closures live in frames +5. **Bump allocator** in `js_malloc` (line 1495) with `heap_base`/`heap_free`/`heap_end` +6. **Buddy allocator** for memory blocks (lines 1727-1837) +7. **Header offset inconsistency** - some structs have header at offset 0, some at offset 8 -### Object Types -- 0: OBJ_ARRAY - Header, Length, Elements[] -- 1: OBJ_BLOB - Header, Length (bits), BitWords[] -- 2: OBJ_TEXT - Header, Length/Hash, PackedChars[] (see Text section below) -- 3: OBJ_RECORD - Header, Prototype, Length, Key/Value pairs -- 4: OBJ_FUNCTION - Header, Code, Outer (always stone, 3 words) -- 5: OBJ_CODE - Header, Arity, Size, ClosureSize, Entry, Disruption (in context memory) -- 6: OBJ_FRAME - Header, Function, Caller, ReturnAddr, Slots[] -- 7: OBJ_FORWARD - Forwarding pointer (used during GC) +## Implementation Steps -### Text (Type 2) - Two Modes +### Phase 1: Define No-Op DupValue/FreeValue (To Enable Compilation) -Text has two forms depending on the stone bit: - -**Pretext (stone=0)**: Mutable intermediate representation -- `capacity` = max chars it can hold -- `length` word = actual number of characters -- Used during string building/concatenation - -**Text (stone=1)**: Immutable user-facing string -- `capacity` = length (they're equal for stoned text) -- `length` word = hash (for record key lookup) -- All text keys in records must be stoned -- Text literals are stoned and interned +Add these near line 100 in `source/quickjs.c`: ```c -typedef struct { - objhdr_t hdr; /* type=OBJ_TEXT, cap=capacity, s=stone bit */ - uint64_t len_or_hash; /* length if pretext (s=0), hash if text (s=1) */ - uint64_t packed[]; /* 2 UTF32 chars per word */ +/* Copying GC - no reference counting needed */ +#define JS_DupValue(ctx, v) (v) +#define JS_FreeValue(ctx, v) ((void)0) +#define JS_DupValueRT(rt, v) (v) +#define JS_FreeValueRT(rt, v) ((void)0) +``` + +This makes the code compile while keeping existing call sites (they become no-ops). + +### Phase 2: Standardize Object Headers (offset 0) + +Remove `JSGCObjectHeader` (ref counting remnant) and put `objhdr_t` at offset 0: + +```c +typedef struct JSArray { + objhdr_t hdr; // offset 0 + word_t length; + JSValue values[]; +} JSArray; + +typedef struct JSRecord { + objhdr_t hdr; // offset 0 + JSRecord *proto; + word_t length; + slot slots[]; +} JSRecord; + +typedef struct JSText { + objhdr_t hdr; // offset 0 + word_t length; // pretext: length, text: hash + word_t packed[]; } JSText; + +typedef struct JSBlob { + objhdr_t hdr; // offset 0 + word_t length; + uint8_t bits[]; +} JSBlob; + +/* Simplified JSFunction per memory.md - 3 words */ +typedef struct JSFunction { + objhdr_t hdr; // offset 0, always stone + JSCode *code; // pointer to immutable code object + struct JSFrame *outer; // frame that created this function +} JSFunction; + +/* JSFrame per memory.md */ +typedef struct JSFrame { + objhdr_t hdr; // offset 0 + JSFunction *function; // function being executed + struct JSFrame *caller; // calling frame (NULL = reduced/returned) + word_t ret_addr; // return instruction address + JSValue slots[]; // args, closure vars, locals, temps +} JSFrame; + +/* JSCode - always in immutable (stone) memory */ +typedef struct JSCode { + objhdr_t hdr; // offset 0, always stone + word_t arity; // max number of inputs + word_t frame_size; // capacity of activation frame + word_t closure_size; // reduced capacity for returned frames + word_t entry_point; // address to begin execution + word_t disruption_point;// address of disruption clause + uint8_t bytecode[]; // actual bytecode +} JSCode; ``` -**DELETE JSString** - just use JSText (currently named mist_text). +### Phase 3: Complete gc_object_size for All Types -### Text Interning (Per-Context) - -Each context maintains its own text interning table: -- Texts used as record keys are stoned and interned -- Text literals are stoned and interned -- During GC, a new interning table is built as live objects are copied -- This prevents the table from becoming a graveyard +Update `gc_object_size` (line 1850) to read header at offset 0: ```c -/* In JSContext */ -JSText **text_intern_array; /* indexed by ID */ -uint32_t *text_intern_hash; /* hash table mapping to IDs */ -uint32_t text_intern_count; /* number of interned texts */ -uint32_t text_intern_size; /* hash table size */ +static size_t gc_object_size(void *ptr) { + objhdr_t hdr = *(objhdr_t*)ptr; // Header at offset 0 + uint8_t type = objhdr_type(hdr); + uint64_t cap = objhdr_cap56(hdr); + + switch (type) { + case OBJ_ARRAY: + return sizeof(JSArray) + cap * sizeof(JSValue); + case OBJ_BLOB: + return sizeof(JSBlob) + (cap + 7) / 8; // cap is bits + case OBJ_TEXT: + return sizeof(JSText) + ((cap + 1) / 2) * sizeof(uint64_t); + case OBJ_RECORD: + return sizeof(JSRecord) + (cap + 1) * sizeof(slot); // cap is mask + case OBJ_FUNCTION: + return sizeof(JSFunction); // 3 words + case OBJ_FRAME: + return sizeof(JSFrame) + cap * sizeof(JSValue); // cap is slot count + case OBJ_CODE: + return 0; // Code is never copied (immutable) + default: + return 64; // Conservative fallback + } +} ``` ---- +### Phase 4: Complete gc_scan_object for All Types -## Phase 1: Add Buddy Allocator to JSRuntime [DONE] +Update `gc_scan_object` (line 1924): -### File: source/quickjs.c - -**1.1 Add buddy allocator structures** ```c -#define BUDDY_MIN_ORDER 16 /* 64KB minimum block */ -#define BUDDY_MAX_ORDER 28 /* 256MB maximum */ -#define BUDDY_LEVELS (BUDDY_MAX_ORDER - BUDDY_MIN_ORDER + 1) +static void gc_scan_object(JSContext *ctx, void *ptr, uint8_t **to_free, uint8_t *to_end) { + objhdr_t hdr = *(objhdr_t*)ptr; + uint8_t type = objhdr_type(hdr); -typedef struct BuddyBlock { - struct BuddyBlock *next; - struct BuddyBlock *prev; - uint8_t order; /* log2 of size */ - uint8_t is_free; -} BuddyBlock; - -typedef struct BuddyAllocator { - uint8_t *base; /* 256MB base address */ - size_t total_size; /* 256MB */ - BuddyBlock *free_lists[BUDDY_LEVELS]; -} BuddyAllocator; + switch (type) { + case OBJ_ARRAY: { + JSArray *arr = (JSArray*)ptr; + for (uint32_t i = 0; i < arr->length; i++) { + arr->values[i] = gc_copy_value(ctx, arr->values[i], to_free, to_end); + } + break; + } + case OBJ_RECORD: { + JSRecord *rec = (JSRecord*)ptr; + // Copy prototype + if (rec->proto) { + JSValue proto_val = JS_MKPTR(rec->proto); + proto_val = gc_copy_value(ctx, proto_val, to_free, to_end); + rec->proto = (JSRecord*)JS_VALUE_GET_PTR(proto_val); + } + // Copy table entries + uint32_t mask = objhdr_cap56(rec->hdr); + for (uint32_t i = 1; i <= mask; i++) { // Skip slot 0 + JSValue k = rec->slots[i].key; + if (!rec_key_is_empty(k) && !rec_key_is_tomb(k)) { + rec->slots[i].key = gc_copy_value(ctx, k, to_free, to_end); + rec->slots[i].value = gc_copy_value(ctx, rec->slots[i].value, to_free, to_end); + } + } + break; + } + case OBJ_FUNCTION: { + JSFunction *func = (JSFunction*)ptr; + // Code is immutable, don't copy - but outer frame needs copying + if (func->outer) { + JSValue outer_val = JS_MKPTR(func->outer); + outer_val = gc_copy_value(ctx, outer_val, to_free, to_end); + func->outer = (JSFrame*)JS_VALUE_GET_PTR(outer_val); + } + break; + } + case OBJ_FRAME: { + JSFrame *frame = (JSFrame*)ptr; + // Copy function pointer + if (frame->function) { + JSValue func_val = JS_MKPTR(frame->function); + func_val = gc_copy_value(ctx, func_val, to_free, to_end); + frame->function = (JSFunction*)JS_VALUE_GET_PTR(func_val); + } + // Copy caller (unless NULL = reduced frame) + if (frame->caller) { + JSValue caller_val = JS_MKPTR(frame->caller); + caller_val = gc_copy_value(ctx, caller_val, to_free, to_end); + frame->caller = (JSFrame*)JS_VALUE_GET_PTR(caller_val); + } + // Copy all slots (args, closure vars, locals, temps) + uint32_t slot_count = objhdr_cap56(frame->hdr); + for (uint32_t i = 0; i < slot_count; i++) { + frame->slots[i] = gc_copy_value(ctx, frame->slots[i], to_free, to_end); + } + break; + } + case OBJ_TEXT: + case OBJ_BLOB: + case OBJ_CODE: + // No internal references to scan + break; + } +} ``` -**1.2 Update JSRuntime** +### Phase 5: Fix gc_copy_value Forwarding + +Update `gc_copy_value` (line 1883) for offset 0 headers: + ```c -struct JSRuntime { - BuddyAllocator buddy; - int class_count; - JSClass *class_array; - struct list_head context_list; - /* REMOVE: gc_obj_list, gc_zero_ref_count_list, gc_phase, malloc_gc_threshold */ - /* REMOVE: malloc functions, malloc_state - contexts use bump allocation */ -}; +static JSValue gc_copy_value(JSContext *ctx, JSValue v, uint8_t **to_free, uint8_t *to_end) { + if (!JS_IsPtr(v)) return v; // Immediate value + + void *ptr = JS_VALUE_GET_PTR(v); + + // Stone memory - don't copy (includes Code objects) + objhdr_t hdr = *(objhdr_t*)ptr; + if (objhdr_s(hdr)) return v; + + // Check if in current heap + if ((uint8_t*)ptr < ctx->heap_base || (uint8_t*)ptr >= ctx->heap_end) + return v; // External allocation + + // Already forwarded? + if (objhdr_type(hdr) == OBJ_FORWARD) { + void *new_ptr = (void*)(uintptr_t)objhdr_cap56(hdr); + return JS_MKPTR(new_ptr); + } + + // Copy object to new space + size_t size = gc_object_size(ptr); + void *new_ptr = *to_free; + *to_free += size; + memcpy(new_ptr, ptr, size); + + // Leave forwarding pointer in old location + *(objhdr_t*)ptr = objhdr_make((uint64_t)(uintptr_t)new_ptr, OBJ_FORWARD, 0, 0, 0, 0); + + return JS_MKPTR(new_ptr); +} ``` -**1.3 Implement buddy functions** [DONE] -- `buddy_init(BuddyAllocator *b)` - allocate 256MB, initialize free lists -- `buddy_alloc(BuddyAllocator *b, size_t size)` - allocate block of given size -- `buddy_free(BuddyAllocator *b, void *ptr, size_t size)` - return block -- `buddy_destroy(BuddyAllocator *b)` - free the 256MB +### Phase 6: Complete GC Root Tracing ---- +Update `ctx_gc` (line 1966) to trace all roots including JSGCRef: -## Phase 2: Restructure JSContext for Bump Allocation [DONE] - -### File: source/quickjs.c - -**2.1 Update JSContext** ```c -struct JSContext { - JSRuntime *rt; - struct list_head link; +static int ctx_gc(JSContext *ctx) { + // ... existing setup code ... - /* Actor memory block - bump allocation */ - uint8_t *heap_base; /* start of current block */ - uint8_t *heap_free; /* bump pointer */ - uint8_t *heap_end; /* end of block */ - size_t current_block_size; /* 64KB initially */ - size_t next_block_size; /* doubles if <10% recovered */ + // Copy roots: global object, class prototypes, etc. (existing) + ctx->global_obj = gc_copy_value(ctx, ctx->global_obj, &to_free, to_end); + ctx->global_var_obj = gc_copy_value(ctx, ctx->global_var_obj, &to_free, to_end); + // ... other existing root copying ... - /* Stack (VM execution) */ - JSValue *value_stack; - int value_stack_top; - int value_stack_capacity; - struct VMFrame *frame_stack; - int frame_stack_top; - int frame_stack_capacity; + // Copy GC root stack (JS_PUSH_VALUE/JS_POP_VALUE) + for (JSGCRef *ref = ctx->top_gc_ref; ref; ref = ref->prev) { + ref->val = gc_copy_value(ctx, ref->val, &to_free, to_end); + } - /* Roots */ - JSValue global_obj; - JSValue *class_proto; - JSValue current_exception; + // Copy GC root list (JS_AddGCRef/JS_DeleteGCRef) + for (JSGCRef *ref = ctx->last_gc_ref; ref; ref = ref->prev) { + ref->val = gc_copy_value(ctx, ref->val, &to_free, to_end); + } - /* Text interning (per-context, rebuilt on GC) */ - JSText **text_intern_array; - uint32_t *text_intern_hash; - uint32_t text_intern_count; - uint32_t text_intern_size; + // Copy current exception + ctx->current_exception = gc_copy_value(ctx, ctx->current_exception, &to_free, to_end); - /* Other context state */ - uint16_t class_count; - int interrupt_counter; - void *user_opaque; - - /* REMOVE: JSGCObjectHeader header */ -}; + // Cheney scan (existing) + // ... +} ``` -**2.2 Implement bump allocator** [DONE] +### Phase 7: Trigger GC on Allocation Failure + +Update `js_malloc` (line 1495): + ```c -static void *ctx_alloc(JSContext *ctx, size_t size) { - size = (size + 7) & ~7; /* 8-byte align */ - if (ctx->heap_free + size > ctx->heap_end) { - if (ctx_gc(ctx) < 0) return NULL; /* triggers GC */ - if (ctx->heap_free + size > ctx->heap_end) { - return NULL; /* still OOM after GC */ +void *js_malloc(JSContext *ctx, size_t size) { + size = (size + 7) & ~7; // Align to 8 bytes + + if ((uint8_t*)ctx->heap_free + size > (uint8_t*)ctx->heap_end) { + if (ctx_gc(ctx) < 0) { + JS_ThrowOutOfMemory(ctx); + return NULL; + } + // Retry after GC + if ((uint8_t*)ctx->heap_free + size > (uint8_t*)ctx->heap_end) { + JS_ThrowOutOfMemory(ctx); + return NULL; } } + void *ptr = ctx->heap_free; - ctx->heap_free += size; + ctx->heap_free = (uint8_t*)ctx->heap_free + size; return ptr; } ``` ---- +### Phase 8: Frame Reduction (for closures) -## Phase 3: Unify Object Headers (Remove JSGCObjectHeader) - -### 3.1 New unified object layout - -All heap objects start with just `objhdr_t`: +When a function returns, "reduce" its frame to just closure variables: ```c -/* Array */ -typedef struct { - objhdr_t hdr; /* type=OBJ_ARRAY, cap=element_capacity */ - uint64_t len; - JSValue elem[]; -} JSArray; +static void reduce_frame(JSContext *ctx, JSFrame *frame) { + if (frame->caller == NULL) return; // Already reduced -/* Text (replaces both mist_text and JSString) */ -typedef struct { - objhdr_t hdr; /* type=OBJ_TEXT, cap=capacity, s=stone bit */ - uint64_t len_or_hash; /* len if pretext (s=0), hash if text (s=1) */ - uint64_t packed[]; /* 2 UTF32 chars per word */ -} JSText; + JSCode *code = frame->function->code; + uint32_t closure_size = code->closure_size; -/* Record (object) */ -typedef struct JSRecord { - objhdr_t hdr; /* type=OBJ_RECORD, cap=mask, s=stone bit */ - struct JSRecord *proto; - uint64_t len; - uint64_t tombs; - uint16_t class_id; - uint16_t _pad; - uint32_t rec_id; /* for record-as-key hashing */ - JSValue slots[]; /* key[0], val[0], key[1], val[1], ... */ -} JSRecord; - -/* Function */ -typedef struct { - objhdr_t hdr; /* type=OBJ_FUNCTION, always stone */ - JSValue code; /* pointer to code object */ - JSValue outer; /* pointer to enclosing frame */ -} JSFunction; - -/* Frame */ -typedef struct { - objhdr_t hdr; /* type=OBJ_FRAME, cap=slot_count */ - JSValue function; /* JSFunction */ - JSValue caller; /* JSFrame or null */ - uint64_t return_addr; - JSValue slots[]; /* args, locals, temporaries */ -} JSFrame; - -/* Code (in context memory, always stone) */ -typedef struct { - objhdr_t hdr; /* type=OBJ_CODE, always stone */ - uint32_t arity; - uint32_t frame_size; - uint32_t closure_size; - uint64_t entry_point; - uint64_t disruption_point; - uint8_t bytecode[]; -} JSCode; -``` - -### 3.2 Delete legacy types - -**DELETE:** -- `JSString` struct - use `JSText` instead -- `JSGCObjectHeader` struct -- `mist_text` - rename to `JSText` -- All `p->header.ref_count` accesses -- All `p->header.gc_obj_type` accesses -- `JS_GC_OBJ_TYPE_*` enum values -- `add_gc_object()`, `remove_gc_object()` functions -- `js_alloc_string_rt()` - no runtime-level string allocation - ---- - -## Phase 4: Implement Cheney Copying GC [DONE - needs update] - -### 4.1 Core GC function - -```c -static int ctx_gc(JSContext *ctx) { - size_t old_used = ctx->heap_free - ctx->heap_base; - - /* Request new block from runtime */ - size_t new_size = ctx->next_block_size; - uint8_t *new_block = buddy_alloc(&ctx->rt->buddy, new_size); - if (!new_block) return -1; - - uint8_t *to_base = new_block; - uint8_t *to_free = new_block; - uint8_t *to_end = new_block + new_size; - - /* Reset text interning table (will be rebuilt during copy) */ - ctx->text_intern_count = 0; - memset(ctx->text_intern_hash, 0, ctx->text_intern_size * sizeof(uint32_t)); - - /* Copy roots */ - ctx->global_obj = gc_copy_value(ctx, ctx->global_obj, &to_free, to_end); - ctx->current_exception = gc_copy_value(ctx, ctx->current_exception, &to_free, to_end); - for (int i = 0; i < ctx->class_count; i++) { - ctx->class_proto[i] = gc_copy_value(ctx, ctx->class_proto[i], &to_free, to_end); - } - - /* Copy stack */ - for (int i = 0; i < ctx->value_stack_top; i++) { - ctx->value_stack[i] = gc_copy_value(ctx, ctx->value_stack[i], &to_free, to_end); - } - - /* Cheney scan: process copied objects */ - uint8_t *scan = to_base; - while (scan < to_free) { - gc_scan_object(ctx, scan, &to_free, to_end); - scan += gc_object_size(scan); - } - - /* Return old block */ - buddy_free(&ctx->rt->buddy, ctx->heap_base, ctx->current_block_size); - - /* Update context */ - size_t new_used = to_free - to_base; - size_t recovered = old_used - new_used; - - ctx->heap_base = to_base; - ctx->heap_free = to_free; - ctx->heap_end = to_end; - ctx->current_block_size = new_size; - - /* If <10% recovered, double next block size */ - if (recovered < old_used / 10) { - ctx->next_block_size = new_size * 2; - } - - return 0; + // Shrink capacity to just closure variables + frame->hdr = objhdr_make(closure_size, OBJ_FRAME, 0, 0, 0, 0); + frame->caller = NULL; // Signal: frame is reduced } ``` -### 4.2 Copy functions per type +### Phase 9: Remove Unused Reference Counting Code -```c -static JSValue gc_copy_value(JSContext *ctx, JSValue v, uint8_t **to_free, uint8_t *to_end) { - if (!JS_IsPtr(v)) return v; /* immediate value */ +Delete: +- `gc_decref`, `gc_decref_child` functions +- `gc_scan_incref_child`, `gc_scan_incref_child2` functions +- `JS_GCPhaseEnum`, `gc_phase` fields +- `JSGCObjectHeader` struct (merge into objhdr_t) +- `ref_count` fields from any remaining structs +- `mark_function_children_decref` function +- All `free_*` functions that rely on ref counting - void *ptr = JS_VALUE_GET_PTR(v); - objhdr_t hdr = *(objhdr_t *)ptr; +## Files to Modify - /* Already forwarded? */ - if (objhdr_type(hdr) == OBJ_FORWARD) { - /* Extract forwarding address from cap56 field */ - return JS_MKPTR(JS_TAG_PTR, (void *)(uintptr_t)objhdr_cap56(hdr)); - } +1. **source/quickjs.c** - Main implementation: + - Add DupValue/FreeValue no-op macros (~line 100) + - Restructure JSArray, JSBlob, JSText, JSRecord (lines 468-499) + - Simplify JSFunction to 3-word struct (line 1205) + - Add JSFrame as heap object (new) + - Restructure JSCode/JSFunctionBytecode (line 1293) + - Fix gc_object_size (line 1850) + - Fix gc_copy_value (line 1883) + - Complete gc_scan_object (line 1924) + - Update ctx_gc for all roots (line 1966) + - Update js_malloc to trigger GC (line 1495) + - Delete ref counting code throughout - /* Copy object */ - size_t size = gc_object_size(ptr); - void *new_ptr = *to_free; - memcpy(new_ptr, ptr, size); - *to_free += size; +2. **source/quickjs.h** - Public API: + - Remove JSGCObjectHeader + - Update JSValue type checks if needed + - Ensure JS_IsStone works with offset 0 headers - /* Install forwarding pointer in old location */ - *(objhdr_t *)ptr = objhdr_make((uint64_t)(uintptr_t)new_ptr, OBJ_FORWARD, 0, 0, 0, 0); +## Execution Order - /* If it's a stoned text, re-intern it */ - if (objhdr_type(hdr) == OBJ_TEXT && objhdr_s(hdr)) { - gc_intern_text(ctx, (JSText *)new_ptr); - } - - return JS_MKPTR(JS_TAG_PTR, new_ptr); -} - -static void gc_scan_object(JSContext *ctx, void *ptr, uint8_t **to_free, uint8_t *to_end) { - objhdr_t hdr = *(objhdr_t *)ptr; - switch (objhdr_type(hdr)) { - case OBJ_ARRAY: { - JSArray *arr = ptr; - for (uint64_t i = 0; i < arr->len; i++) { - arr->elem[i] = gc_copy_value(ctx, arr->elem[i], to_free, to_end); - } - break; - } - case OBJ_RECORD: { - JSRecord *rec = ptr; - if (rec->proto) { - JSValue pv = JS_MKPTR(JS_TAG_PTR, rec->proto); - pv = gc_copy_value(ctx, pv, to_free, to_end); - rec->proto = (JSRecord *)JS_VALUE_GET_PTR(pv); - } - uint64_t mask = objhdr_cap56(hdr); - for (uint64_t i = 0; i <= mask; i++) { - rec->slots[i*2] = gc_copy_value(ctx, rec->slots[i*2], to_free, to_end); - rec->slots[i*2+1] = gc_copy_value(ctx, rec->slots[i*2+1], to_free, to_end); - } - break; - } - case OBJ_FUNCTION: { - JSFunction *fn = ptr; - fn->code = gc_copy_value(ctx, fn->code, to_free, to_end); - fn->outer = gc_copy_value(ctx, fn->outer, to_free, to_end); - break; - } - case OBJ_FRAME: { - JSFrame *fr = ptr; - fr->function = gc_copy_value(ctx, fr->function, to_free, to_end); - fr->caller = gc_copy_value(ctx, fr->caller, to_free, to_end); - uint64_t cap = objhdr_cap56(hdr); - for (uint64_t i = 0; i < cap; i++) { - fr->slots[i] = gc_copy_value(ctx, fr->slots[i], to_free, to_end); - } - break; - } - case OBJ_TEXT: - case OBJ_BLOB: - case OBJ_CODE: - /* No references to scan */ - break; - } -} -``` - ---- - -## Phase 5: Remove Old GC Infrastructure - -### Files: source/quickjs.c, source/quickjs.h - -**Delete entirely:** -- `RC_TRACE` conditional code (~100 lines) -- `RcEvent` struct and `rc_log` array -- `rc_log_event`, `rc_trace_inc_gc`, `rc_trace_dec_gc`, `rc_dump_history` -- `gc_decref`, `gc_decref_child`, `gc_decref_child_dbg`, `gc_decref_child_edge` -- `gc_free_cycles`, `free_zero_refcount`, `free_gc_object` -- `add_gc_object`, `remove_gc_object` -- `JS_RunGCInternal`, `JS_GC_PHASE_*` enum -- `mark_children`, `gc_mark` (the old marking functions) -- `JSGCPhaseEnum`, `gc_phase` field in JSRuntime -- `gc_obj_list`, `gc_zero_ref_count_list` in JSRuntime -- `JSRefCountHeader` struct -- `js_alloc_string_rt` - no runtime string allocation -- Stone arena in runtime (`st_pages`, etc.) - each context has its own - -**Update:** -- `JS_FreeValue` - becomes no-op (GC handles everything) -- `JS_DupValue` - becomes no-op (just return value) -- `__JS_FreeValueRT` - remove entirely - ---- - -## Phase 6: Update Allocation Sites [IN PROGRESS] - -### 6.1 Replace js_malloc with ctx_alloc - -All object allocations change from: -```c -JSRecord *rec = js_mallocz(ctx, sizeof(JSRecord)); -rec->tab = js_mallocz(ctx, sizeof(JSRecordEntry) * size); -``` -to: -```c -size_t total = sizeof(JSRecord) + (mask+1) * 2 * sizeof(JSValue); -JSRecord *rec = ctx_alloc(ctx, total); -rec->hdr = objhdr_make(mask, OBJ_RECORD, false, false, false, false); -/* slots are inline after the struct */ -``` - -### 6.2 Update object creation functions - -- `JS_NewObject` - use ctx_alloc, set hdr, inline slots -- `JS_NewArray` - use ctx_alloc, set hdr, inline elements -- `JS_NewStringLen` - use ctx_alloc, create JSText -- `js_create_function` - use ctx_alloc -- String concatenation, array push, etc. - -### 6.3 Delete js_malloc/js_free usage for heap objects - -Keep `js_malloc_rt` only for: -- Class arrays (in runtime) -- VM stacks (external to GC'd heap) -- Temporary C allocations - ---- - -## Phase 7: Delete JSString, Use JSText - -### 7.1 Replace JSString with JSText - -```c -/* OLD - DELETE */ -struct JSString { - JSRefCountHeader header; - uint32_t pad; - objhdr_t hdr; - int64_t len; - uint64_t u[]; -}; - -/* NEW - Single text type */ -typedef struct { - objhdr_t hdr; /* type=OBJ_TEXT, cap=capacity, s=stone */ - uint64_t len_or_hash; /* length if s=0, hash if s=1 */ - uint64_t packed[]; /* 2 UTF32 chars per word */ -} JSText; -``` - -### 7.2 Update string functions - -- `js_alloc_string` -> allocate JSText via ctx_alloc -- Remove `js_alloc_string_rt` entirely -- `js_free_string` -> no-op (GC handles it) -- Update all JSString* to JSText* - -### 7.3 Text stoning - -When a text is used as a record key or becomes a literal: -1. Set stone bit: `hdr = objhdr_set_s(hdr, true)` -2. Compute hash and store in len_or_hash -3. Set capacity = length -4. Add to context's intern table - ---- - -## Phase 8: Update Type Checks - -Replace `JSGCObjectHeader.gc_obj_type` checks with `objhdr_type`: - -```c -/* Old */ -((JSGCObjectHeader *)ptr)->gc_obj_type == JS_GC_OBJ_TYPE_RECORD - -/* New */ -objhdr_type(*(objhdr_t *)ptr) == OBJ_RECORD -``` - -Update helper functions: -- `js_is_record(v)` - check `objhdr_type == OBJ_RECORD` -- `js_is_array(v)` - check `objhdr_type == OBJ_ARRAY` -- `js_is_function(v)` - check `objhdr_type == OBJ_FUNCTION` -- `JS_IsString(v)` - check `objhdr_type == OBJ_TEXT` - ---- - -## Phase 9: Handle C Opaque Objects - -Per docs/memory.md, C opaque objects need special handling: - -**9.1 Track live opaque objects** -```c -typedef struct { - void *opaque; - JSClassID class_id; - uint8_t alive; -} OpaqueRef; - -/* In JSContext */ -OpaqueRef *opaque_refs; -int opaque_ref_count; -int opaque_ref_capacity; -``` - -**9.2 During GC** -1. When copying a JSRecord with opaque data, mark it alive in opaque_refs -2. After GC, iterate opaque_refs and call finalizer for those with `alive=0` -3. Clear all alive flags for next cycle - ---- - -## File Changes Summary - -### source/quickjs.c -- Remove ~800 lines: RC_TRACE, gc_decref, gc_free_cycles, JSGCObjectHeader, JSString, runtime string arena -- Add ~300 lines: buddy allocator, Cheney GC, JSText -- Modify ~400 lines: allocation sites, type checks, string handling - -### source/quickjs.h -- Remove: JSGCObjectHeader, JSRefCountHeader from public API -- Remove: JS_MarkFunc, JS_MarkValue -- Update: JS_FreeValue, JS_DupValue to be no-ops - ---- +1. **First**: Add DupValue/FreeValue macros (enables compilation) +2. **Second**: Standardize struct layouts (header at offset 0) +3. **Third**: Fix gc_object_size and gc_copy_value +4. **Fourth**: Complete gc_scan_object for all types +5. **Fifth**: Update ctx_gc with complete root tracing +6. **Sixth**: Wire js_malloc to trigger GC +7. **Seventh**: Add frame reduction for closures +8. **Finally**: Remove ref counting dead code ## Verification -1. **Build**: `make` should compile without errors -2. **Basic test**: `./cell test suite` should pass -3. **Memory test**: Run with ASAN to verify no leaks or corruption -4. **GC trigger**: Test that GC runs when memory fills, objects survive correctly +1. **Compile test**: `make` should succeed without errors +2. **Basic test**: Run simple scripts: + ```js + var a = [1, 2, 3] + log.console(a[1]) + ``` +3. **Stress test**: Allocate many objects to trigger GC: + ```js + for (var i = 0; i < 100000; i++) { + var x = { value: i } + } + log.console("done") + ``` +4. **Closure test**: Test functions with closures survive GC: + ```js + fn make_counter() { + var count = 0 + fn inc() { count = count + 1; return count } + return inc + } + var c = make_counter() + log.console(c()) // 1 + log.console(c()) // 2 + ``` +5. **GC stress with closures**: Create many closures, trigger GC, verify they still work ---- +## Key Design Decisions (Resolved) -## Dependencies / Order of Work - -1. Phase 1 (Buddy) - DONE -2. Phase 2 (JSContext bump alloc) - DONE -3. Phase 5 (Remove old GC) - Do this early to reduce conflicts -4. Phase 7 (Delete JSString) - Major cleanup -5. Phase 3 (Unify headers) - Depends on 5, 7 -6. Phase 6 (Allocation sites) - Incremental, with Phase 3 -7. Phase 4 (Cheney GC) - After Phases 3, 6 -8. Phase 8 (Type checks) - With Phase 3 -9. Phase 9 (Opaque) - Last, once basic GC works - ---- - -## Notes - -- Each context owns its memory - no runtime-level allocation except buddy blocks -- Text interning is per-context, rebuilt during GC -- OBJ_CODE lives in context memory (always stone) -- Frames use caller=null to signal returnable (can be shrunk during GC) -- Forward pointer type (7) used during GC to mark copied objects -- `heap_free - heap_base` = total memory used by context +1. **JSCode storage**: Lives in stone (immutable) memory, never copied during GC ✓ +2. **Header offset**: Standardized to offset 0 for all heap objects ✓ +3. **Closure variables**: Live in JSFrame objects; frames are "reduced" when functions return ✓ +4. **JSVarRef**: Eliminated - closures reference their outer frame directly ✓