open https://gitlab.synchro.net/main/sbbs/-/issues/1143

## Summary

`MsgBase.get_all_msg_headers()` header objects return `undefined` on the **first** (cold) access of a `LAZY_STRING_TRUNCSP_NULL` field (`to_ext`, `from_ext`, `replyto`, `to_list`, `cc_list`, `summary`, `tags`, `from_org`, …) via **dot/literal** property access — even when the field is populated. Touching any other (non-NULL) field first "primes" the object and the field then reads correctly.

This issue documents the **proven root cause** and leaves the fix direction open for discussion. Two prior commits are involved:

- `ca448cb8b` — eager `JS_DefineProperty("number")` per header. **Ineffective** on real bases (see below).
- `666ff71ce` — eager full-resolve per header. **Works**, but abandons the lazy design, and its commit message overstated the mechanism as proven (it was written before the decisive evidence below existed). The message should be corrected regardless of the final fix.

## Reproduced (read-only, live ~7.2k-msg mail base)

With the pre-fix binary, cold `hdr.to_ext` → **7138 `undefined`** of **7026** actually-populated `to_ext`; priming with `.to` first → all correct. Strictly read-only (no `save_msg`/`remove_msg`). Repro scripts attached.

## Root cause (evidence-based)

SpiderMonkey 1.8.5's **`JSOP_GETPROP` per-(shape, pc) property cache**, interacting with `js_msghdr_class`'s **old-style resolve hook** (`js_get_msg_header_resolve`) that **conditionally** defines `*_NULL` fields — the field becomes an own property iff `p->msg.<field>` is non-NULL.

All bulk headers are created identically (`JS_NewObject(&js_msghdr_class, proto, …)`) and the hook defines fields in a fixed order, so they **share one shape lineage** but have **divergent real property sets** (a field is present on some headers, absent on others). The per-`(shape, pc)` GETPROP cache can't represent "sometimes present", so once an entry for `(shared-shape, .to_ext)` is established it mis-serves `undefined` to same-shape headers.

## Three discriminators that nailed it

| Test | Result | Conclusion |
|------|--------|------------|
| Siblings (`attr`/`number`/`to`) after a cold `to_ext` miss | **alive on all 7140** | `p->enumerated` is **not** set; resolve hook runs fine |
| `hdr["to_ext"]` (computed, **JSOP_GETELEM**) | undef = 213 = true NULL count → **correct** | not the resolve hook / SMB layer / data |
| `hdr.to_ext` (literal, **JSOP_GETPROP**) | undef = 7138 → **the bug** | it's specifically the literal-name property cache |
| cold-defined-but-*wrong* | **0** | purely spurious `undefined`, never bad data |

SM source confirms the shape of it: `js_GetPropertyHelperWithShapeInline` fills the cache **only on the found path** (`jsobj.cpp:5438`) and explicitly **not** on a miss (`nofills++`, `jsobj.cpp:5366`) — i.e. found-then-mispredict across shared shapes, not negative caching.

### Why the prior fixes behaved as they did

- `ca448cb8b` ("define number") can't work: it just moves every header off the empty shape onto **one shared** `{number,…}` shape; the conditional-property divergence is untouched.
- `666ff71ce` (eager full-resolve) **does** address the root cause — every populated field becomes a real own property at construction, so GETPROP finds them by normal scope lookup with consistent shapes, no conditional resolve-add divergence. The cost is the loss of laziness.

## Notes on hypotheses raised in discussion

- **`p->enumerated` spuriously set** — ruled out (siblings alive on the same object after a cold miss).
- **`js_get_msg_header_resolve(…, JSID_VOID)` early-returning through macros** — doesn't happen; in the `name==NULL` path every `LAZY_*` macro only `return`s when `name` is non-NULL, so it falls through and defines all available fields, and that path never sets `enumerated`.

## Candidate fixes (undecided)

1. **`JSCLASS_NEW_RESOLVE`** conversion — the SM-sanctioned way to keep laziness; hook sets `*objp` only when it actually defines a field. Must be built and proven against the live base (not guaranteed to fix it).
2. **Keep eager-resolve** (`666ff71ce`) and just correct the commit message; optionally benchmark eager vs. lazy on the live base to quantify perf.
3. **Lazy-define-as-null variants** that don't change the public contract (e.g. only in the bulk path, or non-enumerable null) so same-shape headers stop diverging.

## Reproducers (attached)

Both run via `jsexec -c /sbbs/ctrl <script>` against the live `mail` base, strictly read-only:

- [probe_to_ext.js](/uploads/e84b9f7807c1bc07057efb35633a6e62/probe_to_ext.js) — cold vs. primed `to_ext` counts, transition point, mismatch detection.
- [probe_enum.js](/uploads/b261bac444a39ba2730c7b0fdaed12dd/probe_enum.js) — the discriminators: sibling-after-miss, and GETELEM-vs-GETPROP per-field cold reads.
--- SBBSecho 3.37-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/1143#note_8987

**SpiderMonkey 1.8.5 per-shape property-cache implementation — references**

All paths relative to the vendored tree `3rdp/src/mozjs/js-1.8.5/js/src/`.

Canonical doc (cited in the header itself, `jspropertycache.h:51-53`): MDN *SpiderMonkey/Internals/Property cache* — `https://developer.mozilla.org/en/SpiderMonkey/Internals/Property_cache` (now in archived Mozilla wiki content).

**Data structures — `jspropertycache.h`**
- `class PropertyCache` — `:160`; fixed-size direct-mapped table, `SIZE_LOG2 = 12` → 4096 entries (`:164-169`).
- `struct PropertyCacheEntry` — `:121`; key = `kpc` (testing bytecode pc, `:123`) + `kshape` (shape of the key object, `:124`); value = `vcap`/`vword`.
- hash over **(pc, kshape)** — `hash()` `:215` (`((pc>>SIZE_LOG2 ^ pc) + kshape) & MASK`).
- entry kinds: `adding()` `:128` (kshape≠vshape, predicted shape transition) vs `directHit()` `:129` (kshape==vshape).
- `PCVal` tagged union (slot / shape / fun-obj) — `:81`.

**Hit/lookup path — `jspropertycacheinlines.h`**, `PropertyCache::test()` `:73`
- `kshape = obj->shape()` `:78`; `entry = table[hash(pc, kshape)]` `:79`.
- validated by `entry->kpc == pc && entry->kshape == kshape` `:84`, then `matchShape(cx, pobj, entry->vshape())` `:92`.
- **On a validated hit it returns the cached PCVal and never calls the resolve hook** — the entry is keyed purely on `(pc, shape)`, and all bulk header objects share one shape lineage. This is the crux.

**Fill — `jspropertycache.cpp`**, `PropertyCache::fill()` `:51`
- where entries are created (the predicted-transition / `adding` bookkeeping), with bail-outs `:62-128` (cache disabled, `!nativeContains`, dictionary-mode `:84`, non-native protos, overdeep chains).

**GETPROP wiring — `jsobj.cpp`**
- found-path fill: `JS_PROPERTY_CACHE(cx).fill(cx, aobj, 0, protoIndex, obj2, shape)` — `:5438` (under `JSGET_CACHE_RESULT`).
- miss path that explicitly does **not** fill (`nofills++`) — `:5366`.

**Shapes — `jsscope.cpp`**: `js_GenerateShape()` `:74` / `:99`; `Shape` in `jsscope.h`.

The empirically-anchored part is just the first half: `test()` keys on `(pc, obj->shape())` and a validated hit bypasses resolve. The exact entry-transition bookkeeping that lets a same-shape header get mis-served `undefined` lives in the interplay of `fill()` (`:51`) and the `adding()`/`directHit()` validation in `test()` — that's the spot to confirm the precise misprediction.
--- SBBSecho 3.37-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/1143#note_8988

A few problems with the diagnosis I want to surface before we move on.

1. JIT is off in the build that produced these counts.

JAVASCRIPT_OPTIONS in sbbsdefs.h:69 is 0x810 = JSOPTION_JIT | JSOPTION_COMPILE_N_GO, which is what global->js.options defaults to (sbbs_ini.c:317) and what jsexec uses (jsexec.cpp:63). JSOPTION_JIT is bit 11, which is TraceMonkey, not MethodJIT, and js-confdefs.h for this tree compiles in JS_METHODJIT, JS_MONOIC, JS_POLYIC but not JS_TRACER. TraceMonkey is gone entirely. MethodJIT (with its PolyIC/MonoIC inline caches) is compiled in but requires JSOPTION_METHODJIT (bit 14 = 0x4000), which is not in 0x810.

The only caching layer that could plausibly produce the reported behavior under defaults is therefore the interpreter PropertyCache in jspropertycache.cpp.

2. The interpreter PropertyCache as written can't mis-serve the way the issue describes.

Trace from fill() to test():

- The GET-path fill at jsobj.cpp:5432 (js_GetPropertyHelperWithShapeInline) passes adding=JS_FALSE (default, no third arg).
- In fill() (jspropertycache.cpp:51), with adding=FALSE and a default getter, control reaches the if (kshape == 0) block at line 256, which sets kshape = obj->shape() and vshape = pobj->shape(). By this point the resolve hook has already called JS_DefineProperty and the shape has transitioned, so this obj->shape() reads post-transition (call it S_post).
- test() at jspropertycacheinlines.h:73 reads kshape = obj->shape() for the receiver currently being looked up (line 78), then rejects unless entry->kshape == kshape (line 84).

A fresh header at S_init reading hdr.to_ext therefore has kshape = S_init in test(). Any entry filled by a populated-sibling header is at kshape = S_post. The kshape check fails, control falls through to fullTest, then to js_GetPropertyHelper, which calls the resolve hook. There is no path in the interpreter cache I can find where a same-S_init header gets a hit on an entry filled by an S_post header.

3. Other candidate mechanisms considered and discarded:

- Cached miss / negative caching of "this id not on this shape": PCVal (jspropertycache.h:81-119) has four states (Null, FunObj, Slot, Shape), no negative state. fill() requires a real Shape* and bails via JS_NO_PROP_CACHE_FILL if !pobj->nativeContains(*shape). Every fill call site passes a real shape.
- Proto-chain contamination (a stray to_ext = undefined defined on the shared msghdr prototype): falsified by the very GETELEM-vs-GETPROP discriminator. If the proto held the entry, GETELEM's proto walk in js_LookupPropertyWithFlagsInline at jsobj.cpp:4985-5009 would find it and return undef too. The reported GETELEM count of 213 (matching true-NULL count) rules this out.
- "Shape priming" from ca448cb8b: not a real SpiderMonkey concept. Shapes transition via PropertyTree::getChild (jspropertytree.cpp:204-243) which returns existing nodes for matching transitions, so two headers both adding "number" land on the same destination shape. There's no per-object priming distinct from shape.
- JIT PolyIC misprediction (methodjit/PolyIC.cpp): would have been my next candidate, except JIT is off per (1).

4. What this leaves.

Either (a) there is a path in fullTest (jspropertycache.cpp:320-410) or the vcapTag()==1 proto-hit branch in test() (jspropertycacheinlines.h:87-90) that I have not traced and that does in fact mis-serve under shared-shape conditions, or (b) the reported counts are not measuring what the writeup says they measure.

I can't rule out (a) without sitting in the debugger. But (b) is the more parsimonious read of where this issue's history has been so far: ca448cb8b's commit-message theory did not survive scrutiny, 666ff71ce's did not either, and the experimental writeup cites a mechanism (per-(shape, pc) cache hit on a same-S_init fresh header) that does not reproduce against a read of the source on JIT-off SM 1.8.5.

Request: please attach probe_enum.js and probe_to_ext.js, or paste them inline.

The numbers (cold to_ext = 7138 undef, GETELEM = 213, sibling-alive = 7140, cold-defined-but-wrong = 0) are load-bearing for the diagnosis, and without seeing how each was counted, I can't tell whether the discriminator was actually exercised, or whether (for example) the GETELEM pass was inadvertently warmed by something that ran earlier in the same script. With the scripts in hand we can re-run them, and either confirm the misprediction is real (in which case the next step is gdb on the interpreter cache to find the path I missed) or identify the script artifact.
--- SBBSecho 3.37-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/1143#note_8991

Zooming out: unchecked JS_* returns is the broader pattern

The lost-exception trap in LAZY_STRING_TRUNCSP_NULL / LAZY_STRING_TRUNCSP is not unique to those macros. It's how nearly the entire JS binding layer is written.

Quick count in this tree:

- js_msgbase.cpp: 28 JS_DefineProperty calls, 0 with a return-value check.
- Whole tree: 683 JS_DefineProperty calls, ~108 inside any if, and most of those are conditional calls (if (cond) JS_DefineProperty(...)) rather than checks on the return. The actual checked-return rate is well under 10%.
- Same pattern for JS_NewObject, JS_NewStringCopyZ, JS_NewArrayObject, JS_SetProperty, JS_DefineElement.

Any allocation failure in any of those sites sets a pending exception on cx and is silently swallowed. The script proceeds with whatever the default observable is (NULL pointers from JS_GetPrivate, undefined from skipped property defines, no-op'd setup). The exception sits on cx until something else trips on it (or jsexec reports it at script termination).

Suggested test to surface the scope

Run the same get_all_msg_headers fetch under a deliberately undersized heap, read several LAZY-resolved fields per header, count undef per field:

```
// Set JavaScriptMaxBytes very low in jsexec.ini first, e.g. 4M or 2M.
var fields = ["to", "from", "subject", "to_ext", "from_ext",
"to_list", "cc_list", "summary", "tags",
"replyto", "from_org", "editor"];
var u = {};
for (var f = 0; f < fields.length; f++) u[fields[f]] = 0;

var caught = null;
try {
for (var n in headers) {
var h = headers[n];
for (var f = 0; f < fields.length; f++) {
if (h[fields[f]] === undefined)
u[fields[f]]++;
}
}
} catch (e) { caught = e; }

for (var f = 0; f < fields.length; f++)
print(fields[f] + ": " + u[fields[f]] + " undef");
print("caught: " + caught);

// flush any pending exception that lingered past loop end
try { var s = ""; for (var i = 0; i < 50000; i++) s += "x"; }
catch (e) { print("post-loop: " + e); }
```

If OOM under a small heap is the trigger, all these fields should show elevated undef counts, not just the *_NULL ones. to/from/subject are routed through LAZY_STRING_TRUNCSP, which has the same swallow-and-undefine behavior on JS_NewStringCopyZ failure. If only to_ext-shaped fields show it, OOM isn't the (whole) story.

I'll file the unchecked-return audit as a separate issue so it doesn't tangle with the property-cache thread in this one. The point of this comment is just: even after #1143 is closed, the same defect pattern is going to keep producing "mysterious" symptoms across the codebase until it gets swept.
--- SBBSecho 3.37-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/1143#note_8993

**OOM / allocation-failure (#1144) ruled out as the trigger for this symptom**

Tested the open question of whether allocation failure (`JS_NewStringCopyZ` → NULL under heap pressure, the LAZY_* fall-through path described in #1144) is what produces the cold-`to_ext`-undefined here. Two independent lines of evidence say no:

**1. Heap invariance.** Ran the cold/primed probe against the live mail base at three heap sizes (`jsexec -m<bytes>`): 16 MB (default), 256 MB, 1 GB. Bit-identical results every time:

| heap | total | cold to_ext undef | BUG mismatches | cold defined-but-wrong | |------|-------|-------------------|----------------|------------------------| | 16 MB | 7294 | 7193 | 6971 | 0 |
| 256 MB | 7294 | 7193 | 6971 | 0 |
| 1 GB | 7294 | 7193 | 6971 | 0 |

If allocation pressure were the trigger, a 64× larger heap would relieve it and the count would drop — it didn't move by a single header. An OOM-driven failure would also be non-deterministic (GC timing / high-water mark dependent); this is fully deterministic.

**2. GETELEM returns the real string** (decisive, heap-independent). If `JS_NewStringCopyZ` had returned NULL, the string would never exist and the property would never be defined — so `hdr["to_ext"]` (JSOP_GETELEM) would *also* read undefined. But GETELEM returns the correct value (`to_ext` undef = 213 = the true NULL count), while `hdr.to_ext` (JSOP_GETPROP) reads undefined. The string **was** allocated and the property **was** defined; only the GETPROP property-cache lookup path fails.

So the symptom in this issue is the `JSOP_GETPROP` property-cache mispredict across same-shape headers, not the #1144 allocation-failure path.

**This does not invalidate #1144.** The unchecked `JS_*` returns are a separable, real latent defect that would bite under genuine allocation pressure (and silently swallow the pending exception), exactly as described there — it's just not the mechanism firing in this reproduction.
--- SBBSecho 3.37-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/1143#note_9007

**Read-only reproduction attempt on a clean copy of the mail base — symptom does not reproduce on FreeBSD**

I took a verbatim copy of your `data/mail.*` files (`mail.shd` + `mail.sdt` + `mail.sda` + `mail.sha` + `mail.sid` + `mail.ini`, all read-only) and ran `probe_to_ext.js` against them on a FreeBSD 14.3 build of the same tree, across three binaries to bracket the fix range.

## Method

- `probe_to_ext_path.js` — byte-identical to the attached `probe_to_ext.js` except the `fetch()` body calls `new MsgBase("/tmp/issue1143/mailbase/mail", true /* is_path */)` instead of `new MsgBase("mail")`, so the test runs against the copy without touching the BBS config.
- All three binaries built from this tree with `gmake` (Clang 19, FreeBSD amd64, debug). Same `JAVASCRIPT_OPTIONS=0x810` (`sbbsdefs.h:69`) — JIT off, as discussed above.
- Same heap size (`-m256000000`), same cwd, same script invocation.

| binary | js_msgbase.cpp content | result | |--------|------------------------|--------|
| `666ff71ce` (current HEAD) | eager-resolve `JSID_VOID` + `defer_listing` | total=7482, cold_undef=213, primed_NULL=213, **BUG mismatches=0**, defined-but-wrong=0 |
| `666ff71ce^` (= ca448cb8b) | eager `JS_DefineProperty("number")` only | total=7482, cold_undef=213, primed_NULL=213, **BUG mismatches=0**, defined-but-wrong=0 |
| `ca448cb8b^` (pre-both fixes) | vanilla lazy resolve, no eager defines | total=7482, cold_undef=213, primed_NULL=213, **BUG mismatches=0**, defined-but-wrong=0 |

`cold_undef == primed_NULL == 213` exactly in every run — i.e. the only `undefined` reads are the genuinely-NULL `to_ext` fields. There are zero spurious undefineds. Both fixes are no-ops on this build, because there is no symptom to fix.

(Investigation lives on local branch `investigate-1143`: two `git revert` commits on top of master.)

## What this tells us

The defect is **not** in the SM-1.8.5 source we share, and it's not in the lazy resolve hook as currently written. On Clang/FreeBSD/debug against Rob's actual mail base, the interpreter PropertyCache behaves exactly the way my earlier source-trace said it should: a fresh-shape header looking up `to_ext` misses the cache (kshape diverges) and goes through resolve normally.

That means whatever is misfiring is environment-specific to Rob's setup in some way the source tree alone doesn't capture. I don't know what platform Rob ran his probe on — his commits are authored `(on Windows 11)` but the `-c /sbbs/ctrl` / `-m<bytes>` invocation style in discussion-4 reads as Unix shell, so I shouldn't assume. Candidates worth narrowing:

- Compiler / build flags. MSVC vs Clang vs gcc codegen on `PropertyCache::test()` / `fill()` / `js_GetPropertyHelperWithShapeInline` — a vectorization, alias-analysis, or signed-overflow assumption that diverges between compilers can land us in a different cache fast path.
- mozjs build configuration. Whether `JS_METHODJIT` / `JS_MONOIC` / `JS_POLYIC` / `DEBUG` / `JS_GC_ZEAL` are compiled into the libjs Rob is linking matches what I tested here. The `3rdp/` mozjs is built once per host config; differences in that build are not visible in the sbbs source tree.
- Release vs Debug. I tested debug here. If Rob's symptom is on a Release build, optimizer-enabled cache paths could behave differently.
- Compile-time options I missed.

## Open questions for Rob

To narrow this further:

1. **What platform / build did your probe runs in discussion-4 actually use?** Specifically: which jsexec binary (path), built on which host, Debug or Release, and which `3rdp/.../mozjs/lib` it links against. I assumed Windows because of the commit author tag and now realize I shouldn't have.
2. What are the build flags on the mozjs library you're linking? `JS_METHODJIT`, `JS_MONOIC`, `JS_POLYIC`, `DEBUG`, `JS_GC_ZEAL`, etc. — anything that affects the cache fast-path is in scope.
3. Does the symptom reproduce on a *fresh local copy* of `data/mail.*` opened by path (`new MsgBase(path, true)`), or only against the live SMB-mounted one? If only against the live one, the SMB-share layer is back in scope as the differentiator.

I can also build a FreeBSD release-optimized jsexec and rerun the probe here, if that helps rule optimization level in or out without waiting on you.

## Side audit (separable from this issue)

While walking `js_get_all_msg_headers` I noticed the read-path smblib errors are swallowed silently:

- `smb_locksmbhdr` failure at `js_msgbase.cpp:1932-1937` → `return JS_TRUE` with the empty `retobj` already set as RVAL. Script sees an empty msgbase, not an error.
- `smb_getmsghdr` failure mid-loop at `js_msgbase.cpp:2010-2015` → silent loop break, partial array returned, script can't tell it was truncated.
- `JS_NewObject` failure mid-loop at `js_msgbase.cpp:2017-2023` → same silent truncation.
- Same pattern in `js_get_msg_header` at lines 1771, 1777, 1783 (single-msg path).

`priv->smb_result` is set in each case, so `mb.last_error` does reflect the failure — but no caller of `get_all_msg_headers` in the tree (msglist.js, hotline.js, the probes themselves) checks `last_error` after the call. Given that Rob's setup accesses the mail base over an SMB-mounted share where locking races have bitten before, this is worth tightening on its own merits.

Doesn't explain the `to_ext`-undef symptom (failed-to-read headers are skipped from the array, not partially populated), but it's a real defect class adjacent to #1144 — file as its own ticket either way.
--- SBBSecho 3.37-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/1143#note_9008

**Root cause is TraceMonkey (`JSOPTION_JIT` / `JS_TRACER`), not the interpreter PropertyCache — and that resolves the FreeBSD/Linux divergence**

@Deuce — your FreeBSD non-repro and your source-trace were both correct. The interpreter PropertyCache *can't* mis-serve here, exactly as you traced. The misprediction is one layer up, in the **trace JIT**, which your build doesn't compile and mine does. Answering your three questions in order, then the decisive A/B.

## Q1 — platform / build of the probe runs

Not Windows (the commit author tag misled us). The repro is on **Linux**:

- `Linux 6.1.0-37-amd64 x86_64`, **gcc (Debian) 14.2.0**, **debug** build of this tree.
- jsexec: built from HEAD with both fix commits reverted (local branch = two `git revert`s on master), in an isolated worktree. mozjs is **statically linked into `libsbbs.so`** (no separate .so).
- Invocation: `jsexec -c /sbbs/ctrl <probe>` against a **read-only copy** of my `data/mail.*` opened by path — same method you used.

## Q2 — mozjs build flags I'm linking

`js-confdefs.h` is **build-generated** here (not tracked in git), and on this Linux/gcc build it contains:

```
#define JS_METHODJIT 1
#define JS_MONOIC 1
#define JS_POLYIC 1
#define JS_POLYIC_TYPED_ARRAY 1
#define JS_PUNBOX64 1
#define JS_THREADSAFE 1
#define JS_TRACER 1 <-- present here; you reported it ABSENT on your tree
```

The linked `libmozjs185-1.0.a` contains the TraceMonkey objects (`TraceRecorder`, `MonitorLoopEdge`, `js::InitJIT`, …). So **TraceMonkey is compiled in on my build and gone on yours** — the single difference that matters.

At runtime, `js.options` reads **`0x810`** = `JSOPTION_JIT (0x800) | JSOPTION_COMPILE_N_GO (0x10)`. That's the active default for non-JSDOOR jsexec (`jsexec.cpp:63`, `JAVASCRIPT_OPTIONS` in `sbbsdefs.h:69`) — you had this right. Note bit 11 = `JSOPTION_JIT` = **TraceMonkey**; `JSOPTION_METHODJIT` (bit 14, 0x4000) is **not** set. So the active JIT in the repro is TraceMonkey alone.

## Q3 — does it reproduce on a fresh local copy opened by path?

**Yes.** Verbatim read-only copy of `data/mail.*` → `/tmp/maildup/mail.*`, opened with `new MsgBase("/tmp/maildup/mail", true /* is_path */)`. Pre-fix binary: total=7522, genuinely-NULL `to_ext`=215, **cold `hdr.to_ext` undefined=7483, BUG mismatches=7268, defined-but-wrong=0.** So **the SMB-mounted-share layer is ruled out** — it's purely the binary/build, against identical bytes.

## The decisive A/B (same binary build procedure, only the JIT bit changes)

I rebuilt jsexec twice, flipping only `JSOPTION_JIT` in the compiled default, and confirmed the active options via `js.options` each time:

| `js.options` | TraceMonkey | total | cold `to_ext` undef | genuinely-NULL | **BUG mismatches** | defined-but-wrong |
|---|---|---|---|---|---|---|
| `0x810` (stock) | **on** | 7522 | 7483 | 215 | **7268** | 0 |
| `0x10` (`COMPILE_N_GO` only) | **off** | 7522 | **215** | 215 | **0** | 0 |

With the tracer off, cold `to_ext` undefined collapses to exactly the genuinely-NULL count (215) — zero spurious undefineds. **TraceMonkey is necessary and sufficient for the symptom.**

(Methodological note that explains why this took a second pass: my first "JIT off" rebuild edited the `#ifdef JSDOOR` branch at `jsexec.cpp:66/68`, which is dead code for a normal jsexec — the active default is `jsexec.cpp:63`. `js.options` still read `0x810`, i.e. JIT was still on. Reading `js.options` at runtime is what caught it; the table above is from editing line 63 and confirming `js.options=0x10`.)

## What this means for the diagnosis and the fix

- The earlier "interpreter `JSOP_GETPROP` per-(shape,pc) PropertyCache mispredict" writeup (and `666ff71ce`'s commit message) named the **wrong cache**. The mechanism is the **same shape** — bulk headers share one shape lineage but diverge in whether a `*_NULL` field is an own property — but it's the **trace recorder's** shape-guarded GETPROP on the hot `for..in` dot-access loop that records a read of an absent slot and replays `undefined`, not the interpreter cache.
- Consistent with the prior evidence: **heap-invariant** (not allocation/#1144), **GETELEM-immune** (`JSOP_GETELEM` traces differently from `JSOP_GETPROP`), **defined-but-wrong = 0** (the guard passes on shape, the slot read just yields a hole). The `for..in` loop is exactly the hot path the tracer targets, which is why priming with a non-NULL field first dodges it.
- Why `666ff71ce` (eager full-resolve) works: every populated field becomes a real own property at construction, so there's no "present on some headers, absent on others" divergence under a shared shape for the tracer to misrecord. Cost is the loss of laziness, as discussed.

## Candidate directions (updated)

1. **`JSCLASS_NEW_RESOLVE`** — still the SM-sanctioned lazy route; would need to be proven against the live base **with TraceMonkey on**, not just in the interpreter.
2. **Keep eager-resolve** (`666ff71ce`) and **correct its commit message** to say "trace-JIT GETPROP misrecord across shared-shape headers with conditional `*_NULL` resolve," not "interpreter property cache."
3. **Disable TraceMonkey** for these workloads — `JS_TRACER` is dead upstream (gone from your tree already); turning bit 11 off (or not compiling it) sidesteps the whole class. Worth deciding whether SBBS should still be shipping `JSOPTION_JIT` at all on builds where `JS_TRACER` is compiled in.

Happy to run any of these against the copy here (Linux/gcc, tracer on) to validate before you commit to one. I can also bisect the exact trace-recorder GETPROP path under gdb if you want the precise misrecord site.

Reproducer used: `probe_to_ext.js` with `new MsgBase("mail")` swapped for `new MsgBase("/tmp/maildup/mail", true)`; everything else byte-identical to the attached scripts.
--- SBBSecho 3.37-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/1143#note_9009

**Correction to my Q1 above: the symptom is on Windows _and_ Linux, not Linux-only**

Rob clarified the history: this was **originally discovered on Windows** (MSVC build) and then **reproduced on both Windows and Linux**. My note above said "Not Windows" — that was wrong, and I'm glad it's wrong, because it makes the picture cleaner, not muddier.

Both the Windows (MSVC) and Linux (gcc 14.2) builds compile `JS_TRACER`. Your FreeBSD/Clang build does not. So the reproduce/no-reproduce split lines up **exactly** with "is TraceMonkey compiled in," across **three** different compilers:

| build | compiler | `JS_TRACER` | reproduces? | |-------|----------|-------------|-------------|
| Windows | MSVC | compiled in | **yes** (original discovery) |
| Linux | gcc 14.2 | compiled in | **yes** (the A/B above) |
| FreeBSD | Clang 19 | not compiled | **no** (your runs) |

This **removes** the "gcc-vs-clang codegen quirk" hypothesis I floated. It isn't a per-compiler codegen accident — MSVC and gcc are very different backends and both reproduce. The single common factor is the trace JIT being present and enabled (`JSOPTION_JIT`, bit 11, in the active `0x810`), and the single factor on your side is its absence. The decisive A/B stands: flipping only that bit on one build (`js.options 0x810 → 0x10`) turns the 7268 spurious-undefineds to 0.

So: TraceMonkey's trace-recorded GETPROP over the shared-shape headers with conditionally-resolved `*_NULL` fields, confirmed across two independent toolchains. The fix candidates in my previous note are unchanged.
--- SBBSecho 3.37-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/1143#note_9008

**Root cause is TraceMonkey (`JSOPTION_JIT` / `JS_TRACER`), not the interpreter PropertyCache — and that resolves the FreeBSD/Linux divergence**

@Deuce — your FreeBSD non-repro and your source-trace were both correct. The interpreter PropertyCache *can't* mis-serve here, exactly as you traced. The misprediction is one layer up, in the **trace JIT**, which your build doesn't compile and mine does. Answering your three questions in order, then the decisive A/B.

## Q1 — platform / build of the probe runs

Not Windows (the commit author tag misled us). The repro is on **Linux**:

- `Linux 6.1.0-37-amd64 x86_64`, **gcc (Debian) 14.2.0**, **debug** build of this tree.
- jsexec: built from HEAD with both fix commits reverted (local branch = two `git revert`s on master), in an isolated worktree. mozjs is **statically linked into `libsbbs.so`** (no separate .so).
- Invocation: `jsexec -c /sbbs/ctrl <probe>` against a **read-only copy** of my `data/mail.*` opened by path — same method you used.

## Q2 — mozjs build flags I'm linking

`js-confdefs.h` is **build-generated** here (not tracked in git), and on this Linux/gcc build it contains:

```
#define JS_METHODJIT 1
#define JS_MONOIC 1
#define JS_POLYIC 1
#define JS_POLYIC_TYPED_ARRAY 1
#define JS_PUNBOX64 1
#define JS_THREADSAFE 1
#define JS_TRACER 1 <-- present here; you reported it ABSENT on your tree
```

The linked `libmozjs185-1.0.a` contains the TraceMonkey objects (`TraceRecorder`, `MonitorLoopEdge`, `js::InitJIT`, …). So **TraceMonkey is compiled in on my build and gone on yours** — the single difference that matters.

At runtime, `js.options` reads **`0x810`** = `JSOPTION_JIT (0x800) | JSOPTION_COMPILE_N_GO (0x10)`. That's the active default for non-JSDOOR jsexec (`jsexec.cpp:63`, `JAVASCRIPT_OPTIONS` in `sbbsdefs.h:69`) — you had this right. Note bit 11 = `JSOPTION_JIT` = **TraceMonkey**; `JSOPTION_METHODJIT` (bit 14, 0x4000) is **not** set. So the active JIT in the repro is TraceMonkey alone.

## Q3 — does it reproduce on a fresh local copy opened by path?

**Yes.** Verbatim read-only copy of `data/mail.*` → `/tmp/maildup/mail.*`, opened with `new MsgBase("/tmp/maildup/mail", true /* is_path */)`. Pre-fix binary: total=7522, genuinely-NULL `to_ext`=215, **cold `hdr.to_ext` undefined=7483, BUG mismatches=7268, defined-but-wrong=0.** So **the SMB-mounted-share layer is ruled out** — it's purely the binary/build, against identical bytes.

## The decisive A/B (same binary build procedure, only the JIT bit changes)

I rebuilt jsexec twice, flipping only `JSOPTION_JIT` in the compiled default, and confirmed the active options via `js.options` each time:

| `js.options` | TraceMonkey | total | cold `to_ext` undef | genuinely-NULL | **BUG mismatches** | defined-but-wrong |
|---|---|---|---|---|---|---|
| `0x810` (stock) | **on** | 7522 | 7483 | 215 | **7268** | 0 |
| `0x10` (`COMPILE_N_GO` only) | **off** | 7522 | **215** | 215 | **0** | 0 |

With the tracer off, cold `to_ext` undefined collapses to exactly the genuinely-NULL count (215) — zero spurious undefineds. **TraceMonkey is necessary and sufficient for the symptom.**

(Methodological note that explains why this took a second pass: my first "JIT off" rebuild edited the `#ifdef JSDOOR` branch at `jsexec.cpp:66/68`, which is dead code for a normal jsexec — the active default is `jsexec.cpp:63`. `js.options` still read `0x810`, i.e. JIT was still on. Reading `js.options` at runtime is what caught it; the table above is from editing line 63 and confirming `js.options=0x10`.)

## What this means for the diagnosis and the fix

- The earlier "interpreter `JSOP_GETPROP` per-(shape,pc) PropertyCache mispredict" writeup (and `666ff71ce`'s commit message) named the **wrong cache**. The mechanism is the **same shape** — bulk headers share one shape lineage but diverge in whether a `*_NULL` field is an own property — but it's the **trace recorder's** shape-guarded GETPROP on the hot `for..in` dot-access loop that records a read of an absent slot and replays `undefined`, not the interpreter cache.
- Consistent with the prior evidence: **heap-invariant** (not allocation/#1144), **GETELEM-immune** (`JSOP_GETELEM` traces differently from `JSOP_GETPROP`), **defined-but-wrong = 0** (the guard passes on shape, the slot read just yields a hole). The `for..in` loop is exactly the hot path the tracer targets, which is why priming with a non-NULL field first dodges it.
- Why `666ff71ce` (eager full-resolve) works: every populated field becomes a real own property at construction, so there's no "present on some headers, absent on others" divergence under a shared shape for the tracer to misrecord. Cost is the loss of laziness, as discussed.

## Candidate directions (updated)

1. **`JSCLASS_NEW_RESOLVE`** — still the SM-sanctioned lazy route; would need to be proven against the live base **with TraceMonkey on**, not just in the interpreter.
2. **Keep eager-resolve** (`666ff71ce`) and **correct its commit message** to say "trace-JIT GETPROP misrecord across shared-shape headers with conditional `*_NULL` resolve," not "interpreter property cache."
3. **Disable TraceMonkey** for these workloads — `JS_TRACER` is dead upstream (gone from your tree already); turning bit 11 off (or not compiling it) sidesteps the whole class. Worth deciding whether SBBS should still be shipping `JSOPTION_JIT` at all on builds where `JS_TRACER` is compiled in.

Happy to run any of these against the copy here (Linux/gcc, tracer on) to validate before you commit to one. I can also bisect the exact trace-recorder GETPROP path under gdb if you want the precise misrecord site.

Reproducer used: `probe_to_ext.js` with `new MsgBase("mail")` swapped for `new MsgBase("/tmp/maildup/mail", true)`; everything else byte-identical to the attached scripts.

— *Authored by Claude (Claude Code), on behalf of @rswindell*
--- SBBSecho 3.37-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/1143#note_9009

**Correction to my Q1 above: the symptom is on Windows _and_ Linux, not Linux-only**

Rob clarified the history: this was **originally discovered on Windows** (MSVC build) and then **reproduced on both Windows and Linux**. My note above said "Not Windows" — that was wrong, and I'm glad it's wrong, because it makes the picture cleaner, not muddier.

Both the Windows (MSVC) and Linux (gcc 14.2) builds compile `JS_TRACER`. Your FreeBSD/Clang build does not. So the reproduce/no-reproduce split lines up **exactly** with "is TraceMonkey compiled in," across **three** different compilers:

| build | compiler | `JS_TRACER` | reproduces? | |-------|----------|-------------|-------------|
| Windows | MSVC | compiled in | **yes** (original discovery) |
| Linux | gcc 14.2 | compiled in | **yes** (the A/B above) |
| FreeBSD | Clang 19 | not compiled | **no** (your runs) |

This **removes** the "gcc-vs-clang codegen quirk" hypothesis I floated. It isn't a per-compiler codegen accident — MSVC and gcc are very different backends and both reproduce. The single common factor is the trace JIT being present and enabled (`JSOPTION_JIT`, bit 11, in the active `0x810`), and the single factor on your side is its absence. The decisive A/B stands: flipping only that bit on one build (`js.options 0x810 → 0x10`) turns the 7268 spurious-undefineds to 0.

So: TraceMonkey's trace-recorded GETPROP over the shared-shape headers with conditionally-resolved `*_NULL` fields, confirmed across two independent toolchains. The fix candidates in my previous note are unchanged.

— *Authored by Claude (Claude Code), on behalf of @rswindell*
--- SBBSecho 3.37-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/1143#note_9016

**The OS split lines up with `3rdp/build/GNUmakefile:38-43` — TraceMonkey is compile-in-or-not based on host OS, and the runtime opt-in is unconditional**

Nice nail-down with the A/B. Walking it back into the build to surface why this divergence existed.

## Smoking gun in 3rdp/build/GNUmakefile

```makefile
# OS/2 and "GNU" (HURD?) also need this, but we never plan to support them. ifneq ($(os),linux)
ifneq ($(os),sunos)
JS_CONFIGURE_ARGS += '--disable-tracejit'
endif
endif
```

`--disable-tracejit` is passed to mozjs configure on every `gmake`-built OS **except Linux and Solaris**. The Windows build path is the MSVC `.vcxproj` chain, which doesn't go through this makefile, so `JS_TRACER` ends up compiled on Windows by mozjs's own defaults.

That gives:

| build path | OS | `--disable-tracejit` | `JS_TRACER` in lib | symptom | |------------|-----|----------------------|--------------------|---------|
| gmake | Linux | no | yes | **yes** |
| gmake | Solaris | no | yes | yes (presumed, not tested) |
| gmake | FreeBSD | yes | no | no |
| gmake | macOS | yes | no | no (presumed) |
| MSVC | Windows | n/a | yes (mozjs default) | **yes** |

Which matches the three-row table in your last note exactly. So the reproduce/no-reproduce split is fully predicted by "is `JS_TRACER` in the linked `libmozjs`," with no per-compiler quirk needed.

## Runtime side

`sbbsdefs.h:69` defines `JAVASCRIPT_OPTIONS = 0x810` = `JSOPTION_JIT (0x800) | JSOPTION_COMPILE_N_GO (0x10)`. That's the default options the BBS/jsexec/services pass to SpiderMonkey unless explicitly overridden. Bit 11 (`JSOPTION_JIT`) is the **TraceMonkey** opt-in. So on any build where `JS_TRACER` is compiled in, the tracer is also engaged at runtime — both halves of the bug are unconditional.

`JSOPTION_METHODJIT` (bit 14, 0x4000) is **not** set, so MethodJIT — which has the same general shape-guarded inline-cache structure (PolyIC/MonoIC) and therefore is plausibly susceptible to the same shared-shape / conditional-resolve mispredict — is dormant. Compiled in on Linux/Windows, never engaged.

## Why the prior diagnoses arrived where they did

- The interpreter-`PropertyCache` writeup in my discussion-2 source-trace was right that the interpreter cache can't mis-serve the way described. We were just looking at the wrong cache. The trace recorder operates **above** the interpreter, recording a shape-guarded GETPROP across the hot `for..in` loop and replaying `undefined` for the slot that wasn't an own property on the shape-twin headers.
- `666ff71ce`'s eager-resolve works because it eliminates the "present on some headers, absent on others under a shared shape" divergence the tracer is recording. The shape transitions per-header reflect actual property sets, and the trace replay slot is correct on every shape-match. So the fix is structurally sound — its commit-message theory just named the wrong cache. With the recorder on, the eager-resolve is the right shape of fix; with the recorder off, it's a no-op (which is consistent with our FreeBSD runs showing zero spurious undefineds across HEAD / ca448cb8b / both-reverted).

## Candidate fix paths (your call which to take — or which combination)

1. **Drop `JSOPTION_JIT` from `JAVASCRIPT_OPTIONS`** (`sbbsdefs.h:69`, change `0x810` → `0x10`). Single-line, ships immediately on every build with no mozjs rebuild needed. Stops the tracer from engaging even where it's compiled in. Probe rerun against the same base should drop spurious undefineds to 0 on Linux/Windows just as it did in your `js.options = 0x10` A/B.

2. **Extend `--disable-tracejit` to all OSes** in `3rdp/build/GNUmakefile:38-43` (remove the linux/sunos exemption) and do the equivalent on the MSVC mozjs build path. Removes `JS_TRACER` from the library entirely; future rebuilds are tracer-free. Requires a one-time mozjs rebuild on Linux and Windows.

3. **Revert `ca448cb8b` + `666ff71ce`** once (1) is in place. Both are correct workarounds for a tracer that's no longer in the picture; with the tracer off the original lazy resolve hook is correct. `defer_listing`, the eager `js_get_msg_header_resolve(JSID_VOID)` call, and the regression test all become unnecessary. (Optional — they're harmless if left in.)

4. **Add a comment near `JAVASCRIPT_OPTIONS`** (or wherever JIT options land) noting that `JSOPTION_METHODJIT` (bit 14) is not safe to enable without first re-running `probe_to_ext.js` / `probe_enum.js` against a real mail base — MethodJIT's PolyIC is shape-guarded the same way TraceMonkey is, and the same shared-shape conditional-resolve pattern would plausibly bite it.

(1) is the minimum surgical fix. (1)+(3) restores the laziness we lost. (2) is hygiene since SM-1.8.5's tracer is dead upstream and we've now demonstrated it's wrong even on the platforms the original exemption thought were OK. (4) is cheap insurance.

Reproducers and binaries are still in place on the FreeBSD investigation host if you'd like any of the above validated here before merging — happy to apply (1), rebuild, and rerun the probes against both the FreeBSD-local live mail base and Rob's copied base, to confirm zero spurious undefineds in both. Or against a Linux build if a peer wants to drive that side.

— *Authored by Claude (Claude Code), on behalf of @Deuce*
--- SBBSecho 3.37-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

close https://gitlab.synchro.net/main/sbbs/-/issues/1143
--- SBBSecho 3.37-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)

https://gitlab.synchro.net/main/sbbs/-/issues/1143#note_9040

**Fixed in `edf752429` (master) — option (1) from the discussion**

Took the minimum surgical fix: dropped `JSOPTION_JIT` from `JAVASCRIPT_OPTIONS` (`sbbsdefs.h:69`, `0x810` → `0x10`), and reverted the two prior workarounds (`ca448cb8b`, `666ff71ce`) since they were diagnosing the wrong cache. Kept `JSOPTION_COMPILE_N_GO`.

### Verified end-to-end on Windows/MSVC against the live VERT mail base

Build: Win32 Release `sbbs.dll` + `jsexec.exe` from `edf752429`. The new `jsexec` confirms the runtime change:

```
$ jsexec -r 'print(js.options.toString(16));'
10
```

Probe against the live VERT mail base (read-only, just `MsgBase("mail")` + `get_all_msg_headers()`):

```
js.options = 0x10
total=7698
cold_undef=216 (first .to_ext access on each header)
primed_NULL=216 (genuinely-NULL to_ext after priming with .to)
BUG mismatches=0
```

`cold_undef == primed_NULL` exactly. Zero spurious undefineds across 7698 headers — same shape of result @Deuce got on FreeBSD/Clang (where the tracer isn't compiled).

### Test coverage

`exec/tests/msgbase/get_all_msg_headers.js` rewritten as a JIT-off regression guard:

1. Hard-asserts `(js.options & 0x800) == 0` and throws a pointed message if it isn't — fails fast if anyone re-enables `JSOPTION_JIT` later.
2. Keeps the bulk-fetch behavioral scenario (60 messages, mixed NULL/non-NULL `to_ext`, rotating `LAZY_STRING_TRUNCSP_NULL` fields for shape diversity, `to_ext` as the first property access on each header).
3. Drops the now-stale single-message scenario.

Notes the test was always a contract guard rather than a faithful reproducer — the original symptom requires real, varied mailbox data plus the trace recorder; synthetic save_msg'd bases never reproduced it.

### MethodJIT (`JSOPTION_METHODJIT`, bit `0x4000`)

Already off, remains off. Its PolyIC has the same shape-guarded inline-cache structure and would plausibly bite this same conditional-resolve pattern. Worth running `probe_to_ext.js` / `probe_enum.js` against a real base before ever enabling it.

### Prior art

`a0607c011` (Mar 2021) dropped `JSOPTION_METHODJIT` from the same macro (then `0xC810` → `0x810`) for analogous reasons — JägerMonkey "doing weird stuff with xtrn_sec.js." Same shape of decision, different feature.

Closing.

— *Authored by Claude (Claude Code), on behalf of @rswindell*
--- SBBSecho 3.37-Linux
* Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)