50 descendants and 11 ancestors of c9c7dbe3d383341a
|
2026-07-20
| ||
| 03:33 |
fix: never parse a compressed body as cards (#104) (#111)
* fix: never parse a compressed body as cards (#104) A clone served by libfossil failed in libfossil's own client with xfer: decode card 16303: xfer: empty line after split on a response that contained 202 cards. The index, the card, and the fault were all fictional. xfer.Decode tried three body framings in order -- raw zlib, the §4.1 4-byte-length container, then uncompressed card text -- and treated any decompression failure as "this is not that framing". decompressBounded failed a well-formed container purely because its decompressed length passed a 50 MB bound, so the fallback chain handed the still-compressed bytes to the card parser. The parser read compressed noise as cards for sixteen thousand iterations before one of them split into no fields, and reported that as the fault. Captured at offset 15,084,752 of the failing body, the "card" the parser was reading is DEFLATE output: "...\xd9\x34\x50\xe7\xcf\xa0\xf8\xd5\xd5\x87l\xff\t\xde\xfb\xe3..." Two changes. Once a body is recognized as a zlib stream, a failure to decode it is reported rather than reinterpreted -- oversize, truncated, or corrupt, the error now names the transport failure instead of an arbitrary card position. Both compressed framings are still attempted, since a length prefix can coincidentally parse as a zlib header, and uncompressed bodies still reach the card parser; only compressed ones no longer can. The bound itself was also wrong, and is the reason this was reachable at all. §4.1 places no limit on the decompressed size of the container; the 50 MB was a local memory guard that sat below what this implementation's own server emits in one clone round, so two libfossil peers could not clone from each other even though each could clone from fossil. It is now 64 MiB and coupled to sync.DefaultCloneBatchBytes by a compile-time guard that fails the build, rather than the clone, if the budget is raised past half of it. The defect predates the size-bounded clone batching of #100. #100 shrank rounds enough to hide it on the corpus that exposed it; it did not touch the decoder. Regression coverage, in two layers. internal/xfer pins the decode behaviour directly: an oversize container, a truncated one, a large in-bounds one, and an uncompressed body. internal/sync adds the configuration no existing test covered -- libfossil serving and libfossil cloning across a real HTTP encode/decode, on a corpus of multi-megabyte incompressible artifacts that crosses the server's per-round budget. Every prior clone test was either served by fossil or wired through MockTransport, which passes *xfer.Message values across untouched and never encodes a wire body at all. Artifact size is the other half of the reproduction: this failed on wapp, whose artifacts are large, and not on pikchr, whose artifacts are small. Verified against the wapp corpus: the round trip that failed now completes with 1038 of 1038 artifacts on both ends. * fix: make the #104 fixture reach the bound, and pin the bound from both sides Review found the flagship regression test passed against pre-fix code. It did. Its corpus was 21.6 MB spread over twelve artifacts, and pagination keeps every round near the 16 MB budget however large such a corpus grows, so the round never came near the 50 MiB bound it was supposed to cross. The 25-line rationale reasoned about crossing DefaultCloneBatchBytes, which is not the quantity that matters and never was; a permanent comment asserting a mechanism that does not exist is worse than no comment. The quantity that matters is one artifact. A round is charged against the budget before each artifact and sends whole the artifact that crosses it, so a round carries budget plus one entire artifact of unbounded size. A single artifact larger than bound-minus-budget is the only corpus that can exceed the bound, and it is also the one path by which a round can still exceed the current bound -- so the fixture guards the live risk and reproduces the historical one with the same bytes. It is now one incompressible artifact of 7/8ths of MaxDecompressedBytes, sized against the constant so it tracks it, behind testing.Short. Verified against a pre-fix tree with both test files dropped in: TestCloneSelfRoundTripOversizeArtifact FAIL Clone: handleFile aad935c2...: UUID mismatch for received file: expected aad935c2..., got 501b4814... which is the sharper end of this defect: pre-fix did not merely misreport, it accepted the compressed bytes as cards and produced an artifact whose content did not hash to its name. The cheap end-to-end round trip is kept as a separate test and described honestly as a coverage plank -- the first test anywhere here to have libfossil decode a body libfossil produced -- rather than as a guard on anything. Second finding, message.go: the claim that format 1 is the "Fossil HTTP sync wire format" is false. Every application/x-fossil body carries the §4.1 length prefix, request side and reply side, both through blob_compress. Format 1 matches nothing canonical emits and is kept as a tolerance, now labelled as one. Its safety was also load-bearing on a constant with no stated relationship to it. For a container of decompressed length N, prefix byte 0 is N>>24, and a zlib CMF requires CM==8 in the low nibble, so no container under 128 MiB can present a valid header at offset 0. Raising MaxDecompressedBytes past that point would leave decompressContainer's prefer-what-decodes ordering as the only remaining defense -- a narrower margin than this code was written against, and #98 is exactly the change that might want a different bound. That is now a build failure rather than a trap, matching the guard already on the other side. Verified to fire at 0x08000000 and to permit 0x07FFFFFF. Also: the MaxDecompressedBytes doc claimed to stop a hostile body from exhausting an embedding process's memory. It does not -- the transport reads the body whole before Decode sees it. It bounds expansion only, and now says so. Test hygiene: the server handshake polls for a live listener instead of sleeping 200ms and dropping ServeHTTP's error, and the 32 MiB and 64 MiB xfer allocations skip under -short like their sync counterpart. * fix: correct three claims about what the #104 guards cover The fix is unchanged. Three statements about it were wrong, and a comment that misstates a mechanism is worse than no comment because the next reader trusts it. The ordering claim was the serious one. decompressContainer was described as "preferring whichever framing actually decodes, so a spurious header match still loses to a real container". It does not. Format 1 returns immediately on success and format 2 is reached only when format 1 fails; the error comparison at the end selects which failure to report when both have already failed, and protects nothing when format 1 spuriously succeeds. Probed: a bare zlib body returns from format 1 with format 2 never attempted. That makes the aliasing guard weaker than its comment claimed, and in a direction neither review had reached: the aliasing byte is the top byte of the length the PEER declared, and this decoder never reads that field -- it slices past it and counts bytes actually inflated. No value of MaxDecompressedBytes constrains what a peer declares. A container declaring 0x081D0000 while carrying 46 bytes presents a parsing zlib header at offset 0, measured. What actually saves a real container in that case is the stream, not arithmetic: after two prefix bytes are eaten as a header, inflate must still parse the remaining two plus the real stream as DEFLATE and match its Adler-32. Measured `flate: corrupt input` at all three aliasing lengths probed, so format 1 fails, format 2 runs, and the container decodes correctly. The guard is kept because it states something true and useful -- Encode writes uint32(len(raw)) as the prefix, so holding the bound below the threshold means no body libfossil emits can carry an aliasing prefix -- but it is now documented as an invariant on our output, not a defence of our input, with the airtight alternatives named and left to their own issues. Second, "a single artifact larger than this bound cannot be cloned" is true but not the operative threshold. A round already holding bytes has that much less room, so the ceiling is the bound minus what the round accumulated first. Measured on this branch with a 58,720,256-byte artifact: 8, 12 and 15 MB of filler ahead of it all fail, and 17 MB passes because it exhausts the budget and pushes the artifact into a round of its own. Non-monotonic in the filler, which is worth knowing. That is #109, a defect this fix exposes rather than causes; the fixture's claim to guard it is withdrawn and it now says plainly that it covers the leading-artifact case only. Third, the changelog's "pin the bound from both sides" overclaimed. The guards pin the bound against the batch budget and against the aliasing threshold. Neither certifies a round. Rebased onto 8b6da9a. The internal/content -race failure reported last round was not a flake and not unrelated -- this branch predated #107, which rewrote available.go. TestIsAvailable_CyclicChainTerminates goes from exceeding its 30s budget to 0.06s across that rebase. --------- Co-authored-by: Dan Mestas <[email protected]> check-in: c892c7a622 user: [email protected] tags: trunk | |
| 02:36 |
perf: memoize delta-chain interiors during crosslink (#107)
* perf: memoize delta-chain interiors during crosslink (#89) manifest.Crosslink ran at roughly 1.4 events/sec on the Fossil SCM repository, which puts a 27,000-artifact clone at two and a half hours of CPU after the transfer has already finished. A CPU profile of a sweep over that corpus attributes 152.77s of Crosslink's 167.20s -- 91% -- to content.Expand, and 126.35s of that to the single blob.Load inside its delta-replay loop. The cause is chain depth, not per-event overhead: not a transaction per event, not a missing index, not a full-table scan. content_deltify stores a file's older versions as deltas against its newer ones, so one file's history is a single chain -- 469 nodes deep on average across that repository, 1,032 for the blobs carrying artifacts, 2,546 at worst. Expand walked each chain from its root on every call and Crosslink calls it once per blob, so the sweep performed 31.7 million delta applications to expand 67,451 blobs. Cache.Expand now stops the chain walk at the deepest ancestor it already holds and keeps every node it materializes on the way forward, and Crosslink runs one such cache for the sweep. Blob content is immutable once written and the candidate rids are snapshotted before the loop, so nothing a sweep does can invalidate an entry. Every node handed to the cache is verified against the UUID its blob row is stored under before it is stored, because a cached interior is served later as if it had been expanded in its own right. The number of hash checks per sweep does not change: each blob is materialized once and verified once, where before each was expanded once and verified once. A plain Expand, with no cache, walks and verifies exactly as it did. IsAvailable folds its two-statements-per-chain-node walk into one. It runs once per F-card of every check-in being crosslinked, over the same deep chains, and every SQL round trip there pays a WAL read-lock acquisition -- the round trips, not the work they do, are the cost. No transaction boundary moves and no batching is introduced. A manifest that fails to crosslink still fails on its own, exactly where it did. Measured on the Fossil SCM repository, 67,451 content blobs and 27,060 artifacts. Time for Crosslink to write its first 300 event rows: 221s before, 131s after. Over the expansion path in isolation, sweeping the same corpus in rid order: 22.9 blobs/sec with no cache, 50.1 at the 256 MiB budget set here, 414 at 1 GiB -- where the process needs 2.9 GB resident, which is why 1 GiB is not the number. A bounded cache cannot close the gap on its own. Chains run from a high-rid root down to low-rid ancestors while the sweep visits rids ascending, so the first candidate of a chain materializes all of it and the remaining nodes are consumed arbitrarily far in the future; the working set of a whole-repository sweep in rid order is the whole repository expanded, 8 GiB for this corpus. Closing it means visiting candidates in delta-chain order, which is how Fossil's own rebuild avoids the cost, and which changes the order-sensitive state Crosslink writes -- leaf, tag propagation, tech-note event replacement, forum thread titles. That is a separate change. TestFossilBinaryReadsCrosslinkedRepo covers the acceptance criterion this cannot self-certify: a repository whose relational tables were written through the accelerated path passes fossil 2.28's test-integrity, shows its tip in fossil timeline, survives fossil rebuild, and its event, plink, leaf and mlink rows match, column for column, what fossil derives from the same blobs. * fix: bound and cycle-check the delta-chain walks (#89) IsAvailable walked the delta chain with no cycle detection, to a bound of 100,000 against real chains whose deepest observed node count is 2,546. It is wire-reachable -- sync/clone.go, sync/handler.go, and once per F-card of every check-in Crosslink processes -- so a peer that plants a two-node delta cycle bought a 100,000-iteration query loop on every call, inside whatever process embeds this library. Both halves are fixed. Each walk now carries a visited set and stops at the first repeat, which is the actual defence: a cycle costs its own length, not the bound. maxDeltaChainDepth replaces the old constant at 16,384, roughly six times the deepest chain measured on the Fossil SCM repository, and it is now a backstop for an acyclic-but-absurd chain rather than a number an attacker spends. walkDeltaChain, which the memoizing expansion path made hotter, previously had no bound at all and now shares the same one. This is what made TestIsAvailable_CyclicChainTerminates sit at ~99% of its 30s budget on main and fail outright under -race here: the cyclic walk was running the full bound. It now finishes in about a second under -race, six runs out of six. walkDeltaChain returns the content it stopped on instead of leaving expandChain to ask the store a second time. The truncation decision and the use of it were separate steps, so a concurrent eviction between them would send a delta row to blob.Load and hand raw delta bytes to the replay as full text. bindToName caught that and failed closed, but the caller saw a hash mismatch on an artifact that was not corrupt, and Crosslink's error path drops those silently. Unreachable while the cache is a single-goroutine local, but Cache is documented concurrency-safe and takes its mutex on every method, so the type promised something the expansion path did not deliver. Cache entries are clipped to their length before being stored, so curSize is the bytes actually retained rather than an allocator-dependent approximation of them. Test corrections: the corpus harness no longer closes the repository handle out from under the sweep goroutine when it returns at the event target; the sweep owns the handle and closes it. snapshotDerived now fails on an empty digest rather than comparing "" to "" and passing without examining a row. The fail-closed injector matches the merged chain-step statement specifically rather than any query mentioning delta, and says what it can and cannot prove now that the two lookups are one statement. The fossilbin comparison documents that it is not full-table equivalence, and which tables it leaves uncompared. MEASUREMENTS, SUPERSEDING THOSE IN 2426591 The figures in the parent commit were taken before clipping, and its "221s -> 131s" pair predates the double-hash and IsAvailable fixes that landed in that same commit. Everything below was re-measured on the code as it ships, one run at a time on an idle machine, against the Fossil SCM repository (67,451 content blobs, 27,060 artifacts). Crosslink to its first 300 event rows, cpuprofile on throughout: original code 221.4s as shipped 103.9s 2.1x Expanding the first 4,000 blobs, which isolates the expansion path: budget throughput peak RSS entries held none 22.1 blobs/s 169 MB 0 0 256 MiB 52.0 blobs/s 1036 MB 543 255 MiB 1 GiB 468.3 blobs/s 2278 MB 15,842 1021 MiB Clipping changed neither throughput nor memory. Crosslink to 300 events runs 103.9s clipped against 105.9s unclipped, peak RSS 1067 MB against 1050 MB, and the cache holds the identical 543 entries and 255 MiB at a 256 MiB budget either way -- at these sizes the buffers already arrive tightly sized. It is worth keeping as an accounting guarantee, not as a saving, and the comment on clip says so. The earlier claim that capacity slack was why a 256 MiB cache reached nearly a gigabyte resident was wrong. Peak RSS tracks the no-cache baseline plus roughly twice the budget, because the cache is live data and the collector sizes the heap against it: 169 + 2x1021 = 2211 against 2278 observed at 1 GiB. That is why the budget stays at 256 MiB -- a gigabyte of cache costs over two gigabytes resident to clone a 72 MB repository. The shape of the curve did not move. * docs: correct the claims around the chain bound and the RSS model (#89) Three claims did not survive checking, and one bound was a node out. maxDeltaChainDepth said it bounds "every delta-chain walk in this package". It does not: deltifyBreaksLoop is a third walk, under deltifyMaxChainWalk, which is 64 times larger. The rationale on the new constant applies to that walk just as well, but changing it would change deltify's shipped behaviour, so the sentence is narrowed to the two read-path walks it actually governs and points at the third rather than quietly covering it. Two comments still named maxAvailabilityChainDepth, which no longer exists, and argued from IsAvailable having a bound "of its own" -- the framing of a world where the two walks had separate bounds. Against a shared constant that argument reads backwards. Both now name the shared bound, and the reason deltifyBreaksLoop keeps its own visited set and its own step cap is stated as the general rule it is: every walk over the peer-shaped delta graph carries both, and none of them relies on another walk's guard. The retraction in the previous commit replaced one unsupported mechanism with another. "Peak RSS runs at roughly the baseline plus twice the budget" is a two-point fit whose worse point is the shipped one: it is 3% off at 1 GiB and 53% under at 256 MiB, where resident memory is four times the budget rather than twice. The measured table stays; the model is demoted to the direction the data supports, that peak RSS grows faster than the budget, with the fit's own weakness stated. walkDeltaChain counted loop iterations where IsAvailable counts nodes visited, so the shared bound bit a node apart: a chain of exactly maxDeltaChainDepth+1 was unavailable to one walk and complete to the other. It now counts nodes, and TestChainBoundBitesAtSameLength pins the agreement at both boundaries -- nothing else ties the two loops together. check-in: 9864120a32 user: [email protected] tags: trunk | |
|
2026-07-19
| ||
| 23:59 |
fix: bound clone batches by output size, not artifact count (#100)
* fix: bound clone batches by output size, not artifact count (#88, #92) The xfer server paginated a clone by a fixed 200-artifact count. A count cannot adapt to artifact size: 200 control artifacts and 200 large blobs produce wildly different messages, and neither corresponds to what the transport can carry. It capped a libfossil-served clone near 20,000 artifacts against our own MaxRounds=100 client, and left the response size unbounded in the other direction -- serving libfossil's own repository produced a single 44.9 MB round. emitCloneBatch now accumulates the wire bytes it emits and stops once they reach DefaultCloneBatchBytes, mirroring canonical's `while( mxSend > blob_size(pOut) )`: the budget is tested before each artifact and the artifact that crosses it is still sent whole, which is what keeps the cursor advancing when one artifact exceeds the entire budget. The card header is charged alongside the payload so a repository of zero-length artifacts still paginates. Section 8.2 leaves the bound to the server ("clients MUST NOT infer count or size from range"). The budget is 16 MB rather than canonical's 5,000,000 max-download because this clone path emits fully expanded artifacts where canonical retransmits stored compressed form (Section 7.2) -- 80x to 220x the bytes on real repositories. Measured on libfossil's own repository (17,218 artifacts): 87 rounds and a 44.9 MB worst-case response before, 65 rounds and 24.7 MB after. Canonical's constant applied to expanded bytes would have needed 184 rounds, breaking clones that work today. Adds the interop half: a canonical fossil 2.28 client cloning a generated corpus from `repo serve`, sized past the pagination boundary, checked for round-trip count, artifact parity with the source, and `fossil test-integrity`. The many-small-artifacts case is what discriminates the two bounds -- 2,056 artifacts clone in 3 round-trips under the size bound and 23 under the count bound. Corpus size is a parameter since the interesting behaviour is at the boundary. A missing fossil binary skips, or fails under REQUIRE_FOSSIL_BIN=1. * review: state the ceiling honestly, fix the overhead comment The size bound replaces an artifact-count ceiling with an expanded-bytes ceiling. It is a better ceiling and it beats the count bound on every corpus measured, but fossil- and sqlite-sized repositories still cannot be cloned from a libfossil server within MaxRounds. Record the full three-repository round table on DefaultCloneBatchBytes and say outright that only Section 7.2 removes the ceiling rather than moving it. State the memory property as budget plus the largest artifact, not as a 24.7 MB ceiling. The crossing artifact is sent whole, so a repository holding one 2 GB artifact still emits a 2 GB round, and concurrent clones multiply it. 45 MB -> 25 MB is a large reduction of a pre-existing unbounded allocation, not a cap. cloneCardOverheadBytes claimed to cover the hash and then len(uuid) was added on top, double-charging it. Drop the addend; 96 over-approximates the ~58-byte framing on its own and the slack absorbs the uncharged PrivateCard. Also: smallBatch sets a one-byte budget rather than a batch size of one, so correct the comments that still describe the old knob; drop the post-loop multi-batch > single-batch check, which passes under both bounds and so discriminates nothing; and make serveRepo report a server that exited early instead of signalling into an uninstalled handler, where SIGINT's default action would kill the test binary outright. * test: count clone batches on the server, not round-trips on the client CI (fossil 2.23) and this machine (2.28) disagreed on an identical server: the multi-batch clone reported 1 round-trip against CI and 9 locally. The clone was correct in both cases. fossil prints "Round-trips: N" carriage-return separated, once per round. 2.28 guards those prints behind bOutIsTty and adds a single final line for non-TTY output, so a captured clone yields exactly one occurrence holding the total. 2.23 has no such guard and emits every round into the capture, so the regex's first match was always 1 no matter how many rounds ran. Verified by cloning one repository from a real fossil server with both binaries: 2.28 prints "Round-trips:" once, 2.23 four times, first line 1 and last line 2. So the assertion silently meant different things on different clients, and many-small-artifacts -- the only case that discriminates a size bound from a count bound -- was passing under 2.23 for the wrong reason, inert rather than green. Count the clone batches the server emits instead. That is the property under test, it cannot be distorted by client-side reporting, and it is sharper: many-small-artifacts is 2 batches under the size bound against 22 under a reinstated count bound, where the round-trip numbers were 3 against 23. This serves via sync.ServeHTTP directly so the handler can be wrapped. RepoServeCmd.Run is a five-line wrapper around the same call and TestRepoServeCmdCanonicalFossilClone still drives the command end to end. It also drops these subtests' use of the process-wide SIGINT. check-in: 37271d3f68 user: [email protected] tags: trunk | |
| 23:33 |
Add libfossil version command (#99)
* Add libfossil version command Sources the version string from runtime/debug.ReadBuildInfo (module version for go-install/tagged builds, falling back to a short VCS commit), with a -ldflags override for release builds where BuildInfo alone would collapse to "(devel)". Output is a fixed, single-line, four-field shape ("libfossil <version> <go version> <os>/<arch>") so a caller can rely on strings.Fields(Version()) unconditionally -- the motivating consumer stamps this into every emitted benchmark record. No global --version flag: repo extract already owns --version as a command-scoped flag (source version to extract), and kong's global flags are visible in every subcommand's context, so a root-level --version would collide with it. version is a subcommand only. Verified kong's existing parse-error path already exits non-zero (exitUsageError, 80) for unrecognized commands and flags against the pinned kong version -- added regression tests locking that in rather than changing behavior that was already correct. * Reject malformed -ldflags version overrides at startup buildVersion was used verbatim with no validation, so a whitespace- or newline-containing -X override silently broke Version()'s promised single-line, four-field contract: a space produced six fields instead of four, and a newline broke the single-line guarantee outright. The stated consumer is a benchmark harness parsing this line as a stable record, so a corrupted version field silently corrupts every record from that build in exactly the field meant to identify it. A bad -X value is a build-time operator mistake, not a runtime condition, so it's rejected at package init rather than sanitized or worked around: sanitizing would silently mangle the operator's intent, and falling back to build info would report a version that is confidently wrong. Both would hide the mistake in the one field whose job is to be trustworthy. Failing the whole binary at startup surfaces a bad build the first time it runs, for any command, not just when someone happens to run `version`. Also simplified versionFromBuildInfo: verified empirically (go1.26, `go build .` inside this checkout, with and without -buildvcs and inside/outside a VCS checkout) that Main.Version is only ever the uninformative "(devel)" in exactly the cases where vcs.revision is also absent from BuildInfo.Settings -- whenever a VCS revision is available, Go has already folded it into Main.Version as a pseudo-version. The separate vcs.revision fallback loop was dead code that could only ever duplicate a hash already present in Main.Version; removed it along with the now-unused shortCommit helper. check-in: 725c694cb9 user: [email protected] tags: trunk | |
| 23:16 |
fix: remove StatusOpts, MergeOpts, CheckoutOpts — no call sites (#97)
* fix: remove StatusOpts, MergeOpts, CheckoutOpts — no call sites No function in the package accepts any of these three types, and nothing constructs one anywhere: Checkout.Status() takes zero arguments, Repo.Merge takes positional string arguments, and CheckoutOpts had no construction site at all. A public type with no call site documents a capability that does not exist -- a consumer reading MergeOpts{Strategy: ...} in the docs could reasonably conclude a strategy-selecting merge API exists; it never did. Repo.Merge and Checkout.Status keep their current signatures. An options-struct refactor for Merge was considered and declined. CheckoutOpts.Force was also a third same-named Force field in this package, alongside the already-removed UpdateOpts.Force (#58) and the real, fully-wired ExtractOpts.Force -- removing it also removes that readability hazard. ExtractOpts.Force and its plumbing (checkout.go Extract -> internal/checkout.ExtractOpts -> extract.go's Force check) are untouched. * docs: match em dash style in CHANGELOG breaking-change bullet The new StatusOpts/MergeOpts/CheckoutOpts entry used ASCII double hyphens where the adjacent CloneOpts and UpdateOpts.Force bullets in the same file use a real em dash. check-in: 1a33f22119 user: [email protected] tags: trunk | |
| 22:42 |
fix: clone_seqno is server-to-client only (#96)
* fix: clone_seqno is server-to-client only (#74) The clone client transmitted a `clone_seqno` card to carry its pagination cursor. That card travels the other way: canonical's page_xfer() only ever writes it (src/xfer.c:1571) and has no parser for it, so a real fossil server answered `bad command: clone_seqno N` and the clone died on round two. Repositories that fit in a single batch never reached the pagination path, which is why wapp and pikchr cloned while fossil and sqlite did not. The request cursor belongs on the clone card itself — `clone VERSION SEQNO`, read from token 2 at src/xfer.c:1553 — and the response cursor belongs on `clone_seqno`, which the client only consumes. Give each direction a single owner: - The client sends its cursor on the clone card and never emits a clone_seqno card. It already consumed the server's card correctly; it now also rejects a negative value, matching src/xfer.c:2769. - The server reads its cursor from the clone card and the CloneSeqNoCard parser is gone, so there is exactly one way to set the cursor rather than two. Aligning with canonical also corrects the cursor's meaning. Ours was the last rid sent, queried with `rid > cursor`; canonical's is the rid of the next blob to send, looped inclusively with `while( seqno<=max )`. Now that the value crosses to a real server it has to mean what canonical means, so emitCloneBatch queries `rid >= cursor` and reports the first rid it did not send, or 0 once the snapshot is exhausted. That collapses the old more/lastSentRID pair into a single nextRID and keeps the truncate hook honest by re-owing the dropped rid instead of skipping it. TestCloneMultiBatchAgainstCanonicalServer drives the client against a stand-in server that implements the canonical contract and rejects a client-sent clone_seqno the way fossil does; it reproduces `bad command: clone_seqno` against the old client. TestHandleCloneCursorFromCloneCard pins the server half, including the inclusive cursor. Verified against fossil 2.28 over a loopback server: fossil and sqlite now transfer every artifact (67,499 and 130,994 blobs), and wapp and pikchr still clone in 3 rounds. * fix: conform clone cursor validation to the spec in both directions (#74) Review caught two rules applied to the wrong side of the wire — the same axis error the original bug was. The client rejected a negative clone_seqno as fatal, citing src/xfer.c:2769. That line is the server's rule for the SEQNO on a clone REQUEST. The reply parser is governed by §3.2 and §8.2: NEXT is recorded only when it is decimal, and a visible but non-decimal NEXT is a defined receiver-tolerance exception that leaves the recorded sequence unchanged and does not stop parsing later reply cards. Aborting the clone was the one thing the spec rules out. The rule belongs in the decoder, where a non-decimal NEXT now decodes to an unknown card: the token is preserved, no consumer records it, and following cards still parse. strconv.Atoi accepted "-1" off the wire, so this was reachable, not theoretical. Conversely the server clamped `clone 3 0` to a cursor of 1 where §8.1 mandates a fatal: clear accumulated output, roll back, emit `error invalid clone sequence number`, and parse no later request card. Nothing is written before processDataCards, so returning early is what the rollback amounts to. Seeing this case at all needs a presence flag, because `clone` and `clone 3 0` both parsed to SeqNo 0 and only the former is exempt. §8.1 also keys on a digit-only SEQNO and withholds the fatal from wrong arity, a VERSION below 2, and a SEQNO that fails digit-only recognition, so the flag records digit-only presence rather than merely arity. Both card rules now turn on the spec's `decimal = 1*DIGIT` production via a shared predicate instead of on whatever strconv.Atoi tolerates. Also promotes emitCloneBatch's `count > 1` truncate guard to a documented liveness constraint. Under the old exclusive `rid > cursor` semantics `count > 0` was safe; under the inclusive cursor, dropping the only card in a batch reports a cursor equal to the one we entered with and the client re-requests forever. A check before the reply card now names a stalled round at its source instead of letting it surface as an opaque MaxRounds failure, and the cursor precondition became an error return rather than a panic — it is enforced ~350 lines away in a network-facing handler. Records the deliberate asymmetry that our server ignores an inbound clone_seqno where canonical answers `bad command`, and renames TestHandleCloneSeqNo, which no longer sends that card. * fix: name the clone SEQNO flag for the shape it records (#74) HasSeqNo described presence, but the field records whether the SEQNO token is digit-only: `clone 3 -1` carries a SEQNO and sets it false. §8.1 keys its fatal on that shape rather than on the parsed value, which is why a plain int cannot stand in for it. SeqNoIsDecimal says so. Reading it as presence would also collapse `clone 0 0` — a distinct wire form that parses to the same Version and SeqNo as a bare `clone` — back into `clone` on re-encode. encodeClone already guarded against that; a round-trip test over every clone wire form now pins it, and fails on exactly that input when the guard is removed. Also records why the §8.1 rollback is currently vacuous: no card handled before processDataCards writes anything, so returning early leaves nothing to undo. Nothing enforces that ordering, and §3.5's ci-lock would break it, so the dependency is written down next to the code that relies on it. Corrects a test comment that claimed a continuation property it did not exercise — the malformed card sits in the trailer, so nothing follows it. That half of §8.2 is covered by the follow-on card in TestDecode_CloneSeqNoNonDecimal. check-in: f2fa2cc751 user: [email protected] tags: trunk | |
| 22:27 |
Enforce card occurrence classes and intra-run ordering (#94)
* Enforce card occurrence classes and intra-run ordering The deck parser enforced card-type ordering and nothing else. Each of the 14 single-occurrence card types was handled by a case that assigned, so a second occurrence silently overwrote the first, and the repeatable types were appended with no sort check at all. draft-fossil-artifact-format-00 §4.5.1 and §4.5.2 define both rules; implement them. Duplicate rejection covers A, B, C, D, E, G, H, I, K, L, N, R, U and W. The guard is separate from the card-type check because that check permits equal consecutive letters. F, J and M gain a strictly-ascending check against the preceding entry, T the two-level (name, target hash) rule with the full sign-prefixed name as primary key. P and Q deliberately get neither: P order is semantic — primary parent first — and Q is unordered-repeatable. P also now appends across cards instead of replacing, which is what §4.5.1 means by repeatable. Z belongs to neither class, so it is accepted with no duplicate or last-card rule; the authoritative checksum is the trailing 35 bytes VerifyZ already verified. All four ordering checks route through one comparator, deck.Compare, which is §4.5.3: unsigned octets, locale-independent, no normalization, no case folding. §4.5.3 exists because the same ordering underlies the delta-manifest merge walk and baseline binary search of §8.2, so a second copy would resolve a delta manifest wrongly rather than merely fail validation. The F check runs on the decoded filename, per §4.5.2. Malformed input is reported through the error return, never a panic -- these bytes arrive from peers. Two writer-side defects surfaced, both of which let this package emit artifacts its own parser now rejects. Marshal emitted J cards in caller-supplied order while already sorting F, M and T; it now sorts them. Marshal also sorted T by the concatenation name+target, which lets the target's leading hex digit compete with the tail of a longer name and reorders tags whose names are prefixes of one another; it now uses the two-level key. The F and M sorts and the R-card file enumeration route through deck.Compare as well, the latter per §6.3. Closes #70 * Order T cards by decoded name and type the Z payload Review found two ways the first pass diverged from canonical fossil in the accept/reject direction. The T ordering key used the raw wire name. §4.7.16 says the name is escape-decoded and that the primary ordering key is the full name token including its leading sign character; manifest.c defossilizes the name before the ordering compare. Because "\s" begins 0x5C and a space is 0x20, the two forms order oppositely, so a run like T *sym-a\sb aaaa... T *sym-a-c bbbb... is valid, fossil accepts it, and we rejected it. Branch names may contain spaces, so this is an ordinary tag, not a corner case -- the same post-decode trap already handled for F. tagOrderKey now builds the decoded key and both the parser check and the marshaller's sort call it, so the two cannot drift apart. The Z card accepted any payload. Accepting a Z line inside the body is right -- §4.5.1 puts Z in neither occurrence class and treats an earlier Z as content covered by the final checksum -- but §4.7.19 still fixes the payload at exactly 32 hexadecimal characters. Validating nothing left an arbitrary-byte channel in an otherwise fully typed grammar and would have stored and forwarded artifacts canonical refuses. Upper-case digits are accepted, matching fossil's validate16. Also: the T sign prefix is now checked against {+,-,*} per §4.7.16, both because canonical checks it and because an unvalidated sign byte >= 0x80 would UTF-8-expand to two bytes when the ordering key is assembled and silently corrupt the comparison. Two stale §4.7.20 citations corrected to §4.7.16, and the two-level T test gained the prefix-name case that actually distinguishes a two-level comparison from a concatenated one. * Cite the normative grammar for the case-insensitive Z token The Z payload's case-insensitivity was justified by inferring it from the word "hexadecimal" in §4.7.19. §6.1 states it normatively: md5-token = 32hexdig-ci, a parse-accept grammar admitting both 0-9a-f and A-F. Cite that instead, and record the split it implies -- the structural check is case-insensitive while §6.2 keeps verification byte-exact against the lowercase form. Also soften deck.Compare's claim that the delta-manifest merge walk relies on F ordering to the future tense parse.go already uses: the merge walk is a map overlay today with no comparator to share. check-in: 5036cae392 user: [email protected] tags: trunk | |
| 20:55 |
Deltify predecessors on the commit path (#87)
* Deltify predecessors on the commit path Every locally-created artifact was stored whole. The delta table was in the schema and stayed empty, so a repository grew with the number of commits rather than the amount of change. blob.StoreDelta existed and worked but had no caller on the commit path; the only delta-store in use was blob.StoreDeltaRaw, from the sync receive path, for deltas that arrive pre-encoded. The write path never made a deltify decision at all. Port canonical Fossil's content_deltify (2.28 src/content.c:852-947) as content.Deltify. Fossil deltifies backwards: a new artifact is always stored whole and its PREDECESSOR is rewritten as a delta against it, so the tip of every chain is full content and reading the newest version of a file stays a single blob load. Chain depth becomes an artifact's age in revisions rather than something that compounds, because an artifact already stored as a delta is never re-deltified. blob.StoreDelta turned out to be the wrong primitive to wire up: it inserts a new row as a delta, whereas the policy needs to rewrite an existing row. It is left alone. content.Deltify lives in the content package because it needs Expand, which blob cannot import. The policy is stated once, in content.Deltify, with each rule's canonical line cited: the 50-byte floors, the 75% ratio, the never-redeltify rule that bounds depth, the private-into-public refusal, and the loop break that undeltas the source and then still declines the pairing. Two call sites pass pairs, mirroring canonical's: insertMlinkRow for file content (manifest.c:1562) and Checkin for the parent manifest (checkin.c:3234). insertMlinkRow is shared by the fresh-commit and crosslink paths, so file deltification cannot drift between them. On a 12-revision, 4-file history: 60 artifacts, 44 delta-encoded (73%), p50 chain depth 4, max 11, and 23,994 content bytes measured as sum(length(content)) -- down from 58,973 with everything whole. A repository produced this way passes fossil 2.28 test-integrity (60/60 blobs, 0 errors) and fossil rebuild, which re-derives every table by walking the chains. * Bound the deltify ancestor walk, require a transaction, harden tests Review follow-ups on the deltification policy. Bound the ancestor walk. It was a bare loop with no cycle guard: a sync peer can shape a cyclic delta graph, because blob.StoreDeltaRaw records delta(rid,srcid) links from the wire without a cycle check, and an unbounded walk would spin inside the write transaction. Guard with the seen-set idiom content.walkDeltaChain already uses, plus a step cap. In the current ordering IsAvailable runs first on both rids and follows the same links under its own depth bound, so Deltify declines before the walk begins -- but that protection is incidental to a different question, so the bound belongs to the loop that needs it. Both behaviours are now pinned: the walk reports a cycle, and Deltify declines an ungrounded source. Move both size floors ahead of the walk. Canonical returns on a sub-50-byte target (content.c:877-885) before entering the candidate loop, so it never reaches the content_undelta on :902. Running the walk first meant a small target whose successor chained back to it would inflate the source to full content and then decline anyway -- a storage regression and exactly the divergence from canonical's layout the surrounding comment claims to avoid. Take *db.Tx instead of db.Querier. The rewrite is two statements, and a blob holding delta bytes with no delta row expands as though it were full content: silent corruption rather than a detectable error. Canonical wraps the pair in db_begin_transaction (content.c:938-941); the signature now makes that non-negotiable at compile time. Both call sites already passed a transaction. Three tests were named for behaviours they did not pin, and each is now verified to fail when the behaviour is reverted: - The 75% ratio test had no assertion at all; deleting the rule left it passing. It now asserts the pair is declined and stays whole. - The loop-break test could not distinguish declining from proceeding, so the undelta-then-proceed behaviour corrected in the previous commit was untested. It now asserts the target is left whole. - The manifest test checked only "at least one", so a regression from 11-of-12 to 3-of-12 passed. It now checks each index and names the manifest that stopped being encoded. Also cover the source-side 50-byte floor, which the tiny-artifact test could not reach because it made both artifacts tiny and returned at the target check first, and assert that Undelta restores blob.size while Deltify leaves it alone (content.c:761 vs :935). Gate the fossil-binary test on REQUIRE_FOSSIL_BIN, set in CI. A skip is invisible without -v, so a run with no fossil binary was byte-identical to a pass -- for the one criterion our own tests cannot substitute for. Document that blob.StoreDelta is not the commit path's primitive. It inserts a new row as a delta; deltifying rewrites an existing one. It has no non-test callers and the package doc advertised it as the delta-storing entry point. * Document what the deltify loop-safety argument rests on Two comments, no behaviour change. The IsAvailable(srcRid) guard is ours, not canonical's: content.c guards only content_is_available(rid) and then calls content_get(srcid) unguarded. It is also load-bearing beyond phantoms, because it screens out the cyclic delta graphs that would otherwise reach the ancestor walk. Say both, at the guard. And say at the walk why its seen-set never fires in practice, so that someone reordering the prologue meets the reason rather than concluding the bound is dead code. The note lived only in the test file, which is not where that reader is looking. check-in: 2368cad91a user: [email protected] tags: trunk | |
| 18:31 |
cli: add repo serve command wiring the xfer HTTP server (#72)
* cli: add repo serve command wiring the xfer HTTP server Repo.ServeHTTP has always been public, but nothing in the CLI called it, so a repo created or cloned with this tool could only ever be a clone leaf. Add `libfossil repo serve` as a thin adapter: open the repo, build the bind address, translate process interrupt into the context cancellation ServeHTTP already honors, and call it. No new server or shutdown logic. Binds to 127.0.0.1:8080 by default; --addr overrides host and/or port. Verified against a stock fossil client (canonical binary, not just this library's own transport) cloning over an actual TCP round trip. * cli: address review nits on serve interop test Document why TestRepoServeCmdRun and TestRepoServeCmdCanonicalFossilClone are allowed to send a real SIGINT to the test process (no t.Parallel(), no competing signal handlers in package cli_test), so a future test in this package doesn't silently break that assumption. Also log the canonical fossil binary path and version the interop test actually exercised. A skip and a pass read identically in a non-verbose CI log; if the apt install step that puts fossil on PATH is ever removed or reordered, this line is what would tell us the test stopped proving anything instead of just staying green. check-in: 7572f7d420 user: [email protected] tags: trunk | |
| 18:31 |
fix: distinguish absent, empty, and valued U-cards in manifest parsing (#73)
* fix: distinguish absent, empty, and valued U-cards in manifest parsing Manifest parsing collapsed a wholly absent U-card and a present-but-empty U-card into the same stored value (empty string), so every crosslink call site that wrote the check-in user into event/attachment/forumpost rows lost the distinction between "no login recorded" (should be SQL NULL) and "login explicitly blanked" (canonical fossil resolves this to the literal "anonymous" at parse time — see src/manifest.c:1008-1016). Change deck.Deck.U from string to *string so the three states are representable: nil (absent), non-nil "anonymous" (present-empty, resolved in parseUCard the same way fossil's own parser does), and non-nil with a value (present). Resolving once in the parser means every downstream crosslink/rebuild call site can keep binding d.U directly — Go's database/sql converts a nil *string argument to SQL NULL and dereferences a non-nil one automatically, so no per-call-site special casing is needed. The type change itself makes the old ambiguous string-binding path a compile error everywhere Deck.U is constructed, closing off the drift that caused issue #29. Added deck.Str, a small pointer-literal helper, since Go can't take the address of a string literal directly; existing tests that built Deck{U: "..."} now go through it. * fix: review nits — whitespace U-cards, rebuild coverage, tag.AddTag, naming Four follow-ups from review of the U-card presence fix: - deck.parseUCard now trims args before the emptiness check, matching canonical fossil's next_token() (skips leading whitespace) and every sibling card parser in this file. Previously "U \n" stored literal whitespace instead of resolving to "anonymous". Added a whitespace-only case to TestParseUCardStates. - Added TestRebuild_UCardPresenceStates covering the rebuild path (rebuild_manifest.go's rebuildCheckin), mirroring TestCrosslink_UCardPresenceStates. The binding site was already correct (same nil-to-NULL pointer semantics as crosslink.go) but had no dedicated regression test. - tag.AddTag now panics on an empty opts.User, matching the panic idiom CheckinOpts.User already establishes. Previously a tag created with an empty User silently crosslinked to user=''; that asymmetry with CheckinOpts was a wart worth closing rather than documenting as intentional. - Renamed deck.Str to deck.User: the name should say what the helper constructs (a Deck.U value), not how it's implemented (a pointer take). check-in: 527759acd5 user: [email protected] tags: trunk | |
| 18:28 |
fix: store delta content against a phantomized base instead of discarding it (#83)
* fix: store delta content against a phantomized base instead of discarding it A delta artifact arriving before its base is normal steady-state behavior during a Fossil transfer, not a fault. The receive path used to discard such deltas outright and phantomize the target instead, so a phantom only ever cleared when its base happened to already be present -- resolution advanced one delta-chain hop per round, making clones of deep-history repos provably unable to converge within MaxRounds. Match canonical Fossil's content_put_ex/rid_from_uuid model: resolve the delta's source via a create-phantom-if-missing lookup, persist the delta bytes unconditionally (reading the target size straight out of the delta header, so the base never needs to be readable to store it), and record the delta-to-source link immediately. Availability resolves lazily via the existing content.IsAvailable chain walk and manifest.Crosslink's deferred retry -- both already correct and untouched here. storeReceivedFile is the single call site shared by clone, sync-pull, and server-push, so fixing it here also closes out the delta-before-base sentinel handling on the pull and push paths (issue #57, folded into this fix): neither path needs to special-case the condition anymore because it stops being an error. Verified against the pikchr corpus via a loopback fossil server: rounds dropped from 100 (exhausted) to 3, files received from 63,155 (22.2x) to 2,980 (1.05x), and the phantom table empties out completely. * fix: converge delta storage on canonical, close integrity gap, harden wire parsing Three defects from review, all in the raw-delta path added for the delta-before-base fix, all reachable from an untrusted peer: 1. delta.getInt accumulated v = v*64 + digit with no overflow check. A twelve-byte header of maximum-value digits wraps to exactly math.MaxUint64, which casts to int64(-1) -- the same sentinel blob.size uses for "phantom". That silently corrupted the row: real content reporting as a phantom, with no phantom-table entry to make it re-requestable. getInt now rejects overflow instead of wrapping, and parseTargetLen (shared by Apply and OutputSize) bounds the parsed target length to a sane maximum before it ever reaches an allocation or a size column. 2. StoreDeltaRaw panicked on a zero-length delta payload. storeReceivedFile guarded payload == nil but not len(payload) == 0, and nothing between wire decode and storage checked for it -- a peer sending an empty delta file card crashed the process. Wire data gets an error return, not an assertion; OutputSize and StoreDeltaRaw are now consistent with delta.Apply's existing "empty delta is an error" contract. 3. The receive-time split (store raw only when the base happens to be unavailable, otherwise eagerly expand and verify) left a gap an attacker chooses the timing of: withhold the base, and a delta claiming any UUID goes down the unverified raw path. Nothing downstream checked it either -- content.Expand applied deltas with no hash check, and content.Verify was reachable only from repo rebuild and one integration test. Closed by converging fully on canonical's model: every delta is now stored raw and linked via a create-phantom-if-missing lookup, regardless of whether its base is currently available -- receive time no longer branches on availability at all. Verification moves to the one choke point every expansion path already goes through: content.Expand now hashes its result against the row's claimed UUID before returning it, catching a mismatch regardless of which caller or how the chain assembled. content.Verify becomes a thin wrapper over the same check. Storage-layer and end-to-end regression tests added for all three, including the exact claim-one-UUID/ship-a-delta-for-another/withhold-the- base reproduction, guarded against degenerating into a no-op (delta must actually compress, base must be genuinely absent, rejection must be a real error rather than merely different bytes). Re-verified against pikchr (2,842 artifacts) and wapp (1,039): pikchr converges in 3 rounds, 2,980 files, 0 phantoms; wapp converges in 3 rounds, 1,115 files, 1 phantom outstanding (pre-existing, unrelated -- clone.go has no DB-phantom rescan the way the pull path's loadDBPhantoms does; separately tracked). On-disk repository size is close to but slightly smaller than storing everything as raw delta bytes should produce (pikchr 6.81MB -> 6.73MB, wapp 5.64MB -> 5.46MB): zlib on full text already captures most of the redundancy an explicit delta would, so the win here is mainly integrity and convergence, not size. * fix: bound delta allocation to actual work, validate delta source hex Two defects from round-two review, plus a stray comment. 1. delta.Apply preallocated its output buffer directly from the delta header's claimed target length (make([]byte, 0, targetLen)). That claim only had to clear maxDeltaTargetSize (4 GiB) to be trusted as an allocation size, so a handful of wire bytes declaring a near-4 GiB target triggered an allocation sized to the claim, not to any work actually done. Harmless as a one-off parse spike; durable once a delta can be stored unverified (see blob.StoreDeltaRaw, this branch) -- a hostile peer plants one such delta and every later Expand of that row repeats the attempt. The initial capacity is now a small fixed bound; append grows it normally for legitimately large content instead of trusting the header's own claim about itself. 2. storeReceivedFile validated the target UUID's hex-ness (hash.IsValidHash) but not the delta source's. The blob table's CHECK constraint only bounds uuid length, not hex-ness, so a non-hex string of valid length passed straight through to blob.StorePhantom and became a permanent, unfillable row -- one the pull path's loadDBPhantoms would then re-request every sync round, forever. deltaSrc now gets the same check as the target uuid, at the same place. Also deleted a stray doc-comment line left over from an earlier refactor. Regression tests for both, RED-verified against reverted code before being fixed: the allocation test had to be corrected mid-way through -- delta headers encode their integer in Fossil's own base-64 alphabet, not decimal, so an initial fmt.Sprintf("%d", ...) fixture parsed as a different (still large but differently-bounded) value and passed vacuously; appendInt is the real encoder and is used directly now, with a round-trip sanity check against OutputSize guarding the fixture itself. check-in: 89535a6c42 user: [email protected] tags: trunk | |
| 18:27 |
checkout: re-stat executable bit at commit time (#80)
* checkout: re-stat executable bit at commit time, not just on changed files Change scanning compares content hashes, so a file whose only edit is a chmod (owner-execute bit gained or lost, no content change) is never flagged as changed. vfile.isexe then carries forward whatever was captured at the last Add/Commit, and the resulting F-card silently keeps the stale permission. Mirror canonical Fossil's belt-and-braces approach (src/checkin.c): manifest generation re-stats each committed file's live on-disk mode independently of whatever change scanning recorded, using the existing modeIsExecutable predicate (added for the prior executable-bit fix) rather than trusting a cached flag a second time. This runs for every file with a vfile row that is part of the commit, not only ones flagged as content-changed, so a chmod-only edit is now captured. Applies only to files actually present in vfile, leaves symlink markers alone, and is a no-op on Windows where os.FileMode carries no owner-execute bit. * checkout: skip missing files during commit-time perm re-stat restatCommitPerms Stats every tracked file that is part of the commit, and a Stat failure on any of them aborted the whole commit. That regressed a case the mode-change fix never meant to touch: a tracked file deleted from disk without Unmanage, with an unrelated file modified, used to commit successfully and carry the missing file's last-committed content forward — after the fix it failed the commit outright, on the implicit "commit everything" path in particular. Skip only os.IsNotExist and carry the existing Perm forward unchanged, matching pre-fix behavior exactly. This fix is about mode changes being missed, not about detecting stale tracking — that is a legitimate feature canonical Fossil also checks for at commit time, but it deserves its own issue rather than arriving as a side effect here. Any other Stat error (permission denied, I/O fault, ...) still fails the commit. check-in: 094abd4b4e user: [email protected] tags: trunk | |
| 18:27 |
fix: give Timeline its own result type so an Ancestry cursor cannot compile into it (#78)
* fix: give Timeline its own result type so an Ancestry cursor cannot compile into it LogEntry (Ancestry's result type) carried a Cursor field that was always the zero value on Ancestry entries — structurally identical to a legitimate Timeline first-page call. Feeding it into TimelineOpts.After silently paginated from page one forever with no error; documenting the field as invalid on Ancestry entries did nothing to stop it, because the zero cursor cannot be told apart from a valid first-call state even in principle. Split the shared type instead of adding a runtime guard: LogEntry loses Cursor entirely and remains Ancestry's result type, while a new TimelineEntry (embeds LogEntry, adds Cursor) becomes Timeline's result type. An Ancestry entry's cursor can no longer be obtained at all, so passing one to Timeline is now a compile error rather than a documented hazard. Applied the same split to the internal manifest.LogEntry / manifest.TimelineEntry pair Timeline and Ancestry are built on. Breaking change; CHANGELOG documents the migration. * doc: warn against hand-assembling TimelineEntry from an Ancestry LogEntry Review found the one remaining escape hatch: TimelineEntry{LogEntry: someAncestryEntry} compiles with a zero-value Cursor, which Timeline reads as "start from newest" rather than an error. No library path produces that value — it requires deliberately embedding an Ancestry LogEntry into a hand-built TimelineEntry — but flag it at the type so whoever does that gets the warning where they're typing. check-in: ad09ba3d97 user: [email protected] tags: trunk | |
| 18:17 |
Remove unused CloneOpts.ProjectCode and ServerCode fields (#77)
Both were accepted by the public CloneOpts but never forwarded to the internal clone path, so setting either silently did nothing. The internal clone options struct has no matching fields at all, so wiring them would mean designing semantics first — is a caller- supplied code a validation assertion or an identity override? — and no caller has asked for either. Delete them rather than guess. CloneResult.ProjectCode and CloneResult.ServerCode are unrelated and unaffected: they are populated from the internal clone's return value (the remote, via protocol negotiation), not from caller options, and continue to work exactly as before. Consistent with the same call made on UpdateOpts.Force. check-in: 9776c25462 user: [email protected] tags: trunk | |
| 17:40 |
Remove unused UpdateOpts.Force field (#69)
Force was accepted by the public UpdateOpts but never forwarded to the internal update path, so setting it silently did nothing. Delete it rather than guess at forcing semantics; a real implementation can be designed and added later as a deliberate, wired field if a caller needs it. Rides along with the UpdateResult signature change already landing on Update in this cycle. check-in: cd674f058b user: [email protected] tags: trunk | |
| 16:50 |
fix: populate mlink parent columns on the Crosslink check-in path (#63)
* fix: populate mlink parent columns on the Crosslink check-in path insertCheckinMlinks (the xfer/Crosslink ingestion path) wrote mlink rows with only (mid, fid, fnid) set, leaving pmid/pid NULL. Fossil's compute_fileage filters mlink.fid != mlink.pid, and in SQLite <int> != NULL evaluates to NULL rather than true, so every row written by this path was silently dropped from ancestry-walk queries - `fossil ls -R <repo> -r <rev>` returned empty for any commit that arrived over the wire, even though `fossil open` worked fine (it reads the manifest blob directly and never consults mlink). Root cause: the two check-in paths that write mlink both needed to implement the same three-case pid convention (src/manifest.c:1668-1679 in canonical Fossil - carried-over/added/added-by-merge) and had drifted. insertMlinks (the direct check-in path) only handled two of the three cases and ignored merge parents entirely; insertCheckinMlinks didn't populate the parent columns at all. Factor the convention into one place (internal/manifest/mlink.go): resolveMlinkParent implements the pid/pmid rule, insertMlinkRow wraps it and also fills pfnid (rename tracking) and mperm (permission encoding), which neither path wrote before. Both insertMlinks and insertCheckinMlinks now call insertMlinkRow, so the two write paths produce indistinguishable rows and cannot drift apart again. insertCheckinMlinks also stopped skipping F-cards with an empty UUID (the delta-manifest deletion marker) - it now writes a row with fid=0, matching canonical's "deleted files have mlink.fid=0" and letting deletions participate in ancestry queries the same way Fossil's own writer does. Un-gates ls_branch_tip_repro_test.go (previously untracked, guarded behind LIBFOSSIL_LS_BRANCH_TIP_REPRO) as a permanent regression test, and adds explicit coverage in internal/manifest/mlink_test.go for all three pid cases (normal add, merge add, delete) on both write paths. * test: pin the deleted-file pid to the parent's exact file rid Strengthens the deleted_file_gets_fid_zero subtest added for #29: it was only checking pid != 0, which would pass even if resolveMlinkParent picked the wrong nonzero value. Compare against the primary parent's actual mlink.fid for the same filename instead. * fix: review findings on #29 — substring perm match, test isolation Blocking: permToMperm switched on the whole perm string, matching only exact "x"/"l". Canonical manifest_file_mperm (src/manifest.c:1482-1492) does a substring test (strstr), and F-card perm fields can carry more than one character — internal/deck/parse.go:194 assigns the field verbatim from remote xfer input, and Fossil itself emits multi-char perm strings (the " w" rename placeholder, see #51). An exact match on e.g. "wx" would silently drop the executable bit, the exact invariant PR #48 landed to protect. Switch to strings.Contains, x checked before l to match canonical's order (the same idiom already used at internal/checkout/vfile.go:52). Added TestPermToMperm_SubstringMatch, including a "wx" case pinned to mperm=1. Test isolation: rowFor in both mlink_test.go table-driven tests closed over the parent *testing.T, so a Fatalf in one subtest's row lookup aborted every subtest after it via runtime.Goexit unwinding the shared parent goroutine. Pass each subtest's own t into rowFor instead. Proven on the reviewer's revert run: after the deletion subtest's Fatalf, the merge-add and normal-add subtests never even started. Tightened the carried-over-file assertion in TestInsertMlinks_MergeParentsGetPidNegativeOne from "pid != 0" (would pass for -1 too) to pin the exact parent fid, matching how the crosslink-path twin test already asserts it. Documented (comment only, no behavior change) a known divergence from canonical in resolveMlinkParent: a file present in a merge parent with different content (a conflict resolved during the merge) should stay pid=0 in canonical Fossil, but this function returns pid=-1 for any file present in a merge parent regardless of content match. Low blast radius — every mlink consumer in this package filters pid != fid, which holds either way — but worth flagging so it isn't rediscovered as a surprise. check-in: 24c35dad87 user: [email protected] tags: trunk | |
| 16:31 |
checkout: Update returns UpdateResult so conflicts are visible (#61)
* checkout: Update returns UpdateResult so conflicts are visible The internal 3-way merge already tracked which files were written, removed, and left with conflict markers, but Checkout.Update discarded all of it and returned a bare error. That collapsed a clean update and one that silently wrote <<<<<<< markers into working-tree files into the same outcome, with no way for a caller to tell them apart. Update now returns (UpdateResult, error). UpdateResult.Conflicted lists paths that were merged but not cleanly -- this is still a successful update (err == nil), not an error, since the working tree is usable and the caller just needs to know which files to show a user. A genuine failure still returns a non-nil error paired with a zero-value UpdateResult. FilesWritten/FilesRemoved are populated the same way, as paths rather than counts. Threading the paths up required changing the internal update's processFileUpdates from returning bare int counts to returning the actual []string paths, since the count was all that existed before. Checkout.Extract is untouched -- it has no conflict concept. Breaking change, clean break at v0.7.0 (pre-1.0). * checkout: sort Update result paths, document FilesWritten/Conflicted overlap Update's result slices were built by ranging over maps.allNames, a map[string]bool, so their order was whatever Go's randomized map iteration happened to produce that run. A caller diffing result.FilesWritten against a golden file, or asserting on it with reflect.DeepEqual, would see it fail intermittently for no reason related to their code. Sort all three slices (sort.Strings) right before Update returns. Added a test that touches four files -- two conflicting, two clean additions, named so map iteration order is very unlikely to hand them back sorted by luck -- and asserts the exact sorted slice on both FilesWritten and Conflicted. Verified it fails reliably (5/5 runs) with the sort calls temporarily removed, then restored the fix. Also stated outright, in both the internal and public UpdateResult doc comments, what "clean or not" was only implying: every path in Conflicted also appears in FilesWritten, since the conflict markers are what got written there. check-in: 75f3521e54 user: [email protected] tags: trunk | |
| 16:30 |
fix: make Timeline pagination cursor an exact total order (#62)
* fix: make Timeline pagination cursor an exact total order #54 (merged as 95cbbb7) split Repo.Timeline into a repository-wide enumeration and Repo.Ancestry, the original first-parent walk under an honest name. Its pagination cursor predicate used a tolerance window around the caller-supplied Before/After pair to work around a lossy julian<->time.Time round trip: mtime < (C - eps) OR (mtime <= (C + eps) AND rid < After) This reintroduces exactly the defect the composite cursor existed to prevent, and makes it worse: rows within eps of the boundary that are not real ties get filtered by rid instead of tie-broken by it, which can skip a row, duplicate another, or fail to advance the cursor at all — unlike canonical fossil's own bare-timestamp slop, which repeats rows but still advances. A pagination test using only exact-tie and one-hour-apart fixtures could not exercise the 0 < delta < eps range where the defect lives, so it stayed green. Fix: make the predicate an exact total order, with rid as a true tie-break only at exact mtime equality: mtime < C OR (mtime = C AND rid < After) and stop deriving C from a time.Time altogether. TimelineOpts.After is now an opaque Cursor (internal/fsltype.Cursor, re-exported as libfossil.Cursor): obtained only from a LogEntry the library already produced, carrying the exact float64 mtime read off that row rather than one reconstructed from a rounded timestamp. A cursor built from parts can silently drift from the row it claims to follow; a cursor that can only ever be handed back cannot. That exactness depends on the row's scanned mtime being faithful to begin with, which surfaced a second bug: under the ncruces/WASM sqlite driver, event.mtime scans as a time.Time via the driver's own internal REAL conversion, carrying ~13us of noise that db.ScanJulianDay's time.Time branch then truncated (via UnixMilli()) into a full missed millisecond — independent of and in addition to the JulianToTime truncation bug #54 already fixed on the read-back side. Fixed db.timeToJulian to round to the nearest millisecond, the same class of fix, at the point where the driver-specific noise actually enters. Also tightened the regression coverage: - Pagination tests now use a fixed loop bound of exactly ceil(n/pageSize) pages rather than an open "while non-empty" loop, so a non-advancing cursor fails the test immediately instead of hanging it. - Added a sub-millisecond-boundary fixture (events 3ms apart, well inside the old epsilon window) with rid order deliberately decoupled from mtime order — a monotonic fixture cannot expose this class of bug, since rid and mtime need not move together in general (backdated commits, imports). - TestJulianRoundTrip now asserts exact equality for values originating from TimeToJulian instead of a *1ms tolerance, which would have passed under both the old truncating JulianToTime and the current rounding one and so was not a regression guard for that fix. No change to Ancestry, event-type filtering, NULL tolerance, or either loop bound; all were already correct. * fix: assert pagination termination; mark Cursor change breaking Both pagination tests (the pre-existing ones and the sub-millisecond regression) ran exactly ceil(n/pageSize) pages and asserted each event appeared once, but neither ever called Timeline one more time to confirm the next page comes back empty. The headline symptom of the epsilon-cursor defect these tests exist to catch was non-termination — a caller paginating until empty never stops — and that specific failure mode was going unguarded: a cursor that enumerates every row correctly but never signals exhaustion would have passed all four tests as they stood. Added one more Timeline call after the last expected page in all four tests, asserting it returns zero entries. Verified this is not vacuous: reverted the tie-break to <= (non-strict) and confirmed the existing per-event count checks catch it; separately, made Timeline phantom-return one row only on an already-exhausted call (isolated from the real per-page counts, which stayed correct in that experiment) and confirmed only the new final-page assertion fails — proving it exercises a real, non-redundant path, not one already covered by the count checks. Both experiments reverted before commit. Also corrected the CHANGELOG bullet describing the Cursor redesign: it described the new Cursor type but never stated that TimelineOpts.Before (time.Time) and TimelineOpts.After (FslID) were deleted outright, not deprecated. A reader diffing releases would have read it as an addition rather than a second compile-breaking change on top of #54. check-in: 168e20fc91 user: [email protected] tags: trunk | |
| 16:00 |
fix: split repository enumeration from ancestry walking (#45) (#54)
Repo.Timeline was an ancestry walk wearing an enumeration's name: it required a start rid, emitted that check-in, followed plink where isprim=1 to the primary parent, and repeated until the chain ended. It never touched the event table and never visited a second parent or a sibling branch head, so it silently omitted any check-in that wasn't a first-parent ancestor of the given start rid — on libfossil's own repository that was 577 of 3,761 check-ins, with no error and no warning. The limit option never bound because the (wrong) walk exhausted first. Split into two operations under honest names: - Repo.Timeline(TimelineOpts) enumerates the event table newest first, every kind by default or a single kind via Type. This matches canonical `fossil timeline`'s -t/--type default (a filter only applies when given and not 'a'). - Repo.Ancestry(LogOpts) keeps the original first-parent walk, unchanged in behavior, under a name that says what it does. Breaking change, no deprecated shim (pre-1.0, clean break): callers of the old Timeline(LogOpts) need to move to Ancestry(LogOpts) if they want the walk, or to Timeline(TimelineOpts) if they want the enumeration fossil's own timeline command performs. The only in-tree callers were the `repo timeline` CLI command and the two examples; all three are migrated to Timeline(TimelineOpts). See CHANGELOG.md. LogEntry gains a Kind field identifying which of event.type's six kinds (ci, e, f, g, t, w) an entry is; Parents is only populated for check-ins, since plink only relates check-in artifacts. Timeline's pagination cursor orders by (mtime DESC, rid DESC) instead of canonical fossil's bare mtime DESC with no tie-break — a deliberate improvement, not an oversight, since fossil's web cursor combines that missing tie-break with a one-second timestamp slop and can repeat or skip rows at a page boundary. Getting the composite cursor to actually round-trip losslessly required fixing a real bug along the way: JulianToTime truncated instead of rounded when converting a julian day back to milliseconds, so TimeToJulian(JulianToTime(m)) was not always equal to m even for exact-millisecond inputs — enough to misplace a row at a page boundary under exactly the same-mtime conditions this issue's acceptance criteria test for. Fixed to round to nearest. Because the same class of drift also appears in the ncruces/WASM sqlite driver's own internal REAL-to-time.Time conversion (independent of this library's code), the cursor comparison also carries a small explicit tolerance rather than relying on bit-exact equality across drivers. Both the ancestry walk's loop and the enumeration's row loop now carry an explicit, testable bound so a corrupt or cyclic plink table can't hang the process. check-in: 9a2697c556 user: [email protected] tags: trunk | |
| 16:00 |
fix: separate blob existence from blob availability (#43) (#52)
* fix: separate blob existence from blob availability (#43) A phantom is a real blob row with size = -1 and NULL content, so blob.Exists — "SELECT rid FROM blob WHERE uuid=?" — answers true for it. Every guard built on Exists therefore admitted artifacts whose content cannot be read, and content.Expand aborted with "rid N is a phantom". Canonical Fossil keeps two predicates and never substitutes one for the other: rid_from_uuid (existence, xfer.c:70) and content_is_available (availability, content.c:163). The latter is transitive — it follows delta.srcid up the chain, so a delta whose base is a phantom is unavailable even though every row in the chain exists. Add content.IsAvailable / content.AvailableByUUID, ported from content_is_available including the chain walk, with an explicit depth bound so a cyclic delta table terminates instead of hanging. The walk lives in exactly one place; no caller touches the delta table. blob.Exists keeps its behavior and gains a doc comment stating that it answers existence only, returns true for phantoms, and naming the availability predicate as the alternative. manifest.blobPresent is deleted. It was a local patch at one call site that tested size >= 0 without the chain walk, so it still reported a delta-on-phantom as present — the same class of bug it was written to work around. Every blob.Exists call site was classified as existence or availability and converted where it guards a content read. Sites that want rid identity — plink/mlink/tagxref population, cluster phantom creation, private-status bookkeeping, the post-store sanity check in handleFile, and phantom filling in storeReceivedFile — keep existence, since delta-before-base is a normal steady state during clone. Verified against a loopback fossil 2.28 server: the 23-artifact repro that previously failed roughly half the time now clones 6 of 6. * fix: IsAvailable must fail closed when the delta lookup errors Only sql.ErrNoRows means the chain is grounded. Any other failure — cancelled context, closed connection, NULL srcid — says nothing about groundedness, so returning true there claims readable content that was never verified. That is the same class of bug this predicate exists to prevent, and it runs on data arriving over the wire during clone. The blob-size lookup above already fails closed on the same pattern; this restores the symmetry. check-in: 670ca0e75f user: [email protected] tags: trunk | |
| 15:31 |
fix: capture executable bit at Add/Manage time and propagate to commit (#48)
Manage() inserted new vfile rows with isexe always defaulting to 0, regardless of the file's on-disk permissions. buildCommitFiles() also never consulted vfile.isexe when constructing the manifest F-card for changed/added files — it only carried forward the parent manifest's Perm string, which is empty for a brand new file. Together this meant an executable file added via Manage always committed without an "x" marker, and a later Extract wrote it back as 0644. Fix: - Manage() now stats the file and records isexe using the same predicate as Fossil's file_perm() (src/file.c:316): owner-execute on a regular file (S_ISREG && S_IXUSR). Group/other-only execute (e.g. 0645) does not count, matching canonical behavior. The detection lives in one place (modeIsExecutable, next to the existing isexe-to-mode helper vfilePerm) rather than being duplicated at call sites. - buildCommitFiles() now derives the F-card Perm for changed/added files from vfile.isexe instead of only inheriting the parent's Perm, without disturbing an existing symlink marker ("l"). Extract already read isexe correctly when writing files back to disk, so no change was needed there. Out of scope, left untouched: symlink (islink) handling, and executable-bit CHANGE detection on an already-tracked file whose mode changes without going through Manage again (Fossil's belt-and-braces re-stat at commit time / VFileIsExec path) — that path appears to still be missing here and should be tracked separately if it matters. check-in: 76dd5241c2 user: [email protected] tags: trunk | |
| 15:25 |
fix: tolerate NULL event.user/comment in manifest.Log (#47)
Fossil's check-in manifests require only the D and Z cards; the U-card (user) and C-card (comment) are optional, and canonical fossil binds a missing U-card as SQL NULL on the event row. manifest.Log scanned both columns into plain strings, so any repository containing such a check-in failed the entire walk with "converting NULL to string is unsupported" and returned zero rows. Scan event.user and event.comment through sql.NullString and use its zero-value String field, so a NULL column becomes "" without changing LogEntry's public shape - no nullable type leaks out, and no caller has to know NULL was ever possible. Non-NULL columns are unaffected. repo timeline now renders an empty user as "?", matching canonical fossil's coalesce(euser,user,'?') TTY output; the substitution stays in the CLI's presentation layer, not the library. Fixes #44 check-in: 9564ccad55 user: [email protected] tags: trunk | |
| 15:24 |
Provision ticket schema at repo creation (#46)
Newly created repositories were missing the ticket and ticketchng tables and the ticketchng_idx1 index that fossil new provisions. Stock Fossil's web UI queries the ticket table unconditionally (e.g. info.c's artifact-detail lookup: SELECT 1 FROM ticket WHERE tkt_uuid GLOB ...), so opening a libfossil-created repository with the stock fossil binary and visiting /info/<uuid> crashed with "no such table: ticket". Add the ticket and ticketchng table definitions and the ticketchng_idx1 index to schemaRepo2, taken verbatim from zRepositorySchema2 in fossil's src/schema.c, in the same position canonical fossil places them (after attachment_idx2, before cherrypick). Verified against a real fossil new repository: sqlite_master table/ index diff and per-table .schema diff for ticket/ticketchng now match exactly. check-in: 099ddb1b7a user: [email protected] tags: trunk | |
| 11:08 |
chore(deps): bump the go_modules group across 2 directories with 1 update (#42)
Bumps the go_modules group with 1 update in the / directory: [golang.org/x/crypto](https://github.com/golang/crypto). Bumps the go_modules group with 1 update in the /observer/otel directory: [golang.org/x/crypto](https://github.com/golang/crypto). Updates `golang.org/x/crypto` from 0.49.0 to 0.52.0 - [Commits](https://github.com/golang/crypto/compare/v0.49.0...v0.52.0) Updates `golang.org/x/crypto` from 0.49.0 to 0.52.0 - [Commits](https://github.com/golang/crypto/compare/v0.49.0...v0.52.0) --- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-version: 0.52.0 dependency-type: direct:production dependency-group: go_modules - dependency-name: golang.org/x/crypto dependency-version: 0.52.0 dependency-type: indirect dependency-group: go_modules ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> check-in: c5e189600f user: [email protected] tags: trunk | |
|
2026-06-30
| ||
| 20:38 |
Rebrand to go-libfossil + add acknowledgements (#40)
* docs: rebrand project to go-libfossil; add acknowledgements Rename the project/repo identity to go-libfossil while keeping the Go module path (github.com/danmestas/libfossil) and the libfossil CLI/package names unchanged — imports and installs are unaffected, GitHub redirects cover the renamed repo URLs. - README: title, Tests badge repo URL, and a note clarifying the repo-name vs import-path split. - Add an Acknowledgements section crediting Stephan Beal (libfossil), Dr. D. Richard Hipp / Fossil SCM, TigerBeetle TigerStyle, John Ousterhout (A Philosophy of Software Design), and Antithesis (DST), with links. - Docs site: hugo.yaml title + GitHub repo links, landing page identity surfaces. baseURL, module paths, and CLI/API names left intact. * docs: add go-libfossil logo to README check-in: 97ae7eb00d user: [email protected] tags: trunk | |
|
2026-06-09
| ||
| 06:11 | fix checkout portability under ncruces (#39) check-in: 6abfdc7c25 user: [email protected] tags: trunk | |
|
2026-05-13
| ||
| 19:09 |
fix: DSN _txlock=immediate to serialize writers at BEGIN (#33) (#34)
Two concurrent libfossil.Repo handles on the same SQLite file fail their writes with SQLITE_BUSY immediately, well under the configured busy_timeout=5000 window. Reproduces ~50% of the time on contended manifest.Checkin paths. Root cause: db.WithTx opens BEGIN DEFERRED. Under WAL, a deferred transaction acquires SHARED on first read, then upgrades SHARED→RESERVED on first write. If two connections race that upgrade, SQLite's deadlock-avoidance returns SQLITE_BUSY *immediately*, bypassing busy_timeout entirely (documented behavior, sqlite.org/c3ref/busy_timeout.html). Fix: prepend _txlock=immediate to the DSN in both modernc and ncruces drivers. Every BEGIN now acquires RESERVED upfront, so the second writer blocks at BEGIN — where busy_timeout *does* apply — and the two transactions serialize cleanly. Regression tests added to both driver packages: - TestBuildDSN_TxlockImmediate covers the DSN shape across empty/non-empty pragma maps. - TestConcurrentWritersDoNotRace reproduces the issue-#33 contention pattern (two conns, SELECT then INSERT under DEFERRED) over 25 iterations and asserts zero SQLITE_BUSY. Without the fix, this test fails 25/25; with the fix, 0/25. Closes #33. check-in: 202576c58c user: [email protected] tags: trunk, v0.6.3, db/driver/ncruces/v0.2.0, db/driver/modernc/v0.2.0 | |
|
2026-05-12
| ||
| 23:48 |
fix: Repo.Commit preserves parent's tracked files (#30) (#32)
Repo.Commit was a thin pass-through to manifest.Checkin: callers who supplied a subset of the parent's tracked files got a partial manifest, so files tracked at the parent silently disappeared from the new commit. A checkout at the child's rev saw only the supplied subset — bones, EdgeSync, and sesh all hit this as silent data loss across slot commits. Default behavior now matches `fossil ci`: when ParentID > 0 the parent's F-cards are merged into the commit's file set (caller wins on name collision; untouched parent files are loaded by UUID and carried forward). For the rare case where a caller intentionally wants omission to mean deletion, CommitOpts.PartialManifest opts out of the merge. The internal diff/deletion tests that previously relied on the partial behavior now set this flag explicitly. A first-class deletion API may replace the escape hatch later. Adds TestCommitPreservesParentFiles as the regression: parent [a,b,c], child supplies only [b'], child's ListFiles must return [a,b',c] with a and c keeping the parent's blob UUIDs. check-in: 862dcb20c3 user: [email protected] tags: trunk, v0.6.2 | |
| 20:01 |
feat: CreateOpts.ProjectCode — accept explicit project-code on Create (#31)
* feat: CreateOpts.ProjectCode — accept explicit project-code on Create Downstream EdgeSync coordinates a shared project-code across sibling Fossil repos created on different machines so its NATS sync subject ("<prefix>.<project-code>.sync") matches across leaves. Today every Create generates a fresh project-code, so commits never propagate. Add CreateOpts.ProjectCode: empty preserves current behavior; non-empty must be 40-char lowercase hex (^[0-9a-f]{40}$) — the same format generated project-codes use. Validation runs at the top of libfossil.Create before any disk I/O, so invalid input cannot leave a half-written file behind. Server-code stays auto-generated. Threaded through repo.Create / repo.CreateWithEnv / db.SeedConfig. SeedConfig validates as defense-in-depth (public package, multiple callers possible). Clone accepting ProjectCode is out of scope; will follow separately if EdgeSync needs it. * test: round-trip explicit ProjectCode through fossil rebuild Asserts that vanilla `fossil` (2.28) accepts a Go-side repo created with an explicit ProjectCode, and that `fossil sql` reads back the same value we set. Closes the gap where the existing TestCreate_FossilValidation only covered the generated-project-code path. check-in: 6b66ea6b2d user: [email protected] tags: trunk, v0.6.1 | |
|
2026-05-05
| ||
| 15:11 |
fix: checkpoint WAL on Close so vanilla fossil can read repo (#28)
Closes #27. libfossil opens repos in WAL journal mode but the Close chain never ran wal_checkpoint, so closed repo files were a 4 KiB stub plus a fat -wal sidecar. Vanilla fossil's validity probe stats the main file before consulting the WAL → repo rejected as "not a valid repository". - Adds a typed CheckpointMode (Passive / Full / Restart / Truncate) and a Checkpoint(mode) method on the public Repo (mirrored through internal repo and db.DB). CheckpointMode and its constants are re-exported from the package root so callers don't have to import db. - db.DB.Close now runs PRAGMA wal_checkpoint(TRUNCATE) before closing the underlying *sql.DB. Checkpoint errors are joined with the close error via errors.Join so the connection is always closed regardless of checkpoint outcome. - Skips the close-time checkpoint on the WASM/WASI build path where WAL isn't used. check-in: faf57ad9c7 user: [email protected] tags: trunk, v0.6.0 | |
|
2026-05-04
| ||
| 18:59 |
feat: support whole-checkin diff via empty filePath (#26)
Repo.Diff has a multi-file return shape ([]DiffEntry) but previously rejected filePath == "" with "filePath is required". Consumers that wanted a whole- checkin diff (the equivalent of `git diff revA..revB` or `fossil diff --from revA --to revB`) had to enumerate changed files themselves and call Diff per file; EdgeSync's Agent.Diff is the in-tree example of that workaround. This relaxes the guard: when filePath == "", Diff now returns one DiffEntry per file that changed between ridA and ridB, sorted by Name for deterministic ordering. The non-empty filePath path is unchanged. Identical checkins return ([]DiffEntry{}, nil). Whole-checkin enumeration is name-keyed (union of files across both manifests where the content UUID differs or the file exists on only one side). Proper rename detection via mlink (so a pure rename produces a single entry rather than an add+delete pair) and permission-only change reporting are deferred to a follow-up; the limitation is documented on Diff's godoc and called out in the PR description. The single-file path is unaffected. The previous TestDiff_EmptyFilePath (which asserted the rejection error) is replaced with positive coverage for identical checkins, single-file change, multi-file change, addition-only, and deletion-only. closes #24 check-in: 764a82ecdb user: [email protected] tags: trunk, v0.5.0 | |
| 18:59 |
feat: add Repo.UUIDFromRID for public rid->uuid lookup (#25)
Consumers (bones, EdgeSync) currently bypass the methods-only Repo API to convert RIDs to UUIDs by reaching into r.DB() and running raw SQL. The lookup is well-defined and recurs throughout the libfossil internals; expose it as a first-class method. Add Repo.UUIDFromRID(rid int64) (string, error) backed by a small internal helper Repo.UUIDByRID on internal/repo so the SELECT lives in one place. Missing rids surface ErrArtifactNotFound (wraps sql.ErrNoRows behind a typed sentinel) with the rid in the message. closes #23 check-in: 61aa4dd022 user: [email protected] tags: trunk | |
|
2026-04-30
| ||
| 14:44 |
docs: cross-repo release automation spec + implementation plan (#22)
* docs: add cross-repo release automation design spec Designs an automated release pipeline across libfossil → edgesync → bones: - repository_dispatch chain replaces manual go.mod bumps - linear-only graph (libfossil does not dispatch directly to bones) - per-repo `make ci` mirrors GitHub CI verbatim; `make ci-fast` for pre-push - `make release VERSION=...` helper with annotated tag template - GoReleaser auto-changelog (edgesync, bones); `gh release --generate-notes` for libfossil - 6-PR rollout: 3 hygiene PRs first, then 3 dispatch wiring PRs See docs/superpowers/specs/2026-04-30-cross-repo-release-automation-design.md * docs: add cross-repo release automation implementation plan Breaks the design spec into 6 sequential PRs across libfossil, edgesync, and bones — 3 hygiene PRs (Makefile + pre-push hook + release helper) followed by 3 dispatch-wiring PRs. Plan includes: - Phase 0 prerequisites (DISPATCH_PAT setup per upstream repo) - Per-PR branch creation, file-level steps, expected commands/outputs, commit messages, and PR-creation gh commands - Post-merge smoke tests for each receiver/sender combo - Final end-to-end cascade validation via v0.4.6-rc1 tag See docs/superpowers/plans/2026-04-30-cross-repo-release-automation.md * docs: correct bones make ci scope to include make check bones' .github/workflows/ci.yml runs make check (fmt-check + vet + lint + race + todo-check) before the otel-tagged lanes. The earlier scope captured only the otel lines, missing the discipline suite that is the primary CI gate. Update both spec and plan to match the actual workflow. The cross-repo leaf-binary integration step (sibling EdgeSync + make leaf) remains intentionally CI-only — it requires a sibling working tree that doesn't generalize to a single-repo make target. * docs: capture libfossil deviation (no pre-push, no ci-fast) PR #3 (#19, merged 2026-04-30) shipped a tighter scope than PR #1/#2 because libfossil's existing pre-commit is already comprehensive. Document the option (A) decision in both spec and plan, and add a multi-module note for the future case if pre-push is ever added. * docs: add go get retry + actor field to receiver workflows PR #4 (bones #90) added a follow-up commit per code review with these two improvements; capture them in spec/plan so PR #5's edgesync receiver picks them up automatically: - 3-attempt retry with 15s backoff covers GOPROXY propagation race (dispatcher fires immediately on tag push, receiver may outrun the module proxy). workflow_dispatch fallback still exists for manual recovery. - Actor field plumbed payload→input→github.actor with triple fallback and rendered as 'Triggered by: @actor' in bump PR body for audit. check-in: 339643c721 user: [email protected] tags: trunk | |
| 14:37 |
feat(ci): add release workflow with downstream dispatch (#21)
On tag push (v*): 1. Create GitHub release with auto-generated notes from PRs/commits since previous tag. Source tarball auto-attached. 2. Dispatch repository_dispatch event to danmestas/EdgeSync to trigger a chore-bump PR there. libfossil is a library — no binaries are published, so goreleaser is overkill. 'gh release create --generate-notes' provides equivalent auto-changelog and source archives in 5 lines. Closes the cascade chain. Once merged, the next libfossil tag will fire PRs in EdgeSync (PR #5 receiver) and via that, in bones (PR #4 receiver). check-in: 38e4948972 user: [email protected] tags: trunk | |
| 13:50 |
chore: add make ci targets and release helper (#19)
* feat(make): add ci targets mirroring test.yml verbatim ci aggregates 3 sub-targets matching the 3 GitHub Actions jobs: ci-default — modernc driver tests + DST + vet + cmd build ci-ncruces — ncruces driver tests with -tags test_ncruces ci-otel-target — observer/otel sub-module tests All use GOWORK=off (matching CI). Existing test/test-drivers/test-otel targets are left untouched — they don't use GOWORK=off and have drifted from CI; ci-* are the single source of truth going forward. No ci-fast / pre-push hook in this PR — libfossil's existing pre-commit hook (modernc + ncruces + vet + otel + build + SDK drift, ~45s) already exceeds what ci-fast would provide. Adding a pre-push would just double push cost without catching new bugs. * feat(make): add release VERSION=... helper Validates clean tree, runs make ci, opens \$EDITOR with auto-populated tag annotation template, creates annotated tag locally. Push is intentionally manual. * feat: add release annotation template Used by 'make release VERSION=...' as the body skeleton for annotated tags. check-in: 5fd8907820 user: [email protected] tags: trunk | |
| 10:13 |
release: v0.4.5
Since v0.4.4: - sync: converge clone against writing hub (#17, #18) check-in: a22373f305 user: [email protected] tags: trunk, v0.4.5 | |
| 10:10 |
fix(sync): converge clone against writing hub (#17) (#18)
A clone against a hub being written to during the session reliably hit "exceeded 100 rounds" once the production scenario exercised the multi-round path. Three compounding root causes: 1. Server-side: emitCloneBatch issued an open-ended `WHERE rid > cursor` query with no upper bound, so blobs committed between rounds extended the queue indefinitely and CloneSeqNoCard{SeqNo:0} never fired. Fixed by capturing max(rid) at the start of the clone session and round- tripping it via CookieCard so the otherwise stateless handler can scope the query: `rid > cursor AND rid <= cap`. 2. Client-side: the libfossil clone client sent CloneCard (clone-mode signal) without CloneSeqNoCard (the actual pagination cursor, per TestHandleClonePagination). Server's cloneSeq stayed 0 every round, masked by every existing clone-client test fitting in DefaultClone- BatchSize=200. Fixed by emitting CloneSeqNoCard on continuation rounds (cycle > 0). 3. Cookie carrier: clone client previously ignored CookieCards on the wire. Now stores and echoes them, mirroring the existing sync client pattern. Adds three regression tests: - TestCloneAgainstWritingHub (direct issue #17 repro) - TestCloneMultiRoundCursor (catches the cursor bug in isolation) - TestCloneSnapshotBoundExcludesPostSnapshotWrites (snapshot semantic) The fix is libfossil-internal: protocol carrier is the existing CookieCard with an opaque `lf-clone-cap:<rid>` value, so non-snapshot-aware peers fall back to pre-fix behavior with no wire-format break. check-in: 25e235b387 user: [email protected] tags: trunk | |
|
2026-04-27
| ||
| 11:13 |
release: v0.4.4
Since v0.4.3: - sync: prevent rid=0 panic on partial xfer (#14, #16) check-in: 5d6bd072ef user: [email protected] tags: trunk, v0.4.4 | |
| 11:10 | fix(sync): prevent rid=0 panic on partial xfer (#14) (#16) check-in: 30bfac2f91 user: [email protected] tags: trunk | |
|
2026-04-26
| ||
| 15:21 |
release: v0.4.3
Auto-version landing now reads this tag at CF build time. Since v0.4.2: - diff: emit "\\ No newline at end of file" for byte-fidelity with fossil (#10) - repo: expose ResolveVersion + ReadFileAt for symbolic versions (#11) - xfer: always emit project code on push/pull cards (#12) - landing: inject version from latest git tag at build time (#13) check-in: d5dd3dbc3e user: [email protected] tags: trunk, v0.4.3 | |
| 15:20 |
feat(landing): inject version from latest git tag at build time (#13)
Replace the three hardcoded "0.4.2" strings in the landing layout with {{ .Site.Params.version }} template directives. Add scripts/sync-version.sh that reads the latest git tag via 'git describe --tags --abbrev=0' and rewrites hugo.yaml's params.version line. Wire it into wrangler.jsonc's build command so Cloudflare's git-integrated build picks up new tags automatically — push a commit to main with a fresh tag, CF rebuilds, the landing shows the new version with no PR required to bump strings. Footer year switched to {{ now.Year }} so 2027 doesn't surprise anyone. Local fallback: when no tags are reachable (very early dev), the script exits cleanly and the layout's 'default' filter falls back to the value already in hugo.yaml. check-in: ab1e500bc4 user: [email protected] tags: trunk | |
| 15:00 |
fix(xfer): always emit project code on push/pull cards (#12)
The encoder emitted bare 'push\n' / 'pull\n' cards in code paths where the project code wasn't passed explicitly. The upstream Fossil C server's xfer parser requires at least the project code on push and both project + server codes on pull, rejecting 0-arg cards. This broke libfossil <-> fossil interop on the first sync round. Make the project code a required parameter on the encoder and update call sites to thread it through. The libfossil server-side decoder is tightened to match — it previously accepted 0-arg cards, which masked the encoder bug during Go-to-Go testing. check-in: 4082bffaa5 user: [email protected] tags: trunk | |
| 15:00 |
feat(repo): expose ResolveVersion + ReadFileAt for symbolic versions (#11)
* feat(repo): expose ResolveVersion + ReadFileAt for symbolic versions Repo.ReadFile took a raw int64 RID, leaving every library caller to re-implement symbolic-version resolution (tip, trunk, branch name, UUID prefix) — work the CLI's resolveRID already does inside the cli package. Move the resolver onto Repo as ResolveVersion, refactor cli/shared.go to call it, and add Repo.ReadFileAt(version, path) so callers can read by name without first calling ResolveVersion themselves. Adds ErrVersionNotFound for the not-found case and an ambiguity error for prefix collisions. * chore(cli): drop unused openInternalRepo helper + repo import Now-dead code: resolveRID was its only caller, and resolveRID is now a one-line wrapper around Repo.ResolveVersion that doesn't need the inner handle. Also removes the now-unused internal/repo import that the helper brought in, and refreshes the cli SDK page accordingly. * chore(go.work.sum): pin transitive deps from docs toolchain gomarkdoc pulls these via go install during make docs-gen-sdk; commit the lockfile entries so contributors see no go.work.sum drift after a docs build. check-in: 7eb46a9635 user: [email protected] tags: trunk | |
| 14:59 |
fix(diff): emit "\ No newline at end of file" for byte-fidelity with fossil (#10)
Upstream fossil's diff output marks any side whose content does not end with a newline using the standard POSIX marker. internal/diff/diff.go was stripping trailing empty lines via splitLines and never emitting the marker, producing diffs that decoded cleanly Go-side but diverged from the upstream tool — recipients reconstructing file content from the diff would silently introduce a newline that was not in the source. Track per-side ends-with-newline status and emit the marker after the final hunk line of each side that lacks the terminator. check-in: 2a11bf319a user: [email protected] tags: trunk | |
| 14:14 |
docs: usability improvements from DX audit (#9)
Implements all seven recommended changes from the docs audit: 1. SDK pages now show package names in nav. Add scripts/prepend-sdk-frontmatter.sh that prepends '--- title: <pkg> ---' to each gomarkdoc-generated api.md; wired into 'make docs-gen-sdk'. Sidebar previously showed five undifferentiated 'API' entries; now shows cli, db, dst, libfossil, observer/otel. 2. /examples/ populated with paragraphs + GitHub deep-links for embed-repo-api and wasm. No longer a placeholder pointing back at the repo. 3. Architecture page gets an inline SVG mirroring the landing-page diagram. Self-contained CSS (uses currentColor + ember accent) so it tracks Hextra's light/dark theme without depending on the landing's CSS scope. 4. Cross-links added: quickstart -> production + extension-points, migration -> architecture, architecture -> extension-points. Extension Points and Production patterns are now reachable via prose, not just the sidebar. 5. /docs/ landing expanded from 3 sentences to a 7-line subsection map calling out each page's purpose. 6. Version indicator in navbar (libfossil v0.4.2). Override layouts/_partials/navbar-title.html mimics Hextra's default and appends a small mono version tag from .Site.Params.version. 7. New /docs/production/ page covering error handling, context cancellation, telemetry wiring, repo-handle lifetime, and driver selection — the patterns the quickstart's log.Fatal shortcuts gloss over. check-in: f30849a466 user: [email protected] tags: trunk | |
| 08:23 |
docs: wire live docs URL into README + hugo baseURL + SDK refresh (#8)
- README: replace placeholder URL with the live CF Workers URL, add a Docs badge to the badge cluster so the link is prominent on the GitHub repo page. - hugo.yaml: set baseURL so Hugo emits absolute canonical <link> tags and a real sitemap.xml instead of root-relative paths. - SDK pages: refresh gomarkdoc output (drifted from current Go doc comments since the original generation). Same regen that PR #7 contains; either-order merge is a no-op for the second. The GitHub repo's About sidebar now also links to the docs site (set via 'gh repo edit --homepage'). check-in: 0ca5c54ee6 user: [email protected] tags: trunk | |
| 08:12 |
site(landing): SVG architecture diagram + SDK refresh (#7)
* site(landing): SVG architecture diagram + SDK regen - Replace §04 ASCII diagram with an SVG mirroring the dagnats style: three columns (callers / core / plugins), ember flow arrows with CALLS / DELEGATE / EVENTS / PERSIST labels, bottom box for .fossil (SQLite). Picks up the layout's existing .diagram CSS with no new styles. - Refresh gomarkdoc-generated SDK pages — they had drifted from current Go doc comments since the original generation. * docs(site): drop redundant body H1s from frontmatter pages Each of the four moved docs (architecture, extension-points, migration-from-fossil, testing) and the two hand-written driver _index pages had a body '# Title' that duplicated their frontmatter 'title:'. Hextra renders both, producing the visible doubled headline. Removing the body H1 leaves Hextra's frontmatter-driven page heading as the single source. SDK api.md pages from gomarkdoc are unaffected — they have no frontmatter, so Hextra promotes their body H1 to the page title without duplication. * site(landing): bump version refs to v0.4.2 check-in: d1016bd2ef user: [email protected] tags: trunk | |
| 07:52 |
docs site + landing + Cloudflare auto-deploy (#6)
* design: docs site + landing page + Cloudflare auto-deploy Mirror dagnats's docs/landing infrastructure for libfossil: a single-page landing at site/index.html (forked CSS/HTML, content rewritten), a Hugo + Hextra docs site at docs/site/, gomarkdoc-generated SDK reference under docs/site/content/docs/reference/sdk/, llms.txt artifacts, and a wrangler.jsonc that lets Cloudflare's git integration auto-deploy on push to main. Existing docs/*.md files move under docs/site/content/docs/. * design: correct landing-page path to Hugo layout override dagnats's working landing page lives at docs/site/layouts/index.html (a Hugo home-page layout override), not at site/index.html. The top-level site/index.html in dagnats is an older draft that the build pipeline ignores. Match the working setup. * design: align visual system + section list with live dagnats landing The earlier description came from dagnats's stale top-level site/index.html (JetBrains Mono + Instrument Serif italic + ember/teal accents, 8 sections). The live Hugo layout that actually serves at / uses Geist Mono only, prefers-color-scheme dual-mode (warm cream / warm dark), a single ember accent, and 7 sections (Install, What it is, Why you might want it, How it fits together, Compared to neighbors, What it is not, Get going). Update the spec to match what we are actually mirroring. * design: drop SDK regen from CF build to match dagnats exactly Earlier draft had CF install Go + gomarkdoc and re-run docs-gen-sdk on every deploy. dagnats does not do that; it relies on the pre-commit hook to keep committed SDK pages in sync, and ships only Hugo + llms-gen + hugo --minify in the CF build. Match that: smaller, faster CF build, no Go-runtime assumption about Workers Builds, and a true 1:1 mirror. * plan: docs site + landing + CF auto-deploy implementation steps 11 tasks covering Hugo+Hextra bootstrap, content moves, gomarkdoc SDK generation with pre-commit drift check, llms.txt artifacts, Makefile targets, landing page fork from dagnats's live Hugo home layout, wrangler.jsonc for CF Workers Static Assets, README update, and PR creation. * docs(site): bootstrap Hugo + Hextra skeleton * docs(site): move existing docs into Hugo content tree * docs(site): fix links + H1s broken by docs-tree move - Drop .md extensions on Hugo cross-doc links - Switch source-code references to absolute github.com/danmestas/libfossil URLs (relative ../ paths broke when files moved 3 dirs deeper) - Align H1 of architecture.md and migration-from-fossil.md with their frontmatter titles * docs(site): add quickstart, examples, and SDK section stubs * docs(site): tighten quickstart snippets + reference index - Make the Sync example self-contained (open + sync, full main, real signatures) - Bridge from Clone to Open with a one-line prose link - Add a sentence to reference/_index.md so it does not render blank - Use the project's canonical "pure-Go, CGo-free" phrasing * docs(site): add docs-gen-sdk Makefile target + commit generated SDK pages * docs(site): pin gomarkdoc version + replace empty driver stubs with prose - Add GOMARKDOC_VERSION Makefile variable so contributors don't drift on output format when a new gomarkdoc ships - Drop the two driver packages (modernc, ncruces) from gomarkdoc and hand-write _index.md pages explaining the register-on-import contract; the stubs were byte-stable but rendered as visibly broken documentation * docs(site): add gen-llms-txt.sh + docs-gen-llms target * docs(site): switch llms-full frontmatter strip from sed to awk The sed range-negation pattern '/^---$/,/^---$/!p' silently re-opens a suppression range on any bare '---' in body content, eating Markdown horizontal rules and embedded YAML in fenced code blocks. The awk pattern strips the frontmatter only when it begins on line 1, leaving body content intact regardless of how many '---' lines it contains. * hooks: pre-commit fails when SDK pages drift from code * hooks: pre-commit re-installs gomarkdoc if it's the wrong version The drift check's existing gate just looks for gomarkdoc on PATH. A contributor with an older gomarkdoc installed locally would have the gate pass and then run regeneration with their stale binary, producing either a silent false-negative or a confusing OUT-OF-DATE error with no version hint. Compare the installed version against GOMARKDOC_VERSION from Makefile and re-install on mismatch. Best-effort; offline contributors can still commit. * docs(site): add docs-serve, docs-build, docs aggregate targets * docs(site): make docs aggregate parallel-safe via explicit deps The docs aggregate previously listed gen-sdk, gen-llms, and build as side-by-side prereqs. Make doesn't guarantee left-to-right ordering under -j, and gen-llms reads what gen-sdk writes, while build reads both. Encode the chain explicitly so 'make -j docs' is correct. Side benefit: 'make docs-build' standalone now produces a complete deployable site, matching contributor expectation. * site(landing): fork dagnats home layout; rebrand chrome * site(landing): rewrite §01–§07 content for libfossil * site(landing): use .ascii class for diagram + fix §03 phrasing * deploy: add wrangler.jsonc for CF Workers Static Assets * docs: point README at the new docs site check-in: e2a2f28861 user: [email protected] tags: trunk | |
| 06:51 |
v0.4.2: defer manifest crosslink until referenced blobs arrive (#5)
* sync: defer manifest crosslink until referenced blobs arrive Repo.HandleSync (added crosslink in v0.4.1 PR #4 fix 3) crosslinked each manifest as soon as its blob arrived, even when the manifest referenced blobs that hadn't yet been received in the same session. A receiver that Pulled during a multi-blob delivery saw the manifest in event/leaf/mlink while one or more file blobs were still in flight, and a downstream Checkout.Update walking those manifests via manifest.ListFiles hit 'expandUUID: blob not found for uuid <hex>'. Surfaced by agent-infra's hub-and-leaf scaling trial under 16-way concurrent fork+merge cycles (agent-infra trial #10, see docs/trials/2026-04-25/trial-report.md). The 3-leaf friendly case in agent-infra's hub-leaf-e2e doesn't exhibit it because the broadcast cadence is slower than the multi-blob delivery window. Fix: manifest.Crosslink now defers a Checkin candidate when any referenced blob (F-card files or B-card baseline for delta manifests) is not present locally. The deferral is implicit: the candidate query selects rids without an event/tagxref/forumpost/attachment row, so a deferred manifest re-appears in the next sweep automatically. Since HandleSync runs Crosslink whenever filesRecvd > 0, the next sync round that delivers the missing blob also re-runs the sweep and links the manifest fully. Divergence from fossil-scm/c: fossil's reference handles this via 'rcvfrom' tracking and ipctest-style deferred crosslink flushes; the Go port reuses the existing whole-repo sweep semantics by checking blob presence at sweep time. Documented inline in manifest.Crosslink and missingCheckinRefs. Test coverage: - TestCrosslink_DefersCheckinWithMissingFileBlob — manifest with missing F-card blob is deferred (no event/leaf/mlink rows), then fully linked once the file blob arrives. - TestCrosslink_DefersDeltaCheckinWithMissingBaseline — delta manifest with missing B-card baseline is deferred; baseline + delta both link in the next sweep once the baseline blob arrives. - TestCrosslink_LinksWhenAllBlobsPresent — confirms the unchanged synchronous path (all blobs present before Crosslink runs). All three tests fail without the fix (verified by reverting the crosslink.go change locally) and pass with it. * manifest: assert non-nil repo and non-empty uuid in blobPresent Tiger Style: panic at the boundary on programmer errors. Calling blobPresent with a nil receiver or empty uuid is always a bug — empty uuid would silently return false (no row matches the empty-string predicate), masking the upstream caller that lost track of which uuid it intended to check. The peer helpers Crosslink and missingCheckinRefs already panic on nil inputs; blobPresent was the gap. * manifest: order Crosslink candidate sweep by rid Without an explicit ORDER BY, SQLite is free to return candidate rows in any order convenient to its query planner. The deferred-crosslink sweep re-discovers the same rids across multiple sync rounds, so any implicit ordering would let blob-arrival timing leak into observable behavior: per-defer slog streams, pending-item processing order, and any future caller that consumes Crosslink's side effects in sequence. ORDER BY b.rid pins the sweep to insertion order — the same order blobs were stored — which is the natural arrival order for a single sync and a stable canonical order across replays. * manifest: emit deferred rids and missing-blob list in Crosslink rollup The session-end Info log previously reported only counts. When a sync ends with deferred manifests, an operator needs to know which manifests are stuck and which UUIDs they are waiting on — otherwise a permanently-stuck deferral (blob never arrives) is invisible until someone diagnoses it from the per-defer Debug stream. Add deferred_rids and missing_blobs fields. Sort missing_blobs so the rollup is byte-identical across runs that defer the same set, regardless of map iteration order. * manifest: cover P-card non-deferral and shared-blob defer cases The original three defer tests covered F-card and B-card defer plus the all-present happy path. Three gaps remained: * Multi-manifest sharing a missing blob: prove both manifests defer together and both link when the shared blob lands. Without this, a regression that links one but not the other would slip past. * Missing-parent (P-card) intentionally does NOT defer: the deferral check is scoped to F-cards and B-cards because P-card targets do not break Checkout.Update. Deferring on missing parents would deadlock partial clones with pruned ancestors. Pin the contract with a regression test. * F-card duplicate-UUID dedup: the seen-map in missingCheckinRefs already deduplicates, but the behaviour was untested. Two F-cards with different filenames pointing at the same blob must defer once and link to two mlink rows on arrival.check-in: b6b6c40218 user: [email protected] tags: trunk, v0.4.2 | |
| 02:57 |
v0.4.1: xfer encoder, multi-round sync, manifest crosslink (#4)
* xfer: handle empty ServerCode/ProjectCode without arg-count error Repo.Sync(ctx, transport, SyncOpts{Push: true, Pull: true}) with both codes empty previously wrote "push \n" to the wire. The decoder's strings.Fields collapsed adjacent spaces and failed with "xfer: pull requires 1-2 args, got 0". Encoder now skips trailing empty args. Decoder accepts the verb with 0 args as "no specific code requested" — the same semantic as the existing 1- and 2-arg forms after fossil's normal arg interpretation. Also fixes session.cardSummary's slice-out-of-range panic on empty codes by clamping the prefix length. Adds a roundtrip test for the empty-codes path. * sync: continue round-loop while either side has pending work Sync's round-loop previously exited when the user passed Pull=false even if the peer was still requesting gimmes. The user's Pull flag controls whether we *request* new content; it should not gate whether we *service* the peer's gimmes. Symptom: agent-infra's Manager.Push had to pass Pull=true as a workaround to drive multi-round transfers. Root cause: HandleSync.handleIGot gated gimme generation on pullOK alone. Push-only clients send igot cards from sendUnclustered every round (they always announce what they have), but the server would silently drop them because pullOK was false — leaving the server holding only what the client pushed proactively. Continue the loop while any of: pending ungots to push, pending pulls to request, gimmes to service. Exit only when a round completes with no work in either direction. Concretely the gate is now (pushOK || pullOK), restoring parity with fossil-scm's client_sync continuation logic. Adds an integration test (TestPushOnly_DrainsServerGimmes) that exercises the push-only path end-to-end and a unit test that asserts gimme emission under push-only handleIGot. * sync: crosslink incoming manifests so hub-side state stays consistent Repo.HandleSync previously stored manifest blobs in 'blob' but did not populate the relational tables (event, leaf, plink, mlink) that the timeline, fork detection, and clone protocol depend on. As a result: - Leaves pulling from a hub got blobs but stale 'leaf' state, so WouldFork reported phantom siblings. - Verifier-clone of a hub repo failed with "sync.Clone: exceeded 100 rounds" because 'mlink' was empty and the clone protocol couldn't traverse manifest parents. - Hub timelines were always empty even after successful Push. HandleSync now invokes manifest.Crosslink (the same scanner Clone and Sync run after convergence) when at least one file landed in the round. The scanner walks blobs that have no event/tagxref/ forumpost/attachment row yet, parses them, and populates the relational tables. Local commits already produced this state via manifest.Checkin's existing crosslink path; incoming syncs now match. Coverage spans every artifact type the existing scanner already supports (ci, wiki, ticket, event, attachment, cluster, forum, control). The agent-infra trial that surfaced this only exercises 'ci' artifacts, so the divergence note in the prompt described the narrowest possible repair; reusing the existing scanner avoids the divergence and matches the C reference's full coverage. Adds two tests: TestHandleSync_CrosslinksReceivedManifest verifies event/leaf/mlink rows after pushing a single checkin; the leaf- transition test verifies the parent leaves the leaf table when the child lands and that plink connects them. check-in: 3f290b159f user: [email protected] tags: trunk, v0.4.1 | |
|
2026-04-25
| ||
| 19:09 | feat: Repo.Pull public wrapper + Checkout.Update coverage check-in: c9c7dbe3d3 user: [email protected] tags: trunk, v0.4.0 | |
| 18:54 | libfossil: Tiger Style coverage for Checkout.Update check-in: 5ba9d7556c user: [email protected] tags: trunk | |
| 18:51 | libfossil: drop unsubstantiated threading claim from Repo.Pull doc check-in: 3a3060d725 user: [email protected] tags: trunk | |
| 18:47 | libfossil: Repo.Pull public wrapper (HTTP transport, hostile-input asserts) check-in: 5502b364b5 user: [email protected] tags: trunk | |
| 17:30 | libfossil: failing tests for Repo.Pull (Tiger Style: hostile inputs) check-in: e853fb95fc user: [email protected] tags: trunk | |
|
2026-04-21
| ||
| 11:13 |
feat: implement Repo.ReadFile (#3)
Expose a public wrapper around the internal blobAt helper so callers can read a file's content at a specific checkin rev without going through a checkout. Returns ErrFileNotFound (wrapped) when the path is not tracked in the given checkin. Diff already used blobAt internally; this just surfaces the primitive. check-in: d8b2b1bca4 user: [email protected] tags: trunk, v0.3.0 | |
| 03:03 |
feat: implement Repo.Merge (branch-to-branch 3-way) (#2)
Adds Repo.Merge(srcBranch, dstBranch, message, user) which resolves each branch tip, finds the common ancestor, runs ThreeWayText.Merge against the union of source and destination filesets, and — if every file merges cleanly — records a merge commit whose primary parent is the dst tip and whose secondary parent is the src tip (plink isprim=1 / 0). Any unresolved conflict aborts the whole operation with a *MergeConflictError listing the affected files; callers can check for it via errors.Is(err, ErrMergeConflict). Agreed deletions are honored, and one-sided modify/delete conflicts are reported as conflicts rather than silently dropped. Supporting changes: - CheckinOpts.MergeParents plumbed through manifest.Checkin so the P-card and plink rows carry every parent (previously the fresh-commit path only tracked one parent, unlike the sync path in crosslink.go). - manifest.Checkin now calls tag.PropagateAll(primaryParent) after inline T-card processing, mirroring crosslink.applyInlineTags. Without this, a child commit on an existing branch would not inherit its parent's branch tag, so BranchTip would return stale tips for any non-initial commit. - Repo.BranchTip(name) resolves a branch to its latest checkin via tagxref; secondary ORDER BY rid keeps the lookup deterministic when multiple commits share a julian-day bucket. - CommitOpts.MergeParents exposed on the public API so arbitrary callers can author merge commits without going through Repo.Merge. Tests cover clean merges of divergent edits, conflicting edits, one-sided additions, missing branches, same-branch input, multi-parent plink verification, and BranchTip resolution on both trunk and named branches. Green on both the modernc and ncruces drivers plus the OTel observer. check-in: 461f168708 user: [email protected] tags: trunk, v0.2.0 | |
| 02:36 |
feat: implement Repo.Diff (#1)
Fills in the TODO stub in repo_history.go so Repo.Diff returns a real unified diff for a file between two checkins instead of a "not yet implemented" error. The implementation piggybacks on internal/diff.Unified, which already produces Myers-algorithm output. A small blobAt helper walks the manifest at each checkin, resolves the file's UUID to a blob RID, and expands the content. An absent file on a side is treated as empty bytes so pure additions and deletions render as clean insert/delete hunks. Identical content on both sides returns an empty slice rather than a DiffEntry with an empty Unified string, which keeps callers from having to distinguish "no diff" from "missing output". Adds repo_history_test.go covering modification, identical content, addition, deletion, absent-both-sides, and the empty-filePath error. check-in: a3fbae135e user: [email protected] tags: trunk | |
|
2026-04-20
| ||
| 10:42 | docs: date v0.1.0 release, drop private-migration notes, ignore shm files check-in: a4fbc79e3a user: [email protected] tags: trunk | |
| 10:15 | chore: regenerate go.sum entries for v0.1.0 dependencies check-in: f368d871af user: [email protected] tags: trunk | |
|
2026-04-19
| ||
| 16:46 |
chore: remove replace directives for go install compatibility
Replace blocks in published go.mod files are ignored by consumers but cause `go install @v0.1.0` to fail module resolution. Local development continues to work via go.work, which substitutes workspace modules without requiring go.mod replace entries. Also add observer/otel to the workspace for consistent local builds. check-in: 71751cf461 user: [email protected] tags: trunk, db/driver/modernc/v0.1.0 | |
| 16:30 |
Initial open source release
libfossil is a pure-Go implementation of the Fossil SCM, reading and writing the same .fossil SQLite repository format as upstream Fossil. This is the first public release of code that previously lived in a private repository. The module path is github.com/danmestas/libfossil and the binary is named libfossil to avoid $PATH collisions with upstream fossil. Included: - Repo lifecycle (create, clone), checkout/checkin, timeline, merge, diff, annotate, shun, tag, branch, bisect, stash, undo/redo - Sync-over-HTTP protocol (client and handler) - Two SQLite drivers: modernc (pure Go, default) and ncruces (wasm-capable) - Observer interfaces (SyncObserver, CheckoutObserver) with an optional OpenTelemetry implementation in observer/otel (separate submodule) - Deterministic simulation testing harness with BUGGIFY fault injection - Kong-based embeddable CLI under cli/ and a standalone binary under cmd/libfossil/ check-in: f5744359d3 user: [email protected] tags: trunk | |