IBM Bob 2.0 Hackathon · Evidence-Gated Legacy Modernization

From legacy monolith
to verified modernization.

IBM Bob reconstructs legacy behavior, modernizes the monolith, diagnoses drift, and repairs the candidate. Aegis seals the reference behavior and independently verifies that the modern system preserves the observable behavior represented by the contract.

CURRENT HARDENED SUBMISSION: READY

Legacy → Modernization → Independent Acceptance

This is the Aegis ContractLock product. The PR workflow later is one concrete proof that the full modernization control loop works.

01 · IBM Bob Contract archaeology

Bob investigates the legacy source and available policy context to reconstruct an executable behavioral contract.

02 · Aegis Baseline seal

Aegis executes the legacy reference and records normalized evidence across scenarios, state, ledger, audit, and invariants.

03 · IBM Bob Modernization

Bob transforms the monolith into a cleaner, modular candidate architecture.

04 · Aegis Twin-run verification

Legacy and modern implementations receive the same scenarios and their observable behavior is compared.

05 · Aegis Drift detection

Any mismatch can block acceptance and produce a behavioral counterexample.

06 · IBM Bob Closed-loop repair

Bob diagnoses from evidence and applies a focused repair without self-approving the result.

07 · Human Evidence-gated merge

Aegis re-verifies independently. Bob may review read-only. A human retains merge authority.

LEGACY MONOLITH
      │
      ▼
IBM BOB — CONTRACT ARCHAEOLOGY
reconstruct executable behavior contract
      │
      ▼
AEGIS — BASELINE SEAL
returns + exceptions + state + ledger + audit + idempotency + invariants
      │
      ▼
IBM BOB — MODERNIZATION
monolith → modular modern candidate
      │
      ▼
AEGIS — TWIN-RUN VERIFICATION
legacy reference ⇄ same scenarios ⇄ modern candidate
      │
      ├──────── behavior matches ────────► ACCEPTED
      │
      └──────── behavior drifts ─────────► BLOCKED
                                              │
                                              ▼
                                   BEHAVIORAL COUNTEREXAMPLE
                                              │
                                              ▼
                                     IBM BOB — REPAIR
                                              │
                                              ▼
                                    AEGIS — RE-VERIFY
                                              │
                                              ▼
                                    HUMAN MERGE DECISION
Scope: Aegis demonstrates behavioral equivalence across the defined contract and executed scenario corpus. It does not claim formal proof of complete program equivalence or independent correctness of the legacy business policy itself.

Why enterprises need this

Legacy systems frequently encode business rules in implementation details that are easy to disturb during AI-assisted refactoring.

Legacy risk

Behavior hides in implementation

Tax rules, tier thresholds, rounding, refunds, ledger effects, audit events, and state transitions may not be fully documented.

AI risk

Green tests can still miss drift

A candidate can be cleaner and pass ordinary tests while silently changing a boundary condition or financial outcome.

Aegis answer

Separate transformation from acceptance

Bob changes the software. Aegis checks the evidence. A human decides whether to merge.

Real PR proof: green tests were not enough

A real pricing refactor looked safe until Aegis exercised the exact VIP boundary.

01 · PR Green pull request

Pricing thresholds were refactored into named policy constants.

02 · Aegis 85 / 86 — BLOCKED

One semantic drift appeared at the exact VIP premium threshold.

03 · Impact +$32.18

$965.25 expected. $997.43 produced by the defective PR.

04 · IBM Bob > → >=

Bob diagnosed the inclusive-boundary regression and repaired one operator.

05 · Aegis 86 / 86 — ACCEPTED

The repaired candidate matched the executed contract.

06 · Challenge 24 / 24 BLOCKED

Every runnable mutation in the finite PR-specific challenge was detected.

07 · Human MERGE_ELIGIBLE

Bob reviewed evidence read-only. Human authorization remained required.

Behavioral counterexample

VIP subtotal exactly $1,000.00

Expected discount rate10%
Defective PR discount rate7%
Expected grand total$965.25
Defective PR grand total$997.43
Customer-impact delta+$32.18
IBM Bob repair

Minimal surgical fix

- subtotal > VIP_PREMIUM_THRESHOLD
+ subtotal >= VIP_PREMIUM_THRESHOLD

Bob preserved the named-constant refactor and changed only the comparison operator.

Hardened public validation package

The current public corpus is broader than the historical Bob clean-room run and remains separately attributed.

86 / 86
Behavioral scenarios
100
Workflow steps
231
Contract invariants
32 / 32
Verifier tests
100.00%
Legacy statements
98.96%
Legacy branches
21 / 21
Seeded regressions
READY
Submission readiness
Structural MetricLegacy MonolithModern CandidateDelta / Decoupling
Python modules111+10 modules with domain separation
Largest module LOC381205-176 LOC (-46.19%)
Classes / types68Typed domain models and errors
Imports from legacy—00 modern imports from legacy
Dependency cycles000 cycles
The hardened acceptance path reads the existing sealed baseline rather than regenerating it. GitHub Actions also checks that verification leaves the protected baseline and tracked evidence unchanged.

IBM Bob clean-room modernization run

The historical 47-scenario corpus proves the modernization lifecycle itself was genuinely executed in IBM Bob.

EVIDENCE CORPUS SEPARATION

Historical IBM Bob clean-room run: 47 scenarios / 60 workflow steps / 183 invariant declarations.
Hardened public package: 86 scenarios / 100 workflow steps / 231 invariants.

They use distinct contracts, baselines, and fingerprints. Their metrics are never combined into synthetic totals.
Task 1 · Bob
Contract archaeology

Bob investigated the legacy system and produced the clean-room behavioral contract and evidence. Historical result: 47 scenarios, 60 workflow steps, 183 declared invariants.

Task 2 · Bob
Modernization architecture

Bob mapped the legacy monolith into a modular target architecture while preserving the behavioral contract as the acceptance boundary.

Task 3 · Bob
Modernization implementation

Bob implemented the candidate. The historical clean-room candidate reached 47 / 47 ACCEPTED.

Safety rehearsal
Intentional blocked state

A deliberate >= → > boundary regression was injected. Aegis returned 46 / 47 BLOCKED with a +$32.18 behavioral drift.

Task 4 · Bob
Drift critic and repair

Bob diagnosed from preserved evidence and applied the one-line repair, restoring 47 / 47 ACCEPTED.

Task 5 · Bob
Final evidence review

Bob reviewed the preserved clean-room evidence while Aegis retained the independent acceptance role.

Who verifies the verifier?

Aegis validates its own sensitivity through verifier tests, seeded negative controls, and separate post-freeze adversarial challenges.

Verifier self-validation

32 / 32 tests

The hardened verifier includes fail-closed contract parsing, relative-import analysis, non-vacuous mutation validation, and sealed-baseline immutability checks.

Curated gauntlet

21 / 21 detected

Seeded semantic regressions test whether the contract can actually detect meaningful behavioral drift.

Broader holdout

54 / 63 runnable

Nine survivors remain disclosed. The frozen contract was not backfitted after observing them.

PR-specific targeted set

24 / 24 runnable mutations blocked

  • 4 inclusive/exclusive boundary attacks
  • 8 threshold shifts by ±$0.01
  • 10 discount-rate mutations
  • 2 tier-routing corruptions
  • 0 survivors, 0 invalid, 0 timeout
100% detection within this finite PR-specific mutation set. This is not an exhaustive mutation score.
Historical generated holdout

85.71% over runnable mutants

  • 65 generated
  • 63 runnable
  • 54 detected
  • 9 survivors disclosed
  • 2 invalid / unrunnable
  • 0 timeout / infrastructure failures

Human governance is a first-class artifact

Aegis is designed to turn raw verification into inspectable evidence that a human reviewer can use before merge.

Evidence certificate

Cryptographic audit artifact

The evidence certificate binds recorded verification artifacts and semantic fingerprints so reviewers can detect evidence changes.

Executive dashboard

At-a-glance evidence review

The standalone dashboard surfaces contract coverage, verification status, architecture evidence, and readiness without requiring a reviewer to inspect every raw file.

Human merge authority

MERGE_ELIGIBLE ≠ auto-merge

Bob may return as a read-only evidence reviewer. Aegis may mark the evidence eligible. The final merge decision remains explicitly human.

IBM Bob usage, provenance & fingerprints

The repository preserves genuine Bob task captures and keeps integrity fingerprints separate from claims of correctness or authorship.

Captured IBM Bob usage

Tasks 1–5 clean-room Bobcoins

Task 1: 4.67 · Task 2: 2.15 · Task 3: 5.08 · Task 4: 2.25 · Task 5: 3.15

Captured total: 17.30 Bobcoins. Values come from preserved IBM Bob UI evidence and are not recomputed by Aegis.
Preserved Bob limitations

Subagent transparency

  • Task 4 diagnostic subagents A and C returned malformed/garbled output; subagent B succeeded; the primary Bob agent independently re-derived the diagnosis.
  • Task 5 review subagent B returned malformed output; the primary agent audited provenance directly.
  • Unanimous subagent agreement is not claimed.
CorpusEvidence DescriptorSHA-256 / IdentifierStatus
Hardened PublicBaseline Semantic Evidence Fingerprint2fd2c2f96614e5bcf8fa807b5c1b777fc702f53707be047ddf01c5a852680da7Sealed
Hardened PublicAccepted Candidate Semantic Evidence Fingerprintd919b4055c09e30ee9654d92e302aa64c46195716d53525d57023e1430216c99Accepted
Bob Clean-RoomHistorical Baseline Semantic Fingerprint2d9d6ef6b3e1d972e0cf68c504083f239391e1becb939c6726fabdad6d906db5Preserved
Bob Clean-RoomHistorical Accepted Candidate Fingerprintf6c79144ab8592e71b459be6729ff774740922c15207397c75950773a4e321cfPreserved
Bob Clean-RoomHistorical Blocked Rehearsal Fingerprintd569d131fdbe2f23dd9943431f72c435138fa06f4cab4be5bf3959f43a5fd654Blocked
Key frozen references
  • Risky PR commit: 5276fa24a3f7c9e8025fe22d9995ea528fe2691e
  • Bob repair commit: fabc7b7a3a960fd1df7b7c6e51c44a27f797f95a
  • Repair freeze tag: aegis-pr-repair-freeze-20260927
  • Human merge commit: 1f79baf
  • Hardened submission tag: aegis-submission-final-hardened-20260927
Scope qualification
  • No formal proof of complete program equivalence.
  • No claim that the legacy business policy itself is correct.
  • No behavioral claim outside the defined contract and executed scenario corpus.
  • No guarantee that every future regression will be detected.
  • No hostile-code sandbox guarantee.
  • SHA-256 fingerprints establish integrity identity, not correctness or authorship.
Submission thesis

Generation is not acceptance.

IBM Bob modernizes legacy systems. Aegis reconstructs and seals the reference behavior, independently detects semantic drift, drives evidence-based repair, and gates the final human merge decision.