NewsLab
Aug 28 13:43 UTC

Show HN: RealDiff – runtime behavior diffing for pull requests (six languages) (github.com)

39 points|by issacnitin||9 comments|Read full story on github.com

Comments (9)

9 shown
  1. 1. vlovich123||context
    I’ve read the readme a few times and still have no idea how this is useful. The motivating example of Sort => SortBy has no description of what’s highlighted and no description of how. Is it running the old test suite with the new changes as a weak form of mutation testing? Something else?
  2. 2. issacnitin||context
    Not mutation testing — nothing is mutated. It builds both branches with instrumentation, runs the same test suite on each, and diffs the recorded arguments and return values. In that example, selectDiscount returns a different discount and the order total goes 85 → 60. Neither function is in the diff; only the sort helper is. The tests still pass — one asserts the total is positive, and 60 is positive.
  3. 3. gardnr||context
    Don’t run your responses through an LLM. People prefer to read your genuine words, even if it may include some incorrect English.
  4. 4. issacnitin||context
    Sure, appreciate the feedback.
  5. 5. ramon156||context
    please try speaking in your own words. i still have no idea what this is for. is it two compare two impls? is it to "optimize" a function (i assume not since there's no mutation)?
  6. 6. issacnitin||context
    hi, this is simply put, to compare the behavior diff of two builds. when i say behavior, i mean the return values of functions etc. it'll help with AI-assisted code reviews etc. since AI looks at the text diff only, and misses behavior diff downstream. it complements static analysis and unit tests for a stronger testing strategy.
  7. 7. jedisct1||context
    No support for Zig?
  8. 8. issacnitin||context
    Not yet. But feel free to add :)
  9. 9. johnsuder||context
    I love the concept and see the value for refactoring. There is tremendous value in knowing that two implementations are run-time equivalent.

    I just wanted to share for others that there is a configuration language call Dhall (https://dhall-lang.org/) that has this kind of guarantee as one of its nicest features. See the "Refactor Fearlessly" section. In Dhall, most behavior preserving refactors produce the exact same hash code so you can know - for an absolute fact - that a refactor is behavior neutral. The point of sharing that other language is just to legitimize that "this is a thing" i.e. it has value recognized by others.