Support for more relaxed matching

[sazzer]

It's often useful to be able to compare the output of something with the previously generated result - e.g. taken from API specs.

However, currently Insta requires that the two are in the exact same format. That is, the fields in the same order, the same whitespace, etc.

It would be very useful if this could be relaxed so that the comparison is that the JSON is semantically the same, rather than being character-wise the same.

For example, the following are semantically the same but would fail on Insta:

{
    "a": 1,
    "b": [
        1,
        2
    ]
}

{
    "b": [1, 2],
    "a": 1
}

Cheers

[max-sixty]

to what extent does assert_json_snapshot cover this?

if not, what's the gap?

ty!

[sazzer]

This test fals:

#[test]
fn test() {
    let value = json!({
        "a": 1,
        "b": [
            1,
            2
        ]
    });

    assert_json_snapshot!(value, @r###"
    {
        "b": [1, 2],
        "a": 1
    }
    "###);
}

With the following:

running 1 test
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Snapshot: test
Source: tests/tests/modules/home/get.rs:46
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Expression: value
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────
-old snapshot
+new results
────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────
    1     1 │ {
    2       │-    "b": [1, 2],
    3       │-    "a": 1
          2 │+  "a": 1,
          3 │+  "b": [
          4 │+    1,
          5 │+    2
          6 │+  ]
    4     7 │ }
────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────
To update snapshots run `cargo insta review`
Stopped on the first failure. Run `cargo insta test` to run all snapshots.

Using Settings::sort_maps doesn't change anything. Nor does behavior.require_full_match: false in .config/insta.yml.

[max-sixty]

ah, I see the difference — we would need assert_json_snapshot!(vec_or_map), not assert_json_snapshot!(json_string)

does that help? or you would like an API like assert_json_snapshot_from_string!? I can see that being helpful in some circumstances but wouldn't think it would be worth the additional API surface without more cases...

[sazzer]

For me personally - and I'm aware that there are so many more users to consider! - the most user friendly would be to keep using assert_json_snapshot!() but have it controlled by a .config/insta.yml or a Settings value.

I will say that I'd not even considered this before now, but I've recently been doing some Go work and assert2 assert.JsonEq works this way as standard - it actually parses both sides to a map and then compares the maps, which in turn means that the whitespace and order of keys are irrelevant. That does in turn make the output harder to work with though, because you get the difference between the maps instead of between the JSON strings.

Cheers

[max-sixty]

ok! I think that much of the time the thing being tested in a rust object, and so the existing macros are sufficient. but let's leave this open and see if there are other requests for something like this

at least in the meantime, you can ofc seralize & deserialze the json to reset the format in a small helper function within a crate...

[sazzer]

That's fair.

My use case is API testing. I'm making HTTP calls to an API, deserializing the response into a serde_json::Value and then using Insta to compare that to the expected value. Which is why I mentioned the idea that the expected value might be taken from API specs.

As I say though, it's never been a problem in the past and was just a passing suggestion that might be useful to people :)

Cheers

[max-sixty]

deserializing the response into a serde_json::Value and then using Insta to compare that to the expected value

but doesn't assert_json_snapshot work with a serde_json::Value? Can you show the serde_json::Value example that doesn't work?

[sazzer]

deserializing the response into a serde_json::Value and then using Insta to compare that to the expected value

but doesn't assert_json_snapshot work with a serde_json::Value? Can you show the serde_json::Value example that doesn't work?

It works fine with serde_json::Value. It's just the whitespace and field ordering that's the issue.

Here's a full example for you:

use assert2::{assert, check};
use http::{StatusCode, header};
use insta::assert_json_snapshot;
use serde_json::Value;

use crate::service::TestService;

#[test_log::test(tokio::test)]
async fn get() {
    let sut = TestService::new().await;

    let response = sut.test_server().get("/").await;

    assert!(response.status_code() == StatusCode::OK);
    check!(response.header(header::CONTENT_TYPE) == "application/vnd.siren+json");

    let body: Value = response.json();
    assert_json_snapshot!(body, @r###"
    {
      "properties": {
        "name": "bella",
        "version": "0.1.0"
      },
      "links": [
        {
          "rel": [
            "self"
          ],
          "href": "/"
        }
      ]
    }
    "###);
}

As written, this works fine. However, the JSON in the API docs is as follows:

{
    "links": [{
        "rel": ["self"],
        "href": "/"
    }],
    "properties": {
        "name": "bella",
        "version": "0.1.0"
    }
}

Semantically, this is the same. But if I copy and paste this into the test as-is then the test will fail because the formatting is different:

And when you're auto-generating snapshots to compare against then this is fine. But when you're providing the snapshots from an external source of truth - such as the API docs in this case - then it gets a bit more awkward to make sure you've not gotten something wrong.

Cheers

[max-sixty]

ah, I see what you mean! thanks for the example.

I think that's a reasonable ask. but also this would be a big change for insta, since we treat the snapshot as opaque text (even as we treat the object-being-snapshotted as a serializable value)

nevertheless, let's leave this open