Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question: How to use TypeScript interfaces with matchers #1054

Closed
agross opened this issue Feb 2, 2023 · 14 comments · Fixed by #1061
Closed

Question: How to use TypeScript interfaces with matchers #1054

agross opened this issue Feb 2, 2023 · 14 comments · Fixed by #1061
Labels
bug Indicates an unexpected problem or unintended behavior

Comments

@agross
Copy link

agross commented Feb 2, 2023

Hello,

this is a question rather than a bug report or feature request.

I'm quite new to Pact and I would like to reuse the TypeScript interfaces my frontend app uses for the consumer-side of things.

The documentation about matching states that in order to use TypeScript interfaces inside e.g. willRespondWith/body you need to wrap them in InterfaceToTemplate or use type instead of interface.

I've tried both but the results where not quite satisfactory.

As far as I understand it would be beneficial to use existing types and interfaces to prevent typos and to define the correct expected value types for the mock server.

On the other hand, Matcher.<something> seems incompatible with the types expected by interface members. The example from the docs only uses the like matcher around the interface, but not for its members.

My question is how I would be able to achieve something like this with Pact:

interface Foo {
  a: string;
}

const f: InterfaceToTemplate<Foo> = { a: like("working example") };

// The above causes:
// Type 'Matcher<"working example">' is not assignable to type 'AnyTemplate'.
//    Type 'Matcher<"working example">' is not assignable to type 'TemplateMap'.
//       Index signature for type 'string' is missing in type 'Matcher<"working example">'.

provider.addInteraction({
  uponReceiving: "a post with foo",
  withRequest: {
    method: "POST",
    path: "/",
    body: like(f)
  },
  ...
})

The only thing that comes to mind is to slap as unknown as string after the matcher.

Software versions

  • OS: macOS Ventura
  • Consumer Pact library: Pact JS 10.4.1
  • Node Version: v16.19.0
@agross agross added the bug Indicates an unexpected problem or unintended behavior label Feb 2, 2023
@TimothyJones
Copy link
Contributor

Firstly apologies for this, these types probably shouldn't have been released in that state, which is my fault. It's one of the last things I did as a maintainer -I bought them in as an experiment on the beta branch, and I forgot to highlight this with anyone before I stepped down. My failure to communicate this meant that the current maintainers (quite reasonably) didn't know about the potential problems with this type when 10.x was released, and I forgot all about it.

(As an aside, I can't reproduce this with Pact 10.4.1 and Typescript 4.9.4 - so changing the typescript version to match might fix it. However, the problem is in Pact, not in your code)

I've ended up writing a long answer - the tl;dr is:

tl;dr

As a workaround, don't use InterfaceToTemplate and instead just spread the Foo object when you pass it to like - body: like({...f}).

Note that the like matcher cascades, so:

like({a: "working example"})

is the same as

like({a: like("working example")})

Read on for all the gory details.


Explanation

The problem stems from trying to bring in a type to represent arbitrary JSON (the motivation was to avoid mistakes like people putting functions or Date objects into their pact expectations). Unfortunately, this isn't possible in Typescript.

The problem is that typescript thinks (correctly) that the narrower interface definition isn't arbitrarily indexable (that is, Foo is explicitly not arbitrary json - if you have a Foo, you can't index it with f['someOtherProp']).

Why did I think forcing arbitrarily indexable objects was an acceptable restriction? Well, I reasoned that the type on the wire (what you're defining in the withRequest section) isn't actually a Foo, it's actually just JSON. Perhaps it later unmarshalls to a Foo, but at the time you're defining the expectation for Pact, it's just JSON with the same structure as a Foo. I thought these types would encourage people to be more explicit, and not put their business object types in the JSON expectations.

Unfortunately, this reasoning (although technically correct) is bad - even though the JSON body isn't technically a Foo, it's an additional layer of safety in your test to enforce the compiler to complain if the structure of your JSON expectation doesn't match Foo. This extra layer of safety is what you're doing in your example.

Your example is a good practice, and these misguided types are preventing you from doing it.


Workarounds

It will work if you use type instead of interface, but that's more of a hack that blinds typescript, rather than the right approach. I don't think Pact should require you to write your types a particular way. So, we want a workaround that doesn't require you to change your types.

Here are two workarounds that don't require you to change your type system. They also blind typescript to the problem, but still let you benefit from compile errors if your Pact expectation deviates from Foo.

  1. You can use Object.freeze to tell typescript that the object is immutable (this happens to have a similar effect to InterfaceToTemplate - or at least, how InterfaceToTemplate should have been written - see the next section for the details).
  2. You can just spread the object. I like this more as it's simpler, but I suppose the drawback is that it's less clear that something weird is going on.

I don't think it matters which you choose.

Here's both options:

      interface Foo {
        a: string;
      }

      const f: Foo = { a: "working example" };

      provider.addInteraction({
        uponReceiving: "a post with foo",
        withRequest: {
          method: "POST",
          path: "/",

// Then either (1):
          body: like(Object.freeze(f)),
// Or (2):
          body: like({...f}),
      ...

Long term fix in Pact

Why doesn't InterfaceToTemplate<Foo> work, when the documentation says it will? Well, the definition of that type in Pact is wrong, too. It is:

export type InterfaceToTemplate<O> = { [K in keyof O]: AnyTemplate };

But it should have been:

export type InterfaceToTemplate<O> = {
    [K in keyof O]: InterfaceToTemplate<O[K]> | Matcher<O[K]>;
 };

If these template types live on, this type should be corrected.

But really, the right fix is to remove all the AnyTemplate, AnyJson, InterfaceToTemplate etc types in the next major version (and whether or not to expose explicit types for the matchers should be rethought).

These types were a mistake, and I'm very sorry.

@mefellows, let me know if you would like a PR that corrects this.

@agross
Copy link
Author

agross commented Feb 4, 2023

Hello @TimothyJones,

thank you very much for your detailed answer! There are a couple of follow-up questions that came up while trying your suggestions.

Nested objects

The interaction I want to specify should return an object that contains other objects. This seems to be the reason your suggestions do not work. On top of that, the interaction should return an array of such objects.

interface Foo {
  a: string;
}

const f: Foo = { a: 'working example' };

Matchers.like({ ...f });
Matchers.like(Object.freeze(f));
Matchers.atLeastLike({ ...f }, 1);
Matchers.atLeastLike(Object.freeze(f), 1);

interface Room {
  id: string,
  foo: Foo
}

const r: Room = { id: 'some guid', foo: { a: "example" }};

Matchers.like({ ...r });
// Argument of type '{ id: string; foo: Foo; }' is not assignable to parameter of type 'AnyTemplate'.
//  Type '{ id: string; foo: Foo; }' is not assignable to type 'null'.

Matchers.like(Object.freeze(r));
// Argument of type 'Readonly<Room>' is not assignable to parameter of type 'AnyTemplate'.
//   Type 'Readonly<Room>' is not assignable to type 'TemplateMap'.
//     Property 'foo' is incompatible with index signature.
//       Type 'Foo' is not assignable to type 'string | number | boolean | JsonArray | JsonMap | TemplateMap | TemplateArray | Matcher<unknown> | null'.
//         Type 'Foo' is not assignable to type 'TemplateMap'.
//           Index signature for type 'string' is missing in type 'Foo'.

Matchers.atLeastLike({ ...r }, 1);
// Argument of type '{ id: string; foo: Foo; }' is not assignable to parameter of type 'AnyTemplate'.
//   Type '{ id: string; foo: Foo; }' is not assignable to type 'null'.

Matchers.atLeastLike(Object.freeze(r), 1);
// Argument of type 'Readonly<Room>' is not assignable to parameter of type 'AnyTemplate'.

How to use anything beyond like for specific members?

Let's say I want to be specific about the format of a particular interface member. As far as I understand like checks for the type, but the data format does not matter. Let's pretend Room.id should be a UUID encoded as a string.

const r: Room = { 
  id: Matchers.uuid(),
  // Type 'RegexMatcher' is not assignable to type 'string'.

  foo: { a: "example" }
};

So in order to use this, Matchers.uuid() needs to return something that looks like a string, but it is a RegexMatcher. I'm not a TypeScript expert enough to know whether TypeScrript supports something like implicit conversions (C# does).

PS: I'm using the same TypeScript and Pact version as you.

@agross
Copy link
Author

agross commented Feb 4, 2023

Just found a suggestion on Stack Overflow: https://stackoverflow.com/a/67406755/149264

Adjusted to the current Pact version (where Matchers.MatcherResult does not exist) this seems to fix the example above.

type WrappedPact<T> = T extends object
  ? {
      [K in keyof T]: WrappedPact<T[K]>;
    }
  : T | Matchers.Matcher<T>;

interface Foo {
  a: string;
}

const f: WrappedPact<Foo> = { a: 'working example' };

Matchers.like(f);
Matchers.atLeastLike(f, 1);

interface Room {
  id: string;
  foo: Foo;
}

const r: WrappedPact<Room> = { id: 'some guid', foo: { a: 'example' } };

Matchers.like(r);
Matchers.atLeastLike(r, 1);

@agross
Copy link
Author

agross commented Feb 4, 2023

Even nested specifications work!

const r: WrappedPact<Room> = { id: Matchers.uuid(), foo: { a: Matchers.regex(/some/, 'example') } };

I think something like this should be included in the distribution.

@TimothyJones
Copy link
Contributor

Thanks for the update and extra cases. I'll take a look at a PR to fix all this up first thing this week.

The nested example unfortunately is expected with that workaround - you have to spread or freeze each object , like Matchers.like({...r, foo: {...r.foo} }).

It looks like WrappedPact will solve the specific problems raised so far, but at a quick glance, I don't think it is a general solution. For example, arrays will pass T extends object, and then would get their function properties mapped (although perhaps that won't matter in practice). I think it would still be a good starting point, though, so thanks very much for the tip.

TimothyJones added a commit to TimothyJones/pact-js that referenced this issue Feb 16, 2023
…as the side effect that functions, Dates and other inappropriate types can now be passed to matchers, and the benefit that people using interfaces don't get spurious errors. Fixes pact-foundation#1054
TimothyJones added a commit to TimothyJones/pact-js that referenced this issue Feb 16, 2023
@TimothyJones TimothyJones mentioned this issue Feb 16, 2023
3 tasks
@TimothyJones
Copy link
Contributor

I've raised a PR that fixes this by removing the restriction that V3 matchers must be passed AnyTemplate (and deprecates AnyTemplate). This approach is nice because it's not a breaking change.

@seyfer
Copy link

seyfer commented Mar 22, 2023

@agross , @TimothyJones I have trouble here with MatchersV3.boolean

Given an interface like

interface Foo {
  bar: boolean;
}

and the value such as {bar: true} or {bar: false}, when trying to use with spread operator and the Typescript type suggested above

const matchedValues: PactMatchable<Foo[]> = values.map(
        (w) => {
          return {
            ...w,
            bar: Matchers.boolean(),
          };
        }
      );

or with an example value bar: Matchers.boolean(w.bar),

I get a typescript error

Type '{ bar: Matchers.Matcher<boolean>; }' is not assignable to type '{ bar: boolean | Matcher<false> | Matcher<true>; }'.
    Types of property 'bar' are incompatible.
      Type 'Matcher<boolean>' is not assignable to type 'boolean | Matcher<false> | Matcher<true>'.
        Type 'Matcher<boolean>' is not assignable to type 'Matcher<false>'.
          Type 'boolean' is not assignable to type 'false'.

Why does Matcher<boolean> is getting transformed to Matcher<false> | Matcher<true> ?

@TimothyJones
Copy link
Contributor

Where is the PactMatchable type coming from? I don't think that's exported from Pact.

@agross
Copy link
Author

agross commented Mar 22, 2023

It's the same as WrappedPact above.

@TimothyJones
Copy link
Contributor

Ah, I see! So, that type shouldn't be necessary after 11.x was released.

But, if you want to use it, you can correct the definition like so:

type PactMatchable<T> = [T] extends [object]
  ? {
      [K in keyof T]: PactMatchable<T[K]>;
    }
  : T | Matcher<T>;

For the explanation, have a read of the Typescript documentation on distributive conditional types.

I'm not even certain that the corrected version works for all use cases - but it will at least work for the case in your comment.

@seyfer
Copy link

seyfer commented Mar 22, 2023

@TimothyJones yes now in v11 everything works without that wrapper interface. Thank you!

@georgiosgiatsidis
Copy link

Ah, I see! So, that type shouldn't be necessary after 11.x was released.

But, if you want to use it, you can correct the definition like so:

type PactMatchable<T> = [T] extends [object]
  ? {
      [K in keyof T]: PactMatchable<T[K]>;
    }
  : T | Matcher<T>;

For the explanation, have a read of the Typescript documentation on distributive conditional types.

I'm not even certain that the corrected version works for all use cases - but it will at least work for the case in your comment.

type PactMatchable<T> = [T] extends [(infer U)[]]
    ? MinLikeMatcher<PactMatchable<U>[]>
    : [T] extends [object]
      ? {
            [K in keyof T]: PactMatchable<T[K]>;
        }
      : Matcher<T>;

I extended your solution to support eachLikecases, ensuring it can handle cases where the type is an array and needs to be wrapped in MinLikeMatcher to properly represent the structure and matchers within the array elements.

@TimothyJones
Copy link
Contributor

TimothyJones commented Nov 29, 2024

I think that will fail with empty arrays - those shouldn't be wrapped in MinLike. It will also fail where you really do want the raw array and don't want to generalise it with an eachLike.

For empty arrays, U would infer as never, and so you could special case it. However, that wouldn't address situations where you want exact arrays in the response. You could special case those too, by checking to see if the object extends an eachlike matcher, but it becomes complex very quickly (have a look at this example from type-fest to see the kind of complexity I mean).

I think the core of the problem is that typescript isn't designed to have a type for "everything that could be assigned as json data" - and what we want here is to extend that (already uncomfortable) type with an additional constraint.

For specific projects, a custom wrapper like this might be ok - because you can get close to a type for arbitrary json if you're able to constrain how the type is used. But, I don't think the library can safely expose a type like this to all users, as there will always be edge cases.

If you really need it, I'd recommend something like the approach used in type-fest linked above - although I would guess there are still cases it can't cover.

Is there a reason you need this type in a recent version of pact?

@georgiosgiatsidis
Copy link

I think that will fail with empty arrays - those shouldn't be wrapped in MinLike. It will also fail where you really do want the raw array and don't want to generalise it with an eachLike.

For empty arrays, U would infer as never, and so you could special case it. However, that wouldn't address situations where you want exact arrays in the response. You could special case those too, by checking to see if the object extends an eachlike matcher, but it becomes complex very quickly (have a look at this example from type-fest to see the kind of complexity I mean).

I think the core of the problem is that typescript isn't designed to have a type for "everything that could be assigned as json data" - and what we want here is to extend that (already uncomfortable) type with an additional constraint.

For specific projects, a custom wrapper like this might be ok - because you can get close to a type for arbitrary json if you're able to constrain how the type is used. But, I don't think the library can safely expose a type like this to all users, as there will always be edge cases.

If you really need it, I'd recommend something like the approach used in type-fest linked above - although I would guess there are still cases it can't cover.

Is there a reason you need this type in a recent version of pact?

You're absolutely right that this can quickly become quite complex.
Just yesterday, I ran into issues dealing with nullable objects, which inreased the complexity of the typing.

In my case, whenever I encounter arrays in an interaction, I always use eachLike. This keeps things manageable in the short term.

The primary reason I’m typing these interactions is to ensure they conform to my DTOs. This helps catch issues like typos, missing properties, or mismatches early in the process.

Here’s a quick example

export function wrapResponse<T>(data: PactMatchable<T>) {
    return { data };
}

pactProvider
    .given(Given.POSTS_EXIST)
    .uponReceiving('a request for a dummy entity')
    .withRequest({
        method: 'GET',
        path: `/${LATEST_API_VERSION}/posts`,
        headers: { ...sharedHeaders },
    })
    .willRespondWith({
        status: 200,
        headers: { 'Content-Type': 'application/json' },
        body: wrapResponse<PostsResponseDTO>(
            MatchersV3.eachLike({
                id: MatchersV3.string(MOCK_ID),
                title: MatchersV3.string(MOCK_TITLE),
                slug: MatchersV3.string(MOCK_SLUG),
                startDate: MatchersV3.timestamp("yyyy-MM-dd'T'HH:mm:ssX", MOCK_STARTDATE),
                // ... rest properties, support for primitives, nested objects, arrays
            }),
        ),
    });

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Indicates an unexpected problem or unintended behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants