Ancestors

Toot

Written by Pamela Fox on 2024-12-06 at 23:02

I had a lot of tricky tests to write today that needed mocks against an async SDK.

To get started, I tried out GitHub Copilot "generate tests".

Observations:

• Copilot really liked using MagicMock, whereas I generally construct the actual class and use monkeypatch.setattr to mock the methods used.
• Copilot's tests did often pass, after a few tweaks.
• When a test didn't pass, I found it harder to debug MagicMock mocks, because they mocked too much- I wasn't getting errors for undefined attrs.

=> More informations about this toot | More toots from pamelafox@fosstodon.org

Descendants

Written by Pamela Fox on 2024-12-06 at 23:03

So I eventually modified most tests to "my style" with monkeypatch.setattr whenever possible.

I'm still grateful for Copilot's help since:

1. I've now learnt more about how to use MagicMock, for when I need it

2. sometimes it's less intimidating to edit an existing failed test than to write a brand new one

=> More informations about this toot | More toots from pamelafox@fosstodon.org

Written by Pamela Fox on 2024-12-06 at 23:03

I'm curious what style mocks y'all prefer! should i be magic mocking more??

=> More informations about this toot | More toots from pamelafox@fosstodon.org

Written by jonny (good kind) on 2024-12-06 at 23:09

@pamelafox

I definitely prefer custom mocks in most cases - magic mocks work when everything is totally unit-ized and so you want to assume nothing about the components surrounding the thing to be tested, but in practice, if you can't actually use the thing in testing, it's good to have at least some imitation of its real behavior, because otherwise you're usually testing nothing

=> More informations about this toot | More toots from jonny@neuromatch.social

Written by jonny (good kind) on 2024-12-06 at 23:11

@pamelafox

I mean I assume you know more than me, but since you asked!

Also im not saying to not magic mock, it's just for a different situation than the ones I usually run into where I need a mock. More a "when I write tests I usually write the mocks" than a "dont use magicmock"

=> More informations about this toot | More toots from jonny@neuromatch.social

Written by Pamela Fox on 2024-12-06 at 23:18

@jonny Okay yeah that makes sense. I think I prefer tests that aren't as clean in terms of "unit-ization" since that gives me additional opportunities to discover issues? Also better type hinting in my tests, since you cant get type hints from Magic mocks. Thanks for sharing your take!

=> More informations about this toot | More toots from pamelafox@fosstodon.org

Written by Josh Rivers on 2024-12-07 at 18:16

@pamelafox I’ve done a lot of mocking in Python and other langs and I am changing my practice. Beyond specific types of fault injection or obscure branch behavior checks, I want to be using fakes. A lot of rework in my past could have been avoided if I wrote my tests as integration tests against multiple modules rather than unit tests a function at a time. I am now trying to enable that through accurate behavior fakes at the right levels of my applications so the tests work against a “real” flow of the code (like using moto instead of MagicMock). The Google SDEV handbook has a good section on testing that speaks to this philosophy (it’s a little over-declarative and chest thumping, and written for a Java shop, but the discussion has a lot of insights)

=> More informations about this toot | More toots from joshrivers@techhub.social

Written by Tomasz Tarczyński on 2024-12-07 at 10:13

@pamelafox working in a different tech stack (Typescript/nodejs) my observation is that without giving an example, the tests generated by Copilot were mostly garbage. After writing a few tests myself, and then asking Copilot to generate other tests based on what I've written, they were actually pretty good, and a few tweaks later passed

=> More informations about this toot | More toots from tarczynski@fosstodon.org