How fitness functions can help us govern and measure AI

Thoughtworks Technology Podcast

Thoughtworks

Careers, Business, 907234, Technology

4.7 • 53 Ratings

🗓️ 6 March 2025

⏱️ 42 minutes

🔗️ Recording | iTunes | RSS

🧾️ Download transcript

Summary

AI is inherently dynamic: that's true in terms of the field itself, and at a much lower level too — models are trained on new data and algorithms adapt and change to new circumstances and information. That's part of its power and what makes it so exciting, but from a business and organizational perspective, that can make governance and measurement exceptionally difficult. How can we know that our AI is optimized for the right thing? How can we be sure it's oriented towards what we want it to be?

This is where the concept of fitness functions can help. Broadly speaking, fitness functions are ways of measuring the extent to which a given solution is fulfilling its goals — so, in the context of AI, they can help teams ensure that AI systems are serving their intended purpose.

In this episode of the Technology Podcast, Rebecca Parsons and Neal Ford — authors (alongside Pat Kua and Pramod Sadalage) of Building Evolutionary Architectures, the book which brought fitness functions into the software architecture space — join host Ken Mugrage to explore how the fitness function concept can help us better manage the dynamism of AI and, in doing so, overcome the challenge of bringing such systems into production.

Learn more about Building Evolutionary Architectures: https://www.thoughtworks.com/insights/books/building-evolutionaryarchitectures-second-edition

Transcript

Click on a timestamp to play from that location

0:00.0	Welcome, everyone to another edition of the ThoughtWorks Technology podcast.
0:12.3	My name is Ken McGrage.
0:13.5	I'm one of your regular hosts.
0:15.6	A little bit of an interesting panel today.
0:17.6	We have both hosts and guests at the same time.
0:21.6	I'll let them introduce themselves.
0:23.6	First, Dr. Rebecca Parsons.
0:25.6	Hello everybody.
0:26.6	Rebecca Parsons, former CTO emerita of ThoughtWorks.
0:31.6	I have been co-host for many of these podcasts, but in this particular case, I'm the guests to talk to
0:39.5	about fitness functions. And Neil Ford. Again, I'm another regular host of the podcast, but
0:47.0	mostly here as a dual duty of both co-host and also guest, because Rebecca and I co-wrote both editions of the Building
0:56.8	Evolution Architecture book, which defines this idea of fitness functions. And one of the
1:01.1	frequent questions we get in the current day and age, of course, because no one can talk about
1:05.3	anything except generative AI in the technology space is, well, how does this concept of fitness
1:10.5	functions apply to AI?
1:13.1	Yeah. So for the listeners, it's kind of interesting. We were just talking to the three of us
1:17.0	before we started recording that this is something we've been talking about for a couple of years.
1:20.9	Obviously, AI has been out for decades, but generative AI really kicked it off. And we're like,
1:26.2	you know, okay, how do you test these things?
1:28.2	You have, you have bias.
1:29.1	You have the illities, which I'm sure one of these, our guests will get into a little bit.
	...

Transcript will be available on the free plan in -23 days. Upgrade to see the full transcript now.

Previous episode | Next episode

Disclaimer: The podcast and artwork embedded on this page are from Thoughtworks, and are the property of its owner and not affiliated with or endorsed by Tapesearch.

Generated transcripts are the property of Thoughtworks and are distributed freely under the Fair Use doctrine. Transcripts generated by Tapesearch are not guaranteed to be accurate.