Generative AI 101: Tokens, Pre-training, Fine-tuning, Reasoning — With Dylan Patel

Big Technology Podcast

Alex Kantrowitz

Technology, Religion & Spirituality, Business News, Business, Religion, Science, Philosophy, Society & Culture, Entrepreneurship, Management, Marketing, Politics, News Commentary, Government, Investing, Tech News, Social Sciences, News

4.6 • 395 Ratings

🗓️ 23 April 2025

⏱️ 41 minutes

🔗️ Recording | iTunes | RSS

🧾️ Download transcript

Summary

Dylan Patel is the founder and CEO of SemiAnalysis. He joins Big Technology Podcast to explain how generative AI work, explaining the inner workings of tokens, pre-training, fine-tuning, open source, and reasoning. We also cover DeepSeek’s efficiency breakthrough, the race to build colossal AI data centers, and what GPT-5’s hybrid training approach could unlock. Hit play for a masterclass you’ll want to send to every friend puzzled (or excited) about the future of AI. --- Enjoying Big Technology Podcast? Please rate us five stars ⭐⭐⭐⭐⭐ in your podcast app of choice. Want a discount for Big Technology on Substack? Here’s 40% off for the first year: https://tinyurl.com/bigtechnology Questions? Feedback? Write to: [email protected]

Transcript

Click on a timestamp to play from that location

0:00.0	If you've ever wondered how generative AI works and where the technology is heading,
0:04.5	this episode is for you. We're going to explain the basics of the technology and then catch
0:09.5	up with modern day advances like reasoning to help you understand exactly how it does what it does
0:14.9	and where it might advance in the future. That's coming up with Semi Analysis founder and chief
0:20.1	analyst Dylan Patel right after this.
0:23.4	Forward thinkers know that when people thrive, organizations thrive. This is the future of work.
0:29.4	This is Work Day. The AI platform that elevates humans.
0:35.8	Did you know that small and medium businesses make up 98% of the global economy,
0:42.2	but most B2B marketers still treat them as a one-size-fits-all?
0:46.9	LinkedIn's Meet the SMB report reveals why that's a missed opportunity
0:51.1	and how you can reach these fast-moving decision makers effectively.
0:55.4	Learn more at LinkedIn.com backslash meet dash the-smb.
1:03.1	Welcome to Big Technology Podcast, a show for cool-headed and nuanced conversation of the tech
1:08.5	world and beyond. We're joined today by Semi-emi Analysis founder and chief analyst Dylan Patel, a leading expert in
1:15.2	Semiconductor and Generative AI research and someone I've been looking forward to speaking
1:19.0	with for a long time.
1:21.4	Now, I want this to be an episode that A helps people learn how generative AI works, and B,
1:27.3	is an episode that people will send to their friends to explain to learn how generative AI works, and B, is an episode that people will send
1:28.5	to their friends to explain to them how generative AI works. I've had a couple of those
1:32.8	that I've been sending to my friends and colleagues and counterparts about what is going on
1:39.0	within generative AI. That includes one, this three and a half hour long video from Andre
1:43.9	Karpathy explaining everything about training large language models.
	...

Transcript will be available on the free plan in 25 days. Upgrade to see the full transcript now.

Previous episode | Next episode

Disclaimer: The podcast and artwork embedded on this page are from Alex Kantrowitz, and are the property of its owner and not affiliated with or endorsed by Tapesearch.

Generated transcripts are the property of Alex Kantrowitz and are distributed freely under the Fair Use doctrine. Transcripts generated by Tapesearch are not guaranteed to be accurate.