Stream: interviews

Topic: 649: Lightspeed search built for devs

Logbot (Jul 10 2025 at 12:19):

We talk with Don MacKinnon, Co-founder and CTO of Searchcraft—a lightspeed search engine built in Rust. We dig into the future of search, how it blends vector embeddings with classic ranking, and what it takes to build developer-friendly, production-grade search from the ground up. :link: https://changelog.fm/649

Siddhartha Golu (Jul 10 2025 at 13:47):

Excited for this one!

Don MacKinnon (Jul 10 2025 at 22:15):

I didn't notice at the time but Adam has Silicon Valley playing in the background behind him :joy:

Chris Duzan (Jul 12 2025 at 16:00):

Great show! Not quite finished with it but it got me thinking about a lot of things. Particularly when you guys got on the AI copyright topic.

I was driving so I couldn't write down my thoughts at the time, but 2 are sticking in my mind.

Chris Duzan (Jul 12 2025 at 16:05):

1. Are writers and illustrators going to become very niche? I don't know the exact term, but I've come across some people who have specialized in old "craftsmanship" professions (eg. fletching, armor smithing, etc). Will this be the case for writers and illustrators where it won't be a profession anymore. But there are highly specialized cases where a small number of people make a living off of them?

Chris Duzan (Jul 12 2025 at 16:08):

2. The mention of libraries when it comes to having the ability to share books. If you've ever used digital downloads for a library, you'll know that they can't give out infinite copies of a digital book. They are restricted to a certain number that can be "checked out" at a time.

Can this be replicated with model training? I don't know what this would actually look like under the hood. But something a long the lines of "you can only work on one instance of this data at a time." And you would have to buy more "licenses" to train on the same data concurrently.

If we are working off the analogy that someone learning from a piece of work is the same as a model being trained on a piece of work, I think you could also make the argument that models training on it are more like multiple people training at the same time. So each "person" would need a copy of the work.

Chris Duzan (Jul 12 2025 at 16:16):

Still, maybe it's not a helpful analogy because these things "learn" on a different timeframe than humans.

Ron Waldon-Howe (Jul 12 2025 at 21:32):

@Chris Duzan I think both of these things are solved by abolishing the concept of Intellectual Property (OpenAI and Meta already behave as though this concept protects yet does not restrain them), and some sort of Universal Basic Income or similar so that any individual that previously relied on IP to feed their families will be able to continue to do so whilst learning whatever the new valuable skills are

Ron Waldon-Howe (Jul 12 2025 at 21:34):

But yes, "artisanal" comes to mind: artisanal poetry, artisanal illustrations, etc
Some folks with means may decide they are okay with paying more for this, just like for artisanal furniture, etc

Tim Uckun (Jul 12 2025 at 23:07):

I honestly think UBI will not work. Any extra money in the pockets of consumers will quickly flow upstream to the landlords, corporations, etc.

A better alternative would be to provide people with free food, shelter and healthcare. This doesn't have to be anything fancy, minimal amount required to keep a person alive and relatively healthy. I am imagining something like dormitories or tiny apartments like those found in Tokyo and Hong Kong. With meals provided as MRE or freeze dried meals like the ones you take camping. There would be building and floor supervisors who would be responsible for making sure people don't abuse the system by using their quarters for storage or being destructive and whatnot.

This would provide a downward pressure on housing, food, and energy costs as the private sector has to deal with the release valve of government provided basic services. At the same time although these conditions are sufficient to keep somebody alive they are not pleasant and would continue to encourage people to seek employment for better living conditions.

Of course if they let me design society the conditions would improve as productivity from automation goes up and who knows maybe some day everybody will get a three bedroom house with a yard.

Andrew O'Brien (Jul 13 2025 at 00:08):

Please tell me Vectron is a Mitchell & Webb reference: https://youtu.be/icTrzUuWlHI?si=ZwcoflFJ-VTJ2tC8

Don MacKinnon (Jul 14 2025 at 19:45):

Andrew O'Brien said:

Please tell me Vectron is a Mitchell & Webb reference: https://youtu.be/icTrzUuWlHI?si=ZwcoflFJ-VTJ2tC8

It would be pretty cool if it was but alas no. We thought we made it up :smiley:

Daniel Buckmaster (Jul 15 2025 at 00:47):

Well hopefully this lightens things up around the office, by Vectron's mighty gaze!

Andrew O'Brien (Jul 15 2025 at 04:39):

@Don MacKinnon May Vectron smile upon you anyway and carry your business aloft on his golden pinions!

James McNally (Jul 15 2025 at 07:24):

You are helping me on the AI moral question in processing my thoughts on it.

You touched on scale and I think that is the key departure here. Similar to issues around privacy.

By the basic concept, folks have published in public for various reasons assuming there is a limit to what can be done reworking that information that means they still see some benefit. But AI changes that scale.

Probably the law needs revisiting with this scale in mind to get back to the intention of copyright, allowing a producer some control over the use of their work

Not a simple task!

Don MacKinnon (Jul 15 2025 at 17:58):

Daniel Buckmaster said:

Well hopefully this lightens things up around the office, by Vectron's mighty gaze!

We loved the clip, now I need to go check out the rest of the show.

Andrew O'Brien (Jul 18 2025 at 14:45):

@Don MacKinnon it’s well worth it.

I’d also highly recommend Peep Show, which is the same guys (plus a younger, now Oscar winning Olivia Colman) in a sitcom that’s sort of an Odd Couple except shot from alternating 1st person perspectives where you hear their inner monologues.

Tim Uckun (Jul 18 2025 at 22:32):

You know sometimes I watch shows from the 60s and 70s and it's amazing how much more "blue" they are than shows today. Something happened in the 80s or 90s which turned television and movies much more prudish and conservative. There is a marked shift from using sexual innuendo and sexy outfits to violence and carnage in order to hold the audience attention. Back then you'd never even see blood on television and movies let alone the kinds of gruesome deaths, spurting blood, mass casualties, non stop explosions etc.

We have become less interested in pleasure and more interested in violence I guess.

Andrew O'Brien (Jul 19 2025 at 00:46):

Yeah, I mean I always feel slightly awkward going from something pretty clean to more mature content with someone who I don’t know what their tolerance is without putting some kind of warning in, especially in a semi-professional space like this.

And I wouldn’t even say it’s that extreme. It’s like Letterkenny, Always Sunny in Philadelphia, or an HBO comedy here. But not HR friendly.

Tim Uckun (Jul 19 2025 at 05:15):

I think it's interesting people consider a double entendre mature content but not somebody getting shot or stabbed or being bit by a vampire.

Ron Waldon-Howe (Jul 19 2025 at 06:39):

@Tim Uckun I think that's cultural
In Europe, parents are more concerned about exposing children to violence
But in the USA, they've given up protecting children from violence (Sandyhook is proof), so they seem to care most about exposing children to sexual content

Jerod Santo (Jul 19 2025 at 13:44):

Overgeneralize much?

Ron Waldon-Howe (Jul 19 2025 at 22:49):

Generalise, sure
But there doesn't seem to be any action to prevent gun violence in schools in the USA, security theatre, but no actual reform
Versus literal book burnings by anti-LGBT folks in the name of protecting children from supposed inappropriate content

Andrew O'Brien (Jul 19 2025 at 23:52):

Look, all’s I was trying to do was prevent an embarrassing work situation. I don’t know what kinds of fuckwords and depictions of crack smoking are permissible at everyone’s places of business.

Tim Uckun (Jul 20 2025 at 04:38):

https://en.wikipedia.org/wiki/%27No_Way_to_Prevent_This,%27_Says_Only_Nation_Where_This_Regularly_Happens

Alex Barnes (Jul 20 2025 at 14:17):

Andrew O'Brien said:

Look, all’s I was trying to do was prevent an embarrassing work situation. I don’t know what kinds of fuckwords and depictions of crack smoking are permissible at everyone’s places of business.

Chance would be a fine thing!

Don MacKinnon (Jul 23 2025 at 16:54):

I had some folks pop into the Searchcraft discord asking about Sled after the episode dropped. Here's a talk that Tyler (aka Spacejam) gave about Sled and io_uring back in 2020 that folks may find interesting https://archive.fosdem.org/2020/schedule/event/rust_techniques_sled/

Last updated: Aug 18 2025 at 01:38 UTC