Stream: interviews

Topic: 626: Turso is rewriting SQLite in Rust

Logbot (Jan 30 2025 at 13:00):

Glauber Costa, co-founder and CEO of Turso, joins us to discuss libSQL, Limbo, and how they're rewriting SQLite in Rust. We discuss their efforts with libSQL, the challenge of SQLite being in the public domain but not being open for contribution, their choice to rewrite everything with Limbo, how this all plays into the future of the Turso platform, how they test Limbo with Deterministic Simulation Testing (DST), and their plan to replace SQLite. :link: https://changelog.fm/626

Danny K (Jan 30 2025 at 18:48):

Didn't know Turso was trying to replace sqlite

Jerod Santo (Jan 30 2025 at 20:06):

Yessir. Started with a fork but later decided to do a full rewrite, which has born early fruit (and lots of excitement/support)

Tim Uckun (Jan 30 2025 at 20:49):

Looking forward to this one.

Ron Waldon-Howe (Jan 30 2025 at 21:59):

There's also #rust but it's currently just me talking to myself :P

Tim Uckun (Jan 31 2025 at 05:17):

@Ron Waldon-Howe I got banned from the go subreddit for basically saying go wasn't the best option for an ecommerce site. Of course people objected and I replied detailing the wonders of rails and laravel for building ecommerce sites and boom they dropped the hammer on me.

I thought I might pick up rust after that happened but I never did. I keep hearing horror stories about how steep the learning curve is.

Ron Waldon-Howe (Jan 31 2025 at 05:28):

Coming from JavaScript and Go, there was a bit of a learning curve for me, but I read the official book ( https://doc.rust-lang.org/book/ ), and then read it again 12 months later, and things just sort of clicked for me
Rust tend to ask more from developers, but in exchange, the result tends to be more reliable and predictable when you compare it to other languages
e.g. I've shipped so many data races and nil pointers into production in Go, and the compiler does not help you prevent these bugs at all

Dustin (Jan 31 2025 at 18:48):

There’s also the easier start to Rust which is cloning and using owned values everywhere. Helps to limit the extent to the compiler yelling at you. Once it works, you can try and write a solution that’s got less allocations and such.

Dustin (Jan 31 2025 at 18:48):

But for my hobby projects the easy way has often been fast enough

Ron Waldon-Howe (Jan 31 2025 at 21:39):

Yep, one doesn't need to learn Rust all at once
.clone() everywhere is a great way to avoid borrowing, thus deferring having to learn about the borrow-checker
The other aspects of the type system like enums (the best I've experienced in any language) can be a bigger part of representing your problem in Rust and help the compiler stop you from making mistakes

Ron Waldon-Howe (Jan 31 2025 at 21:40):

For me and my use cases, it's almost always worth prioritising correctness rather than performance

Ron Waldon-Howe (Feb 01 2025 at 04:38):

Heh, just came across this, pretty fair comparison: https://bitfieldconsulting.com/posts/rust-vs-go

Ron Waldon-Howe (Feb 01 2025 at 04:39):

Excellent summary:

I hope this article has convinced you that both Rust and Go deserve your serious consideration. You should reject the false dilemma that you can only learn one or the other. In fact, the more languages you know, the more valuable you are as a software developer.

Daniel Buckmaster (Feb 01 2025 at 07:52):

TIL sqlite uses bytecode to execute its queries. It's super cool that Turso is taking the same approach and using the same bytecode language, even validating that the same SQL produces the same bytecode. https://sqlite.org/opcode.html

Daniel Buckmaster (Feb 01 2025 at 07:53):

This has sent me back into my "SQL sucks" rabbit hole, wondering if a database could use bytecode as its API instead. The sqlite format is a bit verbose - here's what a basic select looks like:

sqlite> create table tbl1 (name text, age int);
sqlite> create index foo on tbl1 (age);
sqlite> insert into tbl1 (name, age) values ('blah', 30);
sqlite> explain select * from tbl1 where age > 20;
addr  opcode         p1    p2    p3    p4             p5  comment
----  -------------  ----  ----  ----  -------------  --  -------------
0     Init           0     11    0                    0   Start at 11
1     OpenRead       0     2     0     2              0   root=2 iDb=0; tbl1
2     OpenRead       1     3     0     k(2,,)         0   root=3 iDb=0; foo
3     Integer        20    1     0                    0   r[1]=20
4     SeekGT         1     10    1     1              0   key=r[1]
5       DeferredSeek   1     0     0                    0   Move 0 to 1.rowid if needed
6       Column         0     0     2                    0   r[2]= cursor 0 column 0
7       Column         1     0     3                    0   r[3]= cursor 1 column 0
8       ResultRow      2     2     0                    0   output=r[2..3]
9     Next           1     5     0                    0
10    Halt           0     0     0                    0
11    Transaction    0     0     2     0              1   usesStmtJournal=0
12    Goto           0     1     0                    0

But if you put a little macro language on top of that assembler language for common cases... :thinking:

EDIT: ok, before I detail this episode thread, here's a fork: #unpopular-opinions > SQL is bad, databases should expose APIs instead @ 💬

Don MacKinnon (Feb 01 2025 at 19:23):

I dunno about anyone else but the relationship between Limbo, Turso and libsql was a bit confusing for the first half of the episode. For someone coming into the episode not knowing anything about Turso there wasn't much context.

Last updated: Apr 04 2025 at 01:15 UTC