What trending tech are you most bullish on? · general

I see several technologies that are maturing at the same time. I know Silicon Valley (place not show) is all about AI at the moment. I see the value, but I've been a little more skeptical of its long-term value insofar as it changes our lives for the better.

Among the trending technologies, these ones stick out to me, but there are plenty more:

For me, I think Wasm has a huge potential to disrupt the cloud-native ecosystem. I think we'll see more companies pick it up as a model for deploying their applications safely and efficiently. However, it may be too limiting for certain use cases. I know I can't use it for my current project because I rely on a Git binary in my runtime environment. I'm still looking for the right project to use Wasm in. At work though, we enable users to run their own Wasm as part of a Kafka pipeline in process (Redpanda).

I'm even more excited about local-first applications like Linear and others. I think it offers a genuinely novel way of both developing and using apps that is way better for users. I've completely moved off of Notion and onto Obsidian for this very reason. I think we'll see a new generation of applications using local first to eat the lunch of existing juggernauts like Atlassian, Google Docs, and Notion.

Daniel Buckmaster (Sep 22 2024 at 02:40):

I'd like to co-sign your enthusiasm about both wasm and local-first software! I'm keenly following the local first ecosystem and looking for places where it would make sense in our company.

Daniel Buckmaster (Sep 22 2024 at 02:44):

It's a very long-term vision so probably not what you were thinking of for this topic, but I recently learned about Dynamicland and really love the vision of the future they have.

I've seen people (read: tech bros) get really inspired by Tony Stark's lab in the Iron Man films and think that's the future of tech. Dynamicland feels to me like that, but if it were invented by your friendly neighbourhood anarchist instead of a billionaire arms dealer :joy:

Thomas Eckert (Sep 22 2024 at 14:58):

Matthew Sanabria (Sep 22 2024 at 15:14):

I think embedded devices will become more important as they become more accessible to program. It's easier to create devices for home automation and other monitoring needs. I'd like to see a shift from proprietary embedded devices that have zero right to repair onto open hardware/software that you can extend or replace with your own devices.

Thomas Eckert (Sep 22 2024 at 20:34):

@Matthew Sanabria, I'm wondering if there is some crossover on our interests. I feel like Wasm is the perfect compilation target for an embedded device that is built to run WASI.

Lars Ellingsen (Sep 22 2024 at 21:58):

Are we talking about general audiences here, or developers? For example, I love local-first tools as well (paying Obsidian user) but I think it's mostly a subsection of developers who really care, vs a mass market

I am also bullish on WASM from everything I've heard, and from dealing with Docker daily, although I've yet to really use it myself

Matthew Sanabria (Sep 23 2024 at 01:24):

Agreed. I loved the points you listed above and I didn't want to repeat them. I want to see more wasm support and usage in the industry. I think we're getting there though.

Thomas Eckert (Sep 23 2024 at 01:35):

There's no hard and fast rules here. I think of this from the perspective of a developer, but the scope could be broader. I think Wasm will mostly impact developers, but if it leads to faster services for users, that would be a promising non-dev benefit. Local-first, I believe has a very broad benefit for all users.

Daniel Buckmaster (Sep 23 2024 at 04:59):

I think that local-first is something that end users care about sometimes, but may not understand that they do. It might just be the difference between an app that has a spinner on every interaction versus an app that responds instantly. Or, an app that works well with limited connectivity- which could be important in certain B2B scenarios, e.g. we have customers who travel around rural areas a fair bit.

Tillman Jex (Oct 03 2024 at 17:15):

I'm totally aligned here too. WASM, right-to-repair, ARM laptops, Ollama, embedded devices all signal to me a cultural (consumer and developer) shift towards local first. I think combined with this is an ever growing awareness of online privacy and security, and a degradation of trust in corporations and media.

All in all, it feels a bit like people are taking a cautionary step back from everything. And part of that is the protectionism of having and being able to maintain / build your own stuff.

Tillman Jex (Oct 05 2024 at 06:43):

And speaking of "limited connectivity" this is applies to accessing a website _anywhere_ in Australia. When I was there on an extended visit, I wrote up a little git wrapper for my dotfile management to (among other things) push my repo to the origin and mirror concurrently, so I didn't have to wait so long. It was long enough!

Nick Sweeting (Nov 20 2024 at 22:31):

Dustin (Nov 21 2024 at 08:26):

@Nick Sweeting where are you seeing SQLite in browsers? Or do you just mean that SQLite wasm is possible now and libraries are being built on top of it?

Nick Sweeting (Nov 21 2024 at 18:29):

Yes SQLite WASM is what I was referring to, though I'd love to see it in the browsers directly if that ever happens

Tim Uckun (Nov 21 2024 at 21:10):

I am of the opinion that wasm is amongst the list of technologies that's 80% there but the remaining 20% will cause you heartbreak and pain and therefore the use will be limited. There are just too many restrictions in the sandbox. It also smells a bit too much like the JVM at the moment.

Having said that I do like the fact that every language seems to be targeting it as a back end.

Dustin (Nov 22 2024 at 09:26):

Dustin (Nov 22 2024 at 09:27):

I think wasm is just in that weird phase where it’s far enough along to be useful but not far enough along for the things a lot of people want to use it for.

Tim Uckun (Nov 22 2024 at 20:22):

@Dustin Promises cross platform VM, runs in the browser but had no access to the DOM, sandboxed environment, needs a runtime, deployed as a single binary etc.

In all honesty let's bring back applets and j2ee. In retrospect they were ahead of their time.

Ron Waldon-Howe (Nov 23 2024 at 01:44):

Oracle's ownership makes me weary of investing any time/money in JVM technology
At least WASM isn't owned by them

Tim Uckun (Nov 23 2024 at 21:10):

Java doesn't own the JVM. The JVM is open sourced although there are versions of it made by Java, IBM, and others. AFIK oracle does own the trademark just like they own the trademark on javascript.

Andrew O'Brien (Nov 25 2024 at 16:23):

I don’t know if this is a trend, but I’m trying to steer my career more toward smaller devices deployed out in the world. Writing in a more constrained environment is a fun challenge that gives me an excuse to be closer to the metal. Also no one’s asked me to add any useless AI features. (And the AI features I’m anticipating are actually interesting and worthwhile.)

Thomas Eckert (Dec 23 2024 at 00:33):

@Andrew O'Brien, I think that's smart. Similarly, I write a lot of low-level code and Rust in my free time. At work, I write Go that deploys to resource-rich, gigantic Kubernetes clusters, but the safety and memory-use considerations help me write better Go.

Ron Waldon-Howe (Dec 23 2024 at 00:56):

Piotr (Dec 23 2024 at 02:51):

@Andrew O'Brien curious what sort of devices. Also I think your point relates also to simplification, does it? Myself I transitioned all the new web-related projects to Elixir + Phoenix for the sake of simplicity as well.

Tim Uckun (Dec 23 2024 at 21:10):

After the DHH episode I decided to try ruby again after a very long hiatus. I didn't want to write a rails app (yet) so I decided to write a little app to manage my dotfiles the way I want them managed. So far it's been fun. I am basically recreating something like ansible but of course super light. What's interesting is that the language is directing me to create a DSL for managing my files and makes it so easy to experiment with the language. The code just seems to flow out of you because I just type what I think should work and 90% of the time it just does. Of course the codeium code completion really helps a lot here.

In an ideal world there would be a ruby interpreter that would be a single binary so I could just put it in the same directory as my dotfiles and execute them but OSX and Linux come with ruby already installed so it's not that big of a deal.

Dustin (Dec 23 2024 at 21:14):

I really wanted to know what the "defaults for sqlite" to make it performant are DHH was talking about.

Daniel Buckmaster (Dec 24 2024 at 00:32):

I've been pretty bullish on Cloudflare Durable Objects, and it looks like Rivet just launched something very similar:

Tim Uckun (Dec 24 2024 at 20:45):

@Dustin It involves setting pragmas for wait timeouts, chache sizes, retries etc, using the WAL, and probably a couple of extentions. I am not sure if they set strict mode though (they should IMHO)

Piotr (Dec 25 2024 at 12:49):

Daniel Buckmaster (Dec 26 2024 at 01:48):

@Piotr that looks slightly different to me. Restate, like Temporal, seems to be about _durable execution_, which is different to what Cloudflare calls _durable objects_.

Durable execution is related to workflows, and can be applied to a bunch of different backend concepts. Durable objects are what Jamsocket refers to as "session backends", and are kind of related to the "actor model", though that's a very broad term.

To make it more confusing, Rivet seems to have a durable execution engine, which they used as part of building their durable objects feature called "actors" :joy:

Interesting to see more durable execution engines coming onto the market though, thanks for the link. I'm interested in that too but haven't made the jump to trying any of them out.

Piotr (Dec 26 2024 at 01:54):

Piotr (Dec 26 2024 at 01:55):

@Daniel Buckmaster I was investigating them quite intensively after a project that used a "standard" BPMN workflow engine.

Daniel Buckmaster (Dec 26 2024 at 01:58):

Daniel Buckmaster (Dec 26 2024 at 02:01):

Piotr (Dec 26 2024 at 02:07):

@Daniel Buckmaster I guess the equivalent would be to connect to the Restate server and communicate with other processes managed by it. Restate is quite flexible, and for me one of the benefits was that it was platform agnostic.

Anyway, I am switching to the BEAM VM + Elixir wherever I can now, so don't need to use Restate :)

Tim Uckun (Dec 26 2024 at 21:01):

I hate new year resolutions but something inside me says "make a resolution to write your own language".

Daniel Buckmaster (Dec 27 2024 at 07:09):

@Piotr I wrote up some thoughts on this discussion on my blog, hope this clarifies where I'm coming from! The Erlang section isn't only a reference to you, I have heard that kind of sentiment quite a few times and I do wish someone with more perspective would enlighten me about the BEAM way of solving these kinds of problems! https://crabmusket.net/2024/durable-execution-versus-session-backends/

cevado (Dec 27 2024 at 12:01):

@Daniel Buckmaster it really depends on what you're trying to solve, stateful/stateless backends are "solutions" not problems, if you aproach it's more like a X Y problem.

there are a few solutions in erlang/elixir that are quite easier because you already have abstractions that are "cluster transparent", so once you cluster multiple backend instances, communicating between processes are transparent, you do it in the same way you'd do if they were in the same machine. but it really depends on what you need

cevado (Dec 27 2024 at 12:03):

cevado (Dec 27 2024 at 12:04):

Piotr (Dec 27 2024 at 12:04):

Dustin (Dec 27 2024 at 12:05):

Haven't watched yet, but presumably this solves for session backends and not durable execution?

cevado (Dec 27 2024 at 12:15):

@Dustin again, "session backends" "durable execution" are solutions, not problems. what is that you're trying to solve? are you trying to keep state on disconnect(for a wizard or a complex form)? you have a FE heavy on business logic and you have a shared state between the FE and BE?

Piotr (Dec 27 2024 at 12:16):

The videos present the possibilities of the BEAM VM in general, the first video durable execution partially; but I encourage you to watch it.

cevado (Dec 27 2024 at 12:22):

from what i understand any background job solution would take care of "durable execution"... in Elixir in particular you can solve that with asynchronous tasks, remote execution, or even Oban(background job backed by postgres). it would be even simple to connect oban execution to cycle to a pubsub and have it reported in real time in the FE.

Dustin (Dec 27 2024 at 12:22):

I think Cloudflare Objects weirdly couples the two topics which is what makes it confusing. I always think of durable execution in terms of https://flawless.dev The point being to be able to re-execute any computation independent of software or hardware failure.

Session backends seems like not so new tech in comparison with a lot of differing solutions (one of which seems to be this Cloudflare Durable Objects + event handler)

Dustin (Dec 27 2024 at 12:24):

Coming from that perspective, a background job solution is insufficient since you would need to handle the reconciliation of what parts of your job succeeded if it failed part way.

Dustin (Dec 27 2024 at 12:25):

It definitely doesn't seem like there's anything blocking towards this existing for all languages though!

cevado (Dec 27 2024 at 12:28):

Dustin (Dec 27 2024 at 12:35):

Yeah, that's pretty common functionality for a background job solution. As I mentioned above, the difference between a background job and a durable execution (at least as far as I understand it) is that the retrying/chaining behavior is at the job level for a background job, whereas a durable execution is guaranteeing the execution within the job.

In theory, I think it's possible to model a durable execution inside something like Oban, but it would mean isolating anything state related into it's own job.

// job1
const job1(cliArgs) {
let user = "Adele Goldberg";
let comic_id: u32 = flawless::rand::random();
let url = format!("https://xkcd.com/{comic_id}/");
let content = flawless_http::get(url).send();
let quote = parse_comic(content);
let greeting = format!("Hi {user}! // '{quote}'")
}

Where each of the flawless commands will retain the first execution's result even if the job reruns.

// background job
const job1(cliArgs) {
let user = "Adele Goldberg";
let comic_id: u32 = flawless::rand::random();
}

const job2(cliArgs) {
let url = format!("https://xkcd.com/{comic_id}/");
let content = flawless_http::get(url).send();
}

const job3(cliArgs) {
let quote = parse_comic(content);
let greeting = format!("Hi {user}! // '{quote}'")
}

Piotr (Dec 27 2024 at 12:38):

@Dustin yes so Oban is not stricte durable execution of course, but it really depends on what you want to achieve. because parts of your job might be just retried on each job retry run if they are read-only, or the state might be saved to your data source of truth.

Piotr (Dec 27 2024 at 12:41):

@Dustin to be honest I have been pondering and playing around durable execution after using a BPMN engine in one of my previous projects, and landed on just being able to excute what I need with Oban because it keeps my stack simple and reduces the number of external tool dependencies (like Temporal or Restate)

Dustin (Dec 27 2024 at 12:42):

Yeah, I also feel like I don't really have a use case for it. But, I could see it being a valuable abstraction for a large organization so each team/engineer isn't rebuilding retry behavior/semantics.

Dustin (Dec 27 2024 at 12:44):

Piotr (Dec 27 2024 at 12:48):

@Dustin yes that's true, but it occured to me that I need this only in certain cases, because in most of them I just need plain old transaction ACIDity that a DB engine provides. So in my use case the Oban / Restate error handling etc is useful for connection to third party services.

Piotr (Dec 27 2024 at 12:50):

So if someone is in the microservices mode - yes Temporal / Restate might be great.

Dustin (Dec 27 2024 at 12:50):

Yeah haha. I was just thinking "This seems like a solution for a problem made by microservices"

Dustin (Dec 27 2024 at 13:42):

@Daniel Buckmaster I really liked your post "De-escalate vocabulary, de-escalate hype" reminds me a lot of the awesome essay from George Orwell Politics and the English language I read it like once a year :grinning:

Dustin (Dec 27 2024 at 13:49):

cevado (Dec 27 2024 at 21:38):

but that's what flawless is doing, but with a uniq/dedicate set of apis and syntatic sugar. that's why I focus on what are being solved here. I just found this video that is a great comparison between "javascript world" and "elixir world"... in javascript land you have a bunch of abstractions and a bunch of pre-baked solutions for stuff that is just "use this library" but what if you don't need that amount of abstractions to begin with? if this video catch your attention I suggest to watch the Chris Mccord talk I sent earlier
https://www.youtube.com/watch?v=6C4G8WPLIKQ

Dustin (Dec 28 2024 at 09:42):

Maybe I should have watched this earlier :grinning: I think we're actually saying the same things. All I was trying to say is the promises of durable execution is not something unique to a language or tool but it is a certain kind of abstraction that isn't given to you by something like Oban. You'd have to craft your code in a particular style to achieve that.

cevado (Dec 28 2024 at 15:37):

I agree... my point is that the language/runtime allows you to model the solutions with different abstractions, some more simple than the others.

Alden (Jan 02 2025 at 02:13):

Back on the topic of Wasm I really think it has potential to make a huge impact in software distribution. If you think about the web as an app distribution platform it has incredible UX - you go to a link and it downloads all the dependencies (some of which are probably already cached), JIT compiles write once, run anywhere software (JavaScript), and runs it all in a sandbox so you can be reasonably confident that your computer won’t be hijacked my malware (today anyway). Additionally the browser provides a common (though very opinionated) graphics framework, accessibility, etc.

One of the reasons this works so well is the sandboxing - if you look at most other software distribution platforms there is either a high bar or hurdles for what they will distribute (think Debian packages or windows software signing) or have a questionable supply chain (NPM)

If you wanted to build a new app delivery paradigm Wasm has all of the ideal properties - it’s truly WORA, is a compilation target for many popular languages, has a really solid sandbox design, native-ish performance, you can define the environment (e.g. provide OpenGL bindings) and everything happens in user space which is amazing! (in comparison to docker which requires a Linux kernel running with namespaces, groups, etc.).

So what I envision is an app “browser” if you will where I could type e.g. wasm-run https://ziglang.org/wasm/latest --arg1 … and boom everything downloads and I have the latest zig compiler running in a sandboxed environment in seconds regardless of the platform. Or you could even provide gui apps like games, spreadsheet software, etc. through the same method.

Anyway, this is a long-winded way of saying that I’m also bullish on Wasm if people can appreciate that it’s not just for web or just another standard take on a language VM

Tim Uckun (Jan 02 2025 at 20:49):

All this talk reminds that we are all trying to reinvent JVM and J2EE and Applets. I remember back in the bad old days writing applets that had rich front end interactivity. Sure it took forever to load and froze your browser while loading but once it loaded it was great. I remember being able to create a WAR file and uploading the app via a browser to a J2EE container where it magically upgraded an existing app and the container provided a cache, KV store, configuration, database , queue etc. You could also write your app in different languages because they had groovy, jruby, javascript, python etc running on the JVM back then. Just like Kubernetes does today but more messy.

WASM is the new JVM, so I look forward to somebody creating the equivalent of J2EE container or kubernetes for WASM. Everything old will become new again because our industry loves reinventing everything.

Don MacKinnon (Jan 02 2025 at 22:37):

Dustin (Jan 02 2025 at 22:43):

Dustin (Jan 02 2025 at 22:46):

Apparently there's also research for the browser use case, to try and be able to chunk wasm blobs by component boundaries so that you don't have to load the entire binary in a single call (even if it is streamable!) But, I'm not even sure if the wasm component was agreed upon yet. Still early days for wasm!

Ron Waldon-Howe (Jan 02 2025 at 22:52):

I can't remember the interview (might have been a Change Log episode) but there was a suggestion that browsers / WHATWG could (in the far off future) make WASM the only supported runtime, but retain JavaScript compatibility by implementing V8/JavaScriptCore/SpiderMonkey on top of WASM
JS would be on equal footing to every other language that can run or compile to WASM

Daniel Buckmaster (Jan 02 2025 at 23:13):

@Tim Uckun what do you think the web platform is missing to bring back the applets experience? Browsers have cache, KV store, database etc even without WASM. (Though of course we are still at the mercy of whether browsers implement these specs fully or usefully!)

Tim Uckun (Jan 04 2025 at 20:57):

@Daniel Buckmaster I guess browsers have already brought a lot of what applets offered into the browser itself including stateful connections and graphics. It all seems a lot more clumsy to develop in though. Applets were pretty straightforward java (whatever that means to you)

Stream: general

Topic: What trending tech are you most bullish on?

Thomas Eckert (Sep 21 2024 at 23:41):