Hey everyone! Here is a very brief overview of what we’ve been building to enable us developers to create and share full-stack web apps without storing each other’s data. We’ll be providing more details each week! Don’t hesitate to drop any questions below https://share.descript.com/view/LW69H0AkbuM?transcript=false
Cool! Do you have some kind of protocol to ensure the web apps can access and use the personal cloud data in a reliable way?
@Andy Jakubowski yep! Currently it’s pretty simple but each web app is served on its own subdomain with strict CSP rules to prevent sending data to external servers. To actually persist data an app can connect to any number of “agents” (which are hosted isolated backends) but it must declare which ones it connects to in its manifest so the user can see. Each agent will have access to a separate, isolated part of the users database and are decoupled from the frontend app. So, for example, that means the “calendar agent” can be used by multiple apps so that you can have a canonical place for all your events. And in your frontend app, you don’t have to do any authentication, authorization, or API-key management. You just declare in your config/manifest which agents your app interacts with and then you can use those API’s directly. Does that makes sense?
Jack Rusher ^ my response above might shed some light but we are working on a written overview as well as some preliminary documentation. We’ve also begun pulling out and open-sourcing our Agent runtime (effectively serverless, isolated JS functions) if you’re interested in following that: https://github.com/aspen-cloud/runtime
Matthew Linkous It does make sense—thanks! I like the idea of owning my data and letting third parties access it in a controllable way.
Two thoughts come to mind.
One, it seems similar to how Apple and Google let apps access location, photos, health etc. data. Would you want people to clone this data somehow, or start from scratch?
Two, why don’t we have something like that already? The idea seems good for the user in principle, so I wonder what may be the reasons it hasn’t really materialized in the past, and what may be different now.
I think what you’re working on could be really awesome!
@Andy Jakubowski Great observation! We model a lot of ideas around permissions on the patterns used in mobile app ecosystems, and even take it step further by limiting networking.
- Re: cloning existing data vs starting anew, we decided to build from scratch so a user will have a clean, single source of truth while still having the ability to write some code (and potentially publish it for others to install with a click!) that can import data from other sources with an API. We would happily assist someone in making an agent that pulled, for example, Google calendar data into your Aspen calendar.
- It’s a great question. I think previous attempts at creating personal servers with one-click installable apps have started with existing open-source projects and used ordinary linux VM’s; this however, is fairly costly to the user (paying when not using), still fairly technical to manage, and the existing apps tend to be either low quality or aimed towards developers. Our platform is effectively “serverless” so it is much less expensive to run, has a uniform application model so you can much more easily work with the underlying data and API’s, and should be much easier for non-technical users to use.
The other problem is existing apps tend to roll their own authentication and data storage so making them work with an existing platform is very difficult. What ends up happening is, you log into the “platform” and then have to again log into each app which likely has it’s own Postgres process running, etc. Overall, the existing solutions are very clunky and inefficient.
Great follow-up, thanks Matthew Linkous.
This makes sense to me. It may be indeed the case that nobody’s come up with a working, easy to use solution to this.
Your mention of your platform being serverless stood out to me, because I assume data can be both written and read in my personal cloud. I saw your GitHub repo mentioned an append-only log, I wonder if that has something to do with it? You write by appending, and reading is a matter of reading a “static file”?
I’m interested in your approach because I was thinking of making a similar thing for UI on the web. Many UI elements reuse the same logic, and only differ in styling. So you could define the logic once with statecharts, for example, and then let people style those elements with styles from their “personal style cloud”.
I built a prototype of a thing like this in 2010. The main problems with adoption were: app devs aren't interested in building for a platform without users, and users aren't interested in signing up for a service unless it has a killer app. Do you have any plans to overcome this trap?
Jack Rusher yeah that’s exactly right and easily the biggest risk in building something like this. I think it’s going to be very difficult to overcome no matter what. But we’re trying a couple things:
First, we’re prioritizing getting Aspen in to the hands of people that are both devs and users (like the lovely folks in this community!) so we can all build apps for ourselves and each other.
Second, while we probably won’t directly come up with the killer app on our own, we’re working really hard to provide the right tools and complementary API’s for the developer or designer who does have the brilliant idea. For example, we’ve noticed a lot of really cool productivity apps like Superhuman or Cron, are primarily innovating on the frontend. So we’re trying to provide robust, canonical agent (i.e. backend) implementations of common API’s like calendar, messaging, documents, etc. This way if someone has a great idea for an app, they can skip all the of the tedious authentication, authorization, db management, token management, etc and just build the frontend of their dreams. Same goes for backend devs who don’t want to bother with a frontend: they can just build and upload an agent for others to use from the terminal or connect to their own frontend.
It’s going to be a challenge but we’re very committed to our mission and will spend whatever time and effort it takes to polish the UI, refine the API, and support the developers.
@Andy Jakubowski yeah that’s pretty much it. We’re working on a more detailed overview with explainer graphics to fully illustrate how persistence works. But the main way to persist data is to store events on an append-only log. It’s not a static file but we inject a runtime object into the agents code so they can call methods like pushEvent and create projections of the data through aggregations. If you’ve used Redux, it’s a very similar pattern except in Aspen it runs on the server and is automatically persisted.
Here is an example of a task manager agent I made: https://github.com/matlin/todos-backend/blob/main/src/index.ts
Disclaimer: some of these API’s will very likely change
The main thing to note with that example agent is that it’s a complete backend for multi-tenant, multi-list task manager all in ~100 lines of Typescript
Nice! I haven’t worked with Redux, but I have worked with XState, which also combines all logic in one massive object.
Will be interested to read that overview of your persistence implementation details. In particular, timing and syncing events. Martin Kleppmann’s Designing Data-Intensive Applications had some interesting discussion of this, but I’m sure you know that one already!
Martin’s work is definitely a big influence. Currently everything is serialized on the server but this obviously doesn’t scale in terms of latency from the client so we have an implementation that acts mostly like a CRDT that we’ll share more details on in the future!
