On disk, I just store those data structures as JSON, but it could be any serialization language if you have a parser for it. unist (#3 above) has the same philosophy.

This might be more relevant to interop: https://dion.systems/blog_0002_roadmap.html

Kartik Agaram nice, seems like their doing something in this zone, I’m going to hop on their discord

A bit of personal experience in this space: In the first iteration of my Digital Scientific Notation Leibniz (which is not a programming language, but still a formal language with syntax and semantics), I opted for an XML representation of the AST in order to permit processing Leibniz code more easily, without having to implement a parser. I picked XML over s-expressions because of better tooling support in various languages, and XML over JSON because of the richer structuring possibilities.

I had no intention of doing anything one could call universal, but then, XML or s-expressions may well be as close to universal as one can get: a tree structure. Now I wonder: is there anything beyond the tree structure that is universal about ASTs?

https://en.wikipedia.org/wiki/Concatenative_programming_language

might be relevant.

Indeed. Forth and Factor (the only concatenative languages I know a bit) pose the same challenge for a universal AST as Common Lisp with its reader macros: how to handle languages that permit extending the parser?

@Kirill Chernyshov is the parser extension Konrad mentioned what you were suggesting there? (I thought for a second you meant the ASTs in those langs aren’t well represented as trees?)

Konrad Hinsen what could I search for learn more about that?

I meant to say that it is possible to have a stack of words and atoms as an another form of AST. I have been working on some sort of translator from clojure to concatenative language (in-house language based on joy) some time ago. And that was surprisingly nice.

@Luke Persola Check out Reader Macros in Common Lisp, Parsing words from the Factor documentation, and PARSE from the Forth standard. All three are extension mechanisms for the language's parser, implemented in the language itself. Which means that a complete parser for either of these languages must include an implementation of the entire language.

In Common Lisp, this is not much of an issue except if you want to write a code processing tool in a different language. There is a well-defined representation of code after parsing, which is the level at which macros work. Moreover, reader macros are a rare in practice.

For Forth and Factor, parser extensions are very common and basically mean that there is no notion of an AST at all. As an example, consider the XML parser in Factor. It allows you to embed XML documents in Factor code, not as a string but as a syntax extension. Superficially, Factor code is a just a sequence of space-delimited words, but with embedded XML, it starts to look very different.

Konrad Hinsen thanks!

Have you read https://tonsky.me/blog/reinventing-git-interface/ for starters? 🙂

Just did, thanks for sharing

@Kiril Videlov

By “single source of truth” here you mean having a master branch?

I think there is a lot of power that could be unlocked for devs by continuous sync-ing of code in the cloud.

So I guess the current best way to do this is having a culture where people try to commit smaller PRs more frequently, and frequently rebase their local changes on top of the shared branch. Are you imaging automated that a bit more to we don’t have to rely on individuals as much?

(I just read about the 2-way syncing so I think I understand now,)

The next funny point of reference about synchronization: https://research.chalmers.se/en/publication/232551 🙂

Also check out 3-way merge problem: https://tahoe-lafs.org/%7Ezooko/badmerge/simple.html

Seems like the right way forward is a "patch algebra" like Darcs or better yet Pijul. 🙂

https://getsturdy.com/features/live move local workspace to the cloud

So I kept coming back to

Normally people would just represent this change over time in the code itself

That is, why use something related to VCS as we know it for this purpose? I was thinking “we only ever actually run the most recent version of the code” and then realized that’s not true when I thought of migrating a cluster: then you have multiple versions of the same code running at once and (potentially) interacting with each other. Your system has to be thought of as including these different versions. Tangental but maybe relevant.

It's not true when making music either. Making music is just as valid a use of code as making databases

maybe more valid as we'll always make music.

Yeah, I’m all for artistic programming

More generally:

• Drawing boundaries is an essential aspect of design. Boundaries allow you to make progress by making simplifying assumptions about a temporary area of concern. "Practical" programming often draws a box around "what's running in the computer right now". VCS conventionally lies outside that boundary.
• Boundaries can also stifle creativity. Questioning boundaries is a super useful activity, as is happening in this thread.

I think I disagree - creativity thrives off boundaries. Unbound creativity is noise

Creativity is about pushing against and beyond boundaries but you still need the boundaries

I agree 🙂

As a hack, I wonder if you could use Continuous Integration (eg, github actions, CircleCI) to generate visualisations/aural representations, as you commit code (maybe generate new branches for units of time if need be)…so you can see snapshots of your past in whatever visual/aural form you’d like, to better “see” where you want to scrub back in time to. So to jump to a point in time, you’d just do git reset --hard [sha] . Interactive rebasing could work too, as long as CI regenerates the snapshots

If you are talking about live coding music like an algorave then I think you want something that somehow projects the code into a low dimensional space. In this space, "commits" would just be waypoint markers that are set down. So you could mark where your verse/chorus is then explore freely and get some sense of when you were getting back close to where you were.

While this appears ridiculous at first I've seen some really impressive dimensional reduction things on Chris Olah's blog. If you also limit yourself in terms of starting dimensionality this might work and could really add something to a live coding performance to see the path that the performer took.

I went to an algorave where there was a performance by two people editing the same code. They had two cursors and their edits showed up in different colours. It meant you could see exactly how each of them was remixing each others lines and how they were both contributing. It really added to it.

My Mu project (https://github.com/akkartik/mu) is building a computing stack up from machine code. The goal is partly to figure out why people don't do real prototyping more often. We all know we should throw the first one away, but we rarely do so. The hypothesis is that we'd be better about throwing the first one away if rewriting was less risky. By the time the prototyping phase ends, a prototype often tacitly encodes lots of information that is risky to rewrite.

To falsify this hypothesis, I want to make it super easy to turn any manual run into a reproducible automated test. If all the tacit knowledge had tests (stuff you naturally did as you built features), rewriting would become a non-risky activity, even if it still requires some effort.

Turning manual tests into automated ones requires carefully tracking dependencies and outlawing side-effects. In Mu, functions that modify the screen always take a screen object. That way I can start out with a manual test on the real screen, and easily swap in a fake screen to automate the test. Which brings us to my problem:

How do you represent screens in a test?

Currently I represent 2D screens as 2D arrays of characters. That is doable a lot of the time, but complicates many scenarios:

• Character attributes. If I want to check the foreground or background color, I currently use primitives like check-screen-in-bg which ignores spaces in a 2D array, but checks that non-spaces match the given background attribute. In practice this leads frequently to tests that separately check characters and then go back to check colors and other attributes.
• Non-text. Checking pixels scales poorly, either line at a time or pixel at a time. I've tried ideas like separate buffers for text vs pixels, so that at least text continues to test easily.
• Proportional fonts. Treating the screen as a grid of characters works only when each character is the same width. If widths differ I end up having to go back to treating characters as collections of pixels.
• Unicode. A font like unifont (http://unifoundry.com/unifont/index.html) is mostly fixed-width, but lots of graphemes (e.g. Chinese, Japanese, Korean, Indian) still require double-width to render. That takes us back to the problems of proportional fonts.

Can people think of solutions to any of these bullets in a text-based language? Or a more powerful non-text representation?

My instinct would be to accept that you are ultimately going to be checking pixels and see how far you can get with that. For example, saving draw calls instead of raw pixel values, tracking dirty regions/some kind of copy-on-write. Combining some zealous saving of state with some amount of recalculating can be pretty efficient.

Oh I'm actually not thinking about performance. I meant it's hard to make tests that are self evident when they're about pixel events. Extremely long lines, etc.

So you’ve already decided you need to test for all these things, right? You’re just trying to figure out the best way to model the state of the screen to cover all these cases?

Yeah. It's even just the syntax. In a complex app drawing to screen what's the cleanest way to represent desired screen state in the interface and get nice error messages (that use that representation) on test failure.