expressions are much more difficult to work with, since their structure is inherently variable in both length and depth

writing generic procedures for the common relational operators is a lot more work and will eventually take you to a place where you want a “common data model”

which will probably end up looking like tuples

expressions are much more difficult to work with

That sounds like a UI design challenge, which I'm ready to take on 🙂. They might not work in a text-file-based language, but that's no concern to me.

writing generic procedures for the common relational operators is a lot more work and will eventually take you to a place where you want a “common data model”

I don't understand this part. Are you talking about implementation concerns or something else?

yeah sorry, I was thinking of the implementation level

In general I'm ready to deal with as much implementation complexity as is needed. I'm trying to build a human-centered language, that's all that matters!

It’s hard to outright ignore implementation complexity, because it means you need to innovate on even more things - not just novel UIs and interactions and mental models, but also novel algorithms and foundations. Each one of which is easily a black hole problem 🙂

I’d be really curious to see how you’d do a relational join over arbitrary expressions

I’m not sure you really get away from normalization and keys. In your plain old data example, you’re explicitly stating the key (ID). I think you can’t escape defining identity somewhere and for normalization you either have defined access paths or you don’t. In the former, you’re some level of denormalized, in the latter you’re in something like 6th normal form.

you could argue the user doesn’t need to think about it, but that’s really saying the user can dig their own hole as deep as they want (same is true in SQL and the others too, it’s more than happy to let you do crazy combinations of normalization and identity, e.g. regex joins)

“a human-centered language” is likely as much about guiding people toward something that helps them manage complexity as it is about letting them do something “naturally.”

there’s a really interesting tension between giving the user what they want and trying to steer them clear of pits of failure

tuples force a kind of consistency that expressions don’t and that means there are fewer places for misunderstandings to hide

I plan to start thinking about execution efficiency only after I've developed a full "beta" of a programming language that achieves my goals for the user experience. The biggest risk to my project is that I sacrifice the vision due to short-term implementation worries.

Another reason why I'm holding off: the hardware landscape is changing constantly. One big upcoming change: there will be AI accelerators on the market in a few years, which are quite literally parallel dataflow machines, so I'm not worried about how my language scales on an Intel CPU 🙂. At least some of those accelerators will be feature-ful enough that they can run more than neural networks, in the same way that we now use GPUs for more than 3D graphics. And further: the biggest advantage of dataflow hardware may not even be the parallelism but the pure implementation simplicity of mapping a dataflow-like language to dataflow hardware, rather than von Neumann hardware. It's not fully clear what the "assembly language of a TPU" will end up looking like, but I bet it's good at executing joins 😉.

Point is: I think there will be plenty of compute power available to run whatever language I end up with, even if we have to wait a few years. I don't plan to be pushing efficiency on x86 CPUs within the next 5 years of development.

But this may be a digression. I'll read your other points, hold on 🙂.

I’m not sure you really get away from normalization and keys. In your plain old data example, you’re explicitly stating the key (ID).Yes, but the key emerges naturally from the data. The idea is: if you write an equation that expresses x as a function of y and z, then those become the (compound) key that identifies an x. But keys aren't an actual language construct, and neither are tables, and therefore it's impossible to come up with "badly-designed tables", unlike in SQL or Prolog, where you can lump arbitrary values together and call them a tuple.

So in contrast, I'd say that it's SQL that allows users to dig holes wrt. their data model, and this proposal takes away the shovel.

that works out to 6th normal form then, which I’m a big fan of 🙂

Perhaps it is an implicit, mandated normal form, sure 🙂

so in this world, data is all purely represented by code that must be run

that’ll lead to some really cool research around how you compress that and make it reasonable to transmit to whatever coprocessor it’s running on

Oh, and I didn't mention it in my original post, but in the envisioned model of relational programming, there is no such thing as a "data structure" (not even lists), since data structures re-introduce tuples and tables and all their problems. Data structures (and functions) would emerge naturally by linking variables together through formulas (or you might call them constraints). I'm still working out what the UI to this will look like of course, and that's probably the biggest challenge 🙂

so in this world, data is all purely represented by code that must be runYeah 🙂

The question of "when does the code run" is an important implementation question, but again this is something I'll deal with after I get the UX right.

there is no such thing as a “data structure”So how do I ask for the name of customer 10? Or how would I model that a player has a hand of 5 cards?

one thing I could never figure out is how you’d get rid of things that have sets of other things

customerID = 10

print customerName

(My language won't have print statements, that's just for illustration.)

See my previous #thinking-together post on "function calling". It's deeply related in a non-obvious way 🙂

ah yes

how about the card one?

imagining that cards are just numbers for simplicity

how would I model that a player has a hand of 5 cards?

You'd do it similarly to a Prolog relation holds(Player, Card), except the "connections" between datums are stored implicitly in global (UUID-based) variables, rather than via a named "predicate". You'd have two variables, player and cardInHand. When you constrain player to a value, e.g. player = "Nick", cardInHand will have five values, one representing each card. You could "print" all five values, or use them as part of a larger calculation.

one thing I could never figure out is how you’d get rid of things that have sets of other things

I'm planning a model of mutability based on https://link.springer.com/chapter/10.1007/978-3-642-24206-9_16, i.e. accretion instead of mutation (buzzword thanks to Jack Rusher and Rich Hickey). So you express the "new set" as a function of the "old set".

reminds me a bit of the constraint propagation networks that @William Taysom did really cool stuff with

Yeah this is a form of constraint programming. So are Prolog rules, for that matter 🙂. I'll definitely like to hear what @William Taysom thinks if he's dabbled in it.

I guess I've been summoned. 😈 I'll answer any specific question: not just three. Let me share some idle thoughts with the small advantage that I want to give this stuff another serious go soon, so it's been on my mind.

The idea I liked best about constraint propagation networks is defining the network (system of equations) entirely independently of how any of it gets resolved. It helps with...

"hardware landscape is changing constantly" — Don't commit to an evaluation strategy until you really know what you want to do. Focus instead on interface and sense making. Have a really rich system for exploring toy models, then use black boxes for the big tricky bits.

I don't think very much about computational complexity. If it's too hard calculate, the answer is probably random and not worth knowing. I mean worst case optimization is always terrible because you're essentially finding something random: no structure beyond exhaustive search. For instance graph-database join problems come up because pruning is happening in the wrong order or you don't have a good search heuristic.

"6th normal form" — Think hash-tables with compound keys. In most cases, they offer a better way to think about things tables of tuples because they show functional dependencies and entity-relationship cardinalities. (Given a suit and a rank, you have a card.)

"mutability" — Make time first-class and the rest follows. Another way to think about it is that there is no time only another space-like dimension.

Unordered and ordered collections are entirely different beasts. Always know what you're modeling and never allow for arbitrary orders to be imposted on unordered things.

Your last two paragraphs summarize my primary principles of the last two years, and if I hadn't committed to them I think I would have gotten stuck and failed in my quest by now 🙂. The rest I've only come to realise more recently.

What's the difference between a "constraint propagation network" and "constraint programming language" though? I'm only just starting to look toward that kind of literature.

When I say "propagation network," I basically mean Sussman/Radul Propagators https://groups.csail.mit.edu/genesis/papers/radul%202009.pdf https://dspace.mit.edu/handle/1721.1/44215. I don't see much new on that front.

As for "constraint programming language," that makes me think of SAT solvers and "constraint logic programming" — write in Prolog, then do SAT solving for some parts instead of (or together with) unification.

The other thing I know is optimization along the Linear Programming, Mixed Integer Programming, and Quadratic Programming vein. That all seems to live in a world apart from any other programming community.

I recently (re-)investigated “constraint logic programming” and concluded as you say, that it’s merely logic programming plus “extra support for constraint solving”. Unfortunately, the “logic programming” part implies tuples and tables as discussed earlier 😥. An example of a constraint programming language which is not a logic programming language would be https://www.minizinc.org, which I happily stumbled onto in my undergrad years because it was developed here in Melbourne 🙂.

Maybe an interesting angle to break open your head and allow increased weirdness and possibilities to enter would be to think of your primitives as

your thing : tuple-based data structures :: Signed Distance Functions : 3d meshes

https://www.youtube.com/watch?v=8--5LwHRhjk

Jack Rusher Wait, what do the colons mean? 😅

your thing "is to" tuple-based data structures "as" signed distance functions "is to" 3d mesh 😹

Hmm... I’d say that’s backwards! Tuples are points, which can be used to define meshes. Formulas (my thing) are formulas, which can be used to express SDFs!

that's what I'm saying, lad

Nick Smith Reversing the polarity is what I get for typing a reply while on a Zoom call 🙄

Nick Smith What is your definition of a formula? Is it just a computable expression of type "boolean"? Or do you envisage restrictions that ensure that solutions can be found with reasonable effort (time, space, etc.)?

Yes, essentially just Boolean expressions, though calling them that is misleading, since it suggests that the Boolean value is the "output", and the variables that form the expression are the "inputs", which is not how a constraint language works.

The language may end up having restrictions that guarantee time and space bounds, but again, I want to get the "feel" of the language right before I start imposing those kinds of restrictions, so I haven't given them much thought. (But I have given termination thought).

Thanks Nick Smith. The background of my question was the user's point of view: how much does a user have to know about computation and/or constraint programming to understand what is and is not possible in a formula.

@William Taysom You wrote: “Unordered and ordered collections are entirely different beasts. Always know what you're modeling and never allow for arbitrary orders to be imposted on unordered things.”

How do you model ordered collections “properly”? I’m only used to programming languages/environments where I end up using lists/vectors/arrays, which of course impose arbitrary order. Sure, sometimes there are set data types that at least express that order doesn’t matter, but it sounded like your advice was hinting at more than just picking the correct type — I might also be missing the relevant experience here with constraint languages…

Stefan Lesser I'm wondering if he's thinking in data Nat = Zero | Suc Nat terms...?

Konrad Hinsen I don't know the answer to that yet, as I don't yet have a working programming language 😇

Stefan Lesser Doing ordered collections "properly" means including only as much ordering data as is needed to capture the desired order. So, if you have a collection of elements, and those elements are in a total order, then you can annotate each element with a natural number indicating its position in the sequence. If your elements are in a partial order, then you could encode their relative position using something like a multiset (which are partially ordered by the "inclusion" (⊆) relation) containing "elements that precede this element", or whatever value best explains how the ordering was constructed and how it will be maintained as the elements change. (For example, maybe a nodes-and-edges model is better.)

Stefan Lesser with order there are two challenges:

• Purity: Accidentally ordering unordered collections. It means that you can't print a set without suppling an order. It means you cannot loop over a set either. You can map over a set, but all the results have to be independent. Yes, debugging becomes kind of tricky. When looking under the hood, it's fine to see the order, but for application level debugging, you need different ways of inspecting the unordered collections. Back a long while ago Ruby, my weapon of choice, had unordered Hashes. Given her intrinsically imperative nature, that went badly. So the most Rubyesk of solutions was adopted: all Hashes are ordered by when you insert keys. This has problems, but it is useful and unsurprising.
• Modeling: Having good ways of managing orders. Consish lists are simple and effective. Annotating data with indices leads to madness. A decent expandable Array (push, pop, queue, drop, splice) works 95% of the time. For that last 5%, you'll want some sort of omnibus tree or rope structure together with cursors, ranges, and the like.

In the “future of coding”, your point #2 should definitely be relegated to the language runtime and not appear anywhere in the language itself IMO. I consider it an implementation detail.

I’d say the key is to give users of the language good ways to specific the abstract ordering information they need and ways to operate over such ordered collections

In the language itself, you need to decide on what level or degree sequence munging you want to feel natural. On one extreme, you might go with ordering by time column. In that case, try to avoid use cases where mutable indicies (1, 2, 3) seem like a solution. On the other extreme, you might want to support, I don't know, arbitrary selections of DOMish tree-nodes. Yes, in an especially high-level language you'll likely want the application developer to be able to ignore the underlying representation. (Often this means doing tricks to as to avoid allocating small arrays.) Perhaps providing a DLS escape value for fixed sized arrays, or, I don't know whatever your GPU's shader language supports. It becomes a matter of answering, "What is this system naturally good for?" And the even better question, "What is this system naturally bad at?"

Disclaimer: what follows is my thoughts on language design for "the future of coding" (the next decade), not for a near-term business need or specific market niche.

I have trouble accepting the "you have to make a performance trade-off" hypothesis that I feel is alluded to ("what is it bad at?"). I feel it is invoking the myth that the higher-level your language is, the worse its execution efficiency must be. (Once upon a time, garbage collection overhead was ridiculed.) In reality every program is just a specification and efficiency comes down to how well that specification can be translated into a hardware-consumable language. A smart mapping requires a smart implementation, and also a smart choice of hardware platform. I'm dead certain that CPUs and cache hierarchies are not the future, so assessing a language based on its ability to support "cache locality" and such is a dangerous mental trap. As I've previously said, implementation issues will be a fun puzzle that comes after the language has been designed, and not a moment sooner. (As long as the language has a clear specification, a "suboptimal" implementation will be easy to develop.) The kinds of applications (and therefore markets) for which the language is "fast enough" will be revealed in the course of building the "best" implementation possible on each potential hardware platform, such as a CPU, GPU, or AI chip.

You may not have been hinting that way, and if so just ignore the above 😛

Anyway, if we can throw the "efficiency is a problem" hypothesis in the dustbin, the only real concern is what does a user of the language experience? How do we present the "raw essence" of order and nothing else? I think that's what the interesting problem here. You're right that oftentimes indices may not encode the essence of the order the user is trying to express, and perhaps a graph model (which cons-lists and trees try to approximate) is a more faithful encoding, and better supports the operations that will be performed upon the collection. Happily, logic languages (and the kind of language I'm designing) are particularly good at representing and operating over graphs.

I better clarify. I meant "what is this language bad at expressing?" No performance consideration whatsoever. You have to pick the most natural, idiomatic way to say things. If you answer "I want people to be able to do anything," then you get bloat. Between molecular and kitchen-sink extremes there are a lot reasonable (and unreasonable) tradeoffs.

Consider dynamic/scripting languages, which are all practically the same at some level. (I did a talk promoting them forever ago when Java was king, and the Web was young.) Consider Lua, Python, Ruby, and JavaScript.

Lua strives for a precise, beautiful minimalism. "Lua provides mechanism not policy. ... Instead of littering the language with lots of features, we provided ways for users to program the features themselves, in the way they wanted them, and only for those features they needed."

Python is includes batteries plus "there should be one — and preferably only one — obvious way to do it." Prepare for ridicule if you don't like the one way. (Generators, I'm looking at you.)

Matz says, "I'm trying to make Ruby natural, not simple." Ruby is complicated, overcomplicated but in all the "right" ways. Let's have methods, blocks, procs, and lambdas with multiple notations for each. Let's have objects, classes, singleton classes, metaclasses https://en.wikipedia.org/wiki/Metaclass#In_Ruby, and mixin modules, which can be included, prepended, or extended. Then add refinements https://rubyreferences.github.io/rubyref/language/refinements.html for kicks. For concurrency, we have threads, UNIX processes, coroutines, continuations, and now "Ractors" https://github.com/ruby/ruby/blob/master/doc/ractor.md. Yet somehow Ruby ends up fairly programmer friendly.

Contrast with JavaScript, which is simultaneously overcomplicated https://www.destroyallsoftware.com/talks/wat and underpowered, "Given the process that created JavaScript and made it a de facto standard, we deserve something far worse." (Maybe React, TypeScript, or whatever makes JS better. I honestly get by with jQuery despite its clear shortcomings.)

I believe that (alleged) tradeoff is best handled by defining a general-purpose base language that can support embedded DSLs (in a structured editing environment where non-textual syntax is allowed). Each common task category (e.g. list manipulation) can have its own DSL, which is a thin veneer (a "syntactic sugar") over the base language. In short: we accept there is no universal syntax that intuitively expresses all computational tasks.

If we support user-defined syntaxes atop a common underlying semantics then I truly think "people can do anything" all in the same language environment.

@William Taysom I've spent some time today reading some of Alexey Radul's thesis on "propagation networks", and I find the idea underwhelming to be honest. The biggest issue is that they don't innovate on how to express compound data: they still advocate the use of Cons, and other hierarchical data structures. Additionally, their proposed means of abstracting a common pattern (subnetwork), is to define a Scheme function that parameterizes and can generate instances of the subnetwork. In other words, their semantics isn't "complete"; it needs to be embedded in a functional language!

The one idea that I agree with is basing computation on constraints (e.g. intervals) rather than values. But they haven't sold me on their broader approach.

You make a good point about abstraction/parametrization. It’s certainly where my attention is these days as most rules that I want depend on a context of 2-6 things.

Some of the applied category theory people are interested in the topic of modeling abstraction and composition over general networks.

Yeah I spent a few weeks digging into the applied category stuff but I ended up concluding that it offers no obvious insights into programming language design (other than the overarching idea of letting mathematics and composable patterns guide your designs).

Nick Smith Check out https://youtu.be/mwxknB4SgvM?t=787 (time code 787) of this Sussman talk to see if there's anything there for you in the Propagators idea. Your description above suggests you might have gotten wrong-footed by implementation details.

Jack Rusher I watched that section, and didn't see anything interesting beyond the general idea of constraint programming (which is all about refining constraints / accruing information). I have problems with more than just the implementation of their model: the very model itself demands language users to construct "networks of machines" (propagators), which I find unsalvageable. Why should a program consist of "cells"? They don't map clearly to human reasoning and logic. I'll assert the following: humans don't naturally describe goals (N.B: I'm not talking about steps/strategies) within a searching or measuring task (a computation) in terms of storing and retrieving information from abstract "cells" of knowledge; instead we describe the factors/quantities in our domain, and the relationships/constraints between them (including iterative/recursive relationships). Propagators seem to be adding arbitrary hoops and hurdles on top of that, by asking us to think in terms of machines and memories.

They may have reached this design by asking: "how do we design a language for massively parallel & distributed systems?" (as Alexey does in his thesis) and then falling into the trap of thinking that such a language must be based on a network of machines, so that it can "map naturally" to a distributed system. This is a trap that language designers have always fallen into: start by thinking about the hardware you're targeting, and then design a language that you feel is "appropriate" for that hardware. That might be necessary if you want to bake specific performance guarantees into your language, but you'll always end up with something that demands users to think like a machine.

If that's not the reasoning behind their network model then I'm not sure what is. Alexey explicitly opts-out of justifying the design in his thesis. He says:

I can therefore afford to present [propagator networks] on their own merits, with the promise that they turned out to actually work as my evidence of their validity, rather than documenting the tortuous intellectual path that led here.

Since they haven't justified their design, it feels arbitrary and ideological. I've grown incredibly wary of making arbitrary choices in my own work.

Stay tuned for the (imminently) upcoming episode of the FoC podcast, about Sketch-n-Sketch. We discuss such benchmarks.

Someone gave me a business card at Defcon for a similar-sounding service, something to do with needing ephemeral machines for, um, penetration testing, or maybe testing suspicious links. You might look for niche use cases like that. I suspect there's not a huge market for that kind of thing, though. I can't think of a use case where either DO or AWS lambda wouldn't be better.

It was suggested on HN that http://repl.it might do the trick. I expanded more on what I was looking for: https://news.ycombinator.com/item?id=26450076

Would Codespaces work for your use-case?

Wow @Eric Amodio this looks to be exactly what I was hoping to find! Thank you. I am now on the waitlist.