I'm curious to hear more about Pulumi. I had a similar question after looking at Hashicorp's Terraform CDK over the weekend: what's the point? It has to emit a static configuration. JSON basically. What does having a general-purpose PL buy you if you can't add a loop or conditional to the output? Pulumi seems the same, going by tutorials like https://www.pulumi.com/docs/tutorials/aws/ec2-webserver. Can someone point me at a more complex scenario that it can handle?

I'd like to create a host, then monitor its health for 60 seconds before creating a second identical host. Does Pulumi or any other infrastructure-as-code allow me to do that? If I'm reading it right, the declarative-code article above has the same issues with Pulumi.

I don't know exactly how Pulimi works, but I have some experience with Terraform (I use it on my last project as a corporate employee).

The point is that generally you define a desired state and ask Terraform to do all it can to create/modify the corresponding infrastructure. And for that it analyzes your code, checks that everything is OK and proposes you a "plan" which describes what Terraform will actually do: create new resource, deleting some others or just modifying some. Terraform then asks you if you want to proceed and then actually do the job. Of course, you can tell to Terraform to apply changes automatically without asking, if you are very confident about you code or don't care to break things, but generally this is desired thing to be asked because of the tricky nature of infrastructure management.

So I guess the scenario "create something, wait a bit, create something else" is not very common and is actually "not in the spirit". But if this use case is really needed, if I recall well there are some trick to allow doing this kind of pause. In the definition of the first resource you add a script that sleep 60 seconds, maybe with some monitoring, and make the second resource depends on this resource, which will make Terraform wait the end of the script before creating the second resource. The paradigm is to define resources and dependencies.

For loops, there is something that allows you to create a set of ressources with a single definition. I used it to generate thousands of VM. In fact you have a series of tools for "loop/conditionals" at the language level, but always in a spirit of "templating".

But after a bit of practicing, I felt sometime that I would have prefer to code things with a general purpose language. For things like getting env vars, managing some files, etc. I guess it is what Pulumi is about, and I am very curious about it.

Pulumi is very much like Terraform, it actually uses the Terraform providers behind the scenes (Pulumi does not generate files, as AWS CDK does). The only major difference is that you declare that desired state using an API of a general purpose language. But otherwise, Pulumi does exactly what @nicolas decoster said Terraform does — keeping that desired state, creating a plan, previewing it, applying it, etc.

Regarding Kartik Agaram's problem, I think the solution in Pulumi would be similar to the one in Terrraform. Define a custom resource that does nothing than just block execution for an amount of time and then to the health check, then use this resource as a dependency of the second host you want to create.

Going back to the original question of whether it would make sense to have a visual editor for k8s config files, I don't think it would help that much. The problem with YAML is not that it's YAML, it's that it offers no means composition and abstraction. With only a visual editor that all it does is to offer affordances for YAML concepts, things will be the same. But, if that visual editors starts adding some PL concepts, such as variables and functions, then it can be useful, but at this point I think it's no longer about YAML or k8s, it's just about structural editors in general.

For me, prototype = typescript. Parsimmon is a great parsing library to start with. Or if you feel fancy you can check out ohm which is a compiler-compiler.

https://github.com/harc/ohm

Meta 2 is a really cool paper about compiler-compilers btw. Here’s a nice overview: https://youtu.be/L1rwVBLHGiU

I think I'll try instaparse for my next experiment https://github.com/Engelberg/instaparse and https://github.com/noprompt/meander to manipulate the parse tree

If you are not sure about clojure for language implementation this talk might be interesting for you: https://www.youtube.com/watch?v=t8usj1fN9rs (since strictly typed languages are often considered "the best" for compiler work)

I’ve found this library useful in the past, especially for quick parsing tasks. https://github.com/orangeduck/mpc

Last time I did a language, I rolled my own tokenizer and lexer. I find things easier to understand that way. I know that Jonathan Blow’s Jai language uses the same hand rolled approach.

It has the nice feature of being able to just pass a grammar to the library, and it builds the parser combinators. Very quick and easy.

I find the question a bit underspecified, as which tools one might choose will hinge on both the specifics of what one is building and a variety of personal preferences. In my case, I like languages/environments that allow me to develop new semantics from within them, only worrying about adding syntactic affordances later. If one prefers that approach, it's hard to beat https://racket-lang.org

I use the compiler tools from erlang, leex, yecc, absform and the compiler module

I just use kotlin and wrote my own parser, the language feels so nice and expressive that I just keep coming back to it for everything.

Basically just find special tokens, create a index of [position, token], then recursively parse it breadth first. tho it isn’t a complete “language”, but a DSL language so it’s easier than full on lang :)

and weird, nobody mentioned ANTLR yet

I hand rolled the lexer and parser for Curv in C++, because I didn't want to fight tool limitations in defining the syntax or implementing error recovery. In retrospect,

• A regex-based scanner generator like re2c would have made the scanner easier to write.
• My hand-written recursive descent parser was easy to write and continues to be easy to modify. No regrets.
• After hanging out in FoC for a year, I have IDE envy. To properly implement completions and hints, I need an incremental parser. So maybe I should switch to the tree-sitter parser generator? https://github.com/tree-sitter/tree-sitter
• For me, the hard part is the back end, not the parser/lexical analyser. What are the libraries/DSLs for semantic analysis, optimization and code generation?

Whoa. This is really cool! Thanks Garth Goldwater!

I’ve been working in a project involving datalog as a durable database with p2p collaboration and sync.

I've read the article last night, awesome stuff there. It's been a while since I also started thinking that codebase should be treated as a portmanteau of code + database, so this article was right up my alley. However, it never occurred to me that Datalog could be used for the database, even though I played a bit with Eve. I always thought of using a graph database, such as Neo4j.

@Pete Vilter did you follow any resource for your Datalog implementation? Do you know of any tutorial-like resource that shows how to do that? I'm pretty familiar with unification I'd say, since I wrote a few Hindley-Milner type inference implementations, but I haven't yet come across a good exposition of how to write a Datalog interpreter.

@Ionuț G. Stan the main guide I had is a simple interpreter that a friend wrote, which isn’t open source. So, really the best I can offer up is my own implementation (simpleEvaluate.ts in the repo; not guaranteed to be bug free 😛) or these miniKanren implementations (a minimal logic programming language): http://minikanren.org/

Thanks, @Pete Vilter. I did a bit of research last night and I found some other resources. Some people, though, make a pretty important distinction between Datalog and Prolog. I'm yet to understand the differences, but it will be fun.

to be honest, I’m not completely clear on exactly what differentiates them — just that prolog is somehow more powerful / less decidable. there were some comments on my posts’s HN thread about this: https://news.ycombinator.com/item?id=23888656 (a bit over my head)

Some good links and explanations in that HN thread, thanks.

I've found that the term "datalog" is often used to represent an idea more so than a concrete implementation detail whereas Prolog is an actual syntax+implementation. But in my experience, here are some differences:

• Datalog tends to refer indexed 3-tuples, whereas prolog (and prepositional logic) deals with arbitrary N-tuples which tend to be prohibitively expensive to index.
• Prolog allows for arbitrary recursion (I think) whereas Datalog tends to guarantee polynomial-time execution. <- this might be wrong
• Datalog typically refers to a database where data is persisted whereas Prolog is programming language.

On the topic of Datalog implementations, I found the write-up by Frank McSherry really interesting. He describes the implementation of DataFrog (a bare-bones Rust embedded "datalog" impl) https://github.com/frankmcsherry/blog/blob/master/posts/2018-05-19.md

It took me a couple sessions to encode the ideas, but it is still fascinating! We'll likely be using DataFrog as the basis of many parts of our type system, too!

Are humans any different than machines?

We are. Our “model” does wetware based optimisation, morphing and most importantly localisation, which is something machines cant do while in hardware mode. Yeah, you can rearrange it in memory, but memory is 2D and that means huge overhead we don’t have. You need a multidimensional graph and mocking that in 2D is annoying as hell. And we are geared towards survival and reproduction, so we have “emotions” that reinforce our models

Can we just continue to make better models and achieve understanding or consciousness or intelligence?

Not this way. OpenAI has other stuff that is more geared toward better AI and intelligence. If we just continue to make better models but keep them shallow like this, only thing we will understand that intelligence isn’t pattern recognition.

@Ian Rumac I'd love to read some material about what you describe as "wetware based optimization, morphing and localization" — do you have any pointers to sources that go into depth on these?

Let me look it up, I remember a paper last year that was about how brain uses spatial localisation to optimise (kinda annoying now Neural networks are muddying up my search results 😂)

By “optimization” I mean about constant retraining and optimisation our brains go through.

Our “internal optimization agent” sees we have a graph G(n) of n nodes and a graph G(m) of m nodes. They get combined into a graph of k=(<m*n) nodes, while maybe not retaining the absolute combined precision, but probably having a modulator which is extracted as a separate graph.

For example, Gn is trained for a fluid (oil) and Gm is trained for oil. Then, they are merged/splitted into a model that recognizes “fluid” pattern and a “viscosity” pattern (our brains are constantly looking for patterns. constantly. they are our primary source of optimization, i believe that this skill is critical to our brains evolution and that that is why geometry is important in our brains arrangements). Let’s say you input lava, it would be recognized by both models, one recognizing it’s a fluid and the behaviors of fluid and one that can tell you the viscosity of it. Hope I explained it OK, will try to find papers, they do it better probs (i mean, this is my conclusion from reading multiple of papers, so its dot connecting conclusion from multiple sources).

(https://gtr.ukri.org/projects?ref=MR%2FL009013%2F1#/tabOverview here is one that dabbed into that, but can’t find the one I wanted, an article is at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4598639/ that is a nice intro)

Also, the more I read about GPT-3 it seems like:

“Hey, remember that car we invented? Well, after making the engine a 1000 times bigger we get a lot more horsepower out of it!”

@Ian Rumac Thanks for digging this up; looks promising. I have the same issue, often running into neural network stuff when searching for brain research.

I’m specifically interested in categorization and how semantics manifest in our brains, schemas and frames, essentially the parts between the biochemistry of the brain and the philosophy of thought and reasoning — here’s a good overview from Lakoff: https://youtu.be/WuUnMCq-ARQ

If you or anyone else comes across research in that area, please send it my way.

Reading the conversation in this thread kinda makes me wish this Slack also had a philosophy channel 🙂

Stefan Lesser oooh interesting video, will check it out later when I have time! tldr as far as I see its about research that tries to find primitive model our brain recognizes and which are used as building blocks to build more advanced models by having a few types of interactions between them?

@Ian Rumac Yes, that describes the gist of it. It’s focused on language, which is only one of many parts of thinking — arguably quite an important one. Helps to be familiar with the work of George Lakoff and Mark Johnson on metaphorical structuring. There’ll be plenty of pointers in the lecture.

No clue who any of those people are but lecture sounds awesome, tnx for the link 👍

Re: affecting programming in this frame - less hype chasing, more deep knowledge, research and standardisation. The ego stance of “me and my ass” is affecting programming world a lot, especially through middle management.

re: article - it’s surprisingly bad, 99 cent philosophy. I wrote a lenghty comment on HN about it, but I don’t think this article deserves even a minute more of my time considering how badly thought out it is. https://news.ycombinator.com/item?id=23899952

I quite liked this article. It's stayed with me for the last couple of days.

@Ian Rumac A few years ago I may have felt the same way, that it was just empty words and not very actionable. But I have a little more of a sense lately of how progress happens in the softer aspects of human endeavor, where the problems are harder and incremental progress is harder to measure. And yet, progress can happen if a new way of adding one more level of structure (such as decomposing "long-term thinking" into 6 pieces) can help people talk about things in just a little bit more detail, and influence how we make decisions. I think this dynamic may be a big part of why people tend to over-estimate what they can do in a month but under-estimate what they can do in a decade. Imperfections are easy to see, and it's easy to forget that progress can compound.

Even for past me, though, there's one part that's unambiguously great: the way it lays out possible teleological goals of human existence:

• God, the pearly gates and heaven.

• Material goods.

• Colonizing space. (see https://www.resilience.org/stories/2013-09-05/the-next-ten-billion-years)

• New to this article, for me: keeping this planet thriving (though I've encountered echoes of it before in https://www.amazon.com/Manifold-Origin-Stephen-Baxter/dp/0345430808, now that I think about it)

Just section 6 is worth the time spent on this article, IMO.

That next ten billion years link is a hell of a story... extremely depressing and yet oddly beautiful. It gives me "https://www.planetary.org/explore/space-topics/earth/pale-blue-dot.html" vibes. (It's always worth re-reading Carl Sagan every once and a while imo 😜 )

This linked article is also quite thought-provoking with its radical new visualization of a city's usage and distribution patterns.

https://www.theguardian.com/world/2020/apr/08/amsterdam-doughnut-model-mend-post-coronavirus-economy

I'd be curious if writing code in your native language is a net plus or net minus. Plus, use words you like and are used to. Minus, 70-90% of the world can't look at your code. If your code looks like this

関数 距離（頂点ー、頂点二）｛

　返す 平方根（足す（二乗（頂点ー）二乗（頂点二）））

｝

Only people in your own language can read it. People not in that language can't even type it . If you ask for help you can only get that help from a much smaller set of people.

Also what do you do about libraries? Do you need a localized version of every library just so you can keep everything in your language?

Of course I say this as a "privileged" English speaker. I have no idea what I would feel like using all these languages that are generally based in English if I wasn't fluent in English.

Also, I don't know about China or Korea but as an example when non-ASCII domain names became possible lots of people in Japan though kanji based domain names would take off. They didn't though. It's been 15+ years since it was possible but almost no companies use them.

When I was at university I studied maths, and the faculty was a bit out of town, so when we had lectures it took about 30mins to walk back. We (very!) occasionally discussed the material on the way back, but found that it was almost impossible to even talk about maths without a pen and paper. I remember thinking that a mathematician isn't really a human. No human has ever demonstrated any real ability for maths. A mathematician is human-pen-paper system, like a cyborg.

Counterpoint: supposedly Euler did math even after he went blind

🤔 hmm yes a good point... Although I wonder if something like this might have been going on https://youtu.be/6m6s-ulE6LY (watch to the end)

That is, it's difficult to know if the augmentary effects had already happened and been internalised

Also, you know, it's Euler 😂

That’s interesting Chris Knott cos some code I write is like that I need pen and paper.

This was as fantastic a short story as I expect Ted Chiang to always be. Thank you Chris Knott! Hadn't seen it before.

I liked the ideas about how using a tool heavily actually affects your brain. I am sure my brain has adapted massively to having a computer at my fingertips all the time. I don't remember stuff in full any more, what I do is remember a compressed link/pointer, that I will know I will be able to use to recover the information - via the internet - if necessary.

Not deliberately, I think it's a kind of optimisation my brain's adopted. I find it very, very frustrating sometimes when I can half remember something

My mind is blown by the idea that the transition from oral culture to written culture is an example of a broader phenomenon where the shift in quantity of memory turns into a qualitative change.

Here are TC's notes from the back of Exhalation

Perhaps you would enjoy Orality and Literacy (I don't know anything about it other than this mention!)

https://monoskop.org/images/d/db/Ong_Walter_J_Orality_and_Literacy_2nd_ed.pdf

But the illumination does not come easily. Understanding the relations of orality and literacy and the implications of the relations is not a matter of instant psychohistory or instant phenomenology. It calls for wide, even vast, learning, painstaking thought and careful statement. Not only are the issues deep and complex, but they also engage our own biases. > We—readers of books such as this—are so literate that it is very difficult for us to conceive of an oral universe of communication or thought except as a variant of a literate universe.>

I think people have talked before about how this must apply to us, as programmers. This is why I am sure a key tool in FoC will be studying the experience of highly intelligent, computer illiterate people. How easily can, for example, Ruth Bader Ginsburg learn Python? What are her fail modes? I would be fascinated to find out. (Apologies to RBG if she's actually a computer wiz!).

How about this - could a remote tribesperson from a oral culture be taught Python, skipping normal written language? What about a completely visual PL?

Ivan illich wrote "ABC: the alphabetization of the popular mind", on the transition from orality to literacy. I found it insightful.

Orality and literacy is a popular topic—Neil Postman talks about it as well, across all his work

Nice! I always love it when we as programmers can acutally do some hacking to circumvent artifical problems of monopolies 🙂

(Don’t know if sharing to #general notifies the recipient, so I’m adding an Chris Granger for good measure)

This was from an internal thing I was writing just to layout some of our thoughts. 🙂

In particular, I was contrasting the "summon and place" style of interaction that nodes/blocks have vs the "locate and act" that you have with text:

Text on the other hand works just like normal painting. You locate the area where you want to do something and then act directly in that area. This places more emphasis on the result rather than the change itself. Afterall, the > +> node isn’t what I care about, it’s the actual behavior that we end up with that matters. This fits more of a noun/subject-first way of thinking where you first find what you want to work on and then act directly on that thing. 

Visual density aside, I believe the difference in interaction style is a big part of what holds these other types of editors back. In most situations, locate + act is a much more contextual and expressive way to work. It’d require significant advantages to offset all the awkwardness and busy work that comes with painting off to the side.

We're trying to take a bit of a different approach this time and aren't saying too much publicly until the explorations have settled down a bit. We'd like to already be some level of "successful" first instead of showing lots of things that just end up being cool demos rather than a fully fledged system.

The good news is that after evaluating quite a lot of stuff I think we're ready to put a stake in the ground and move forward. We've learned and created a lot since Eve and I'm pretty excited about where things are headed. 🙂

Exciting to hear— can’t wait to see where it goes. Thanks for the extra context!

I was intrigued by your comment from a while back: “the language I designed based on everything we’d learned through Eve was dataflow with state machines embedded in it, but it turns out if you flip that the other way around you get something pretty neat.” Curious if that had legs, or if the stake landed somewhere else? (Of course if you’d rather not say yet I respect that.)

still very positive on both variants, though where we're landing is a bit muddier on which one is the "embedded" one

in some sense they serve different purposes. I've come to think of hierarchical statecharts as more a form of structure rather than as actual behavior

one thing that I really took from Eve was the idea that what we really need is a system that somehow mixes declarative and imperative expression seamlessly

interestingly dataflow is kind of both, same with statemachines

but I think they address pretty different aspects of the design space

StateCharts with a couple of modifications are a pretty good way of directly describing structure and certain kinds of stateful processes. Dataflow is quite a bit better at describing computation, but at the cost of being a pretty poor structural representation.

So the question is which do you try to bring to the foreground?

Regardless of the choice, the idea of states (in the statemachine sense) being a core primitive of the system seems to be a really good one and makes the expression of many programs significantly better

where "better" = clearer to understand, easier to write, and easier to confidently modify from our experiences.

it'll be interesting to see to what degree our editing workflow will focus on states/dataflow vs having them melt away into the background. I don't really have a good sense for that yet. 🙂

Sweet. Sounds like you’re really onto something. State machines are a cool “tool for thought” in that they can actually help an inexperienced programmer walk through edge cases and design more robust programs. I’ve been kind of barking up a similar tree for an internal project (combining state machines and dataflow), and finding the dividing line sometimes confusing. In my thinking right now it’s something like control flow vs. data flow. The weird thing is that HSM has a concept of “state machines all the way down”, but data flow does too, and you can kind of mix and match as you go down (but it’s not clear to me yet how useful this is in practice)

yeah

simulink is an example that focused on dataflow with HSMs embedded to good effect I think

one interesting thing is that we found that in dataflow you still want to represent control flow meaningfully within the same space.

The latest work in VPLs for game engines have also found that and started doing vertical control flow, horizontal data flow, which I think is really effective

it lets you read the "spine" to get the main idea of what's going on and then look off to the left for the details if you need to

a bunch of our explorations centered around digging into that further and trying to find nice ways to address some of the remaining downsides of the node/wire paradigm

Nice, I’ve had the same vertical/horizontal thought but didn’t realize that about game engine VPLs. Most of my prior art is Labview plus a bunch of demos I’ve seen here. I’ve used matlab but never simulink, I’ll take a look. When you say HSM are good for structure, one way I’m thinking of them is as a sensible way to perform dynamic switching of the links between dataflow nodes. Is that what you mean or something different?

yeah, that's a way of thinking about their purpose from a behavioral structure standpoint. You also need the structure of the entities themselves, which we found can map pretty nicely to states as well if you squint

in a turtles all the way down sense, you tend to have entities and states interleaved throughout the hierarchy

so we just made everything a state

and explored that

very cool model and lots of goodness to be had there

Love it. I resonate with everything you’re saying, but I think your picture on implementation is much clearer than mine. Hopefully we can just swap out my experiment with Eve++ once it’s ready for primetime 😀

🙂

Did you explore and abandon any areas that you think are dead ends?

it depends on how you define dead end. I think most of the things we looked at could/have found success in certain domains

so at some level it depends on what your goals are

I think the "summon and place" workflow is a bit of a dead end, but there are ways to address it

e.g. having composite "formula" nodes and such

the conclusion we arrive at in the document I was quoting before more or less boils down to the idea that making meaningful progress requires finding combinations of things that have largely been considered dichotomous

declarative vs imperative, text vs graphical, implicit vs explicit..

there's a ton more to explore down any one of these, but I think choosing one and assuming it'll be enough likely is a dead end

If we step out of the domain of computers, there's more than enough evidence to start to see what the pros and cons of each end of the spectrum are

unless you can find a way to offset those cons, you'll end up with something that is a difference in degree rather than a difference in kind

I remember a while ago I think you said how something you realised with Eve was it was too hard to go straight for new people/expanding the market, and you need to get people who are coding already on board to be sustainable

(I might have misquoted that a bit)

Is that still how this new project is going?

we kind of always knew that going after end-users requires first getting more technical folks on board, the more recent belief is that I've become far less convinced that going after end-users (unless for pedagogical purposes) is the better path

if we want to frame things in terms of global utility, it's likely the case that over a shortish timeframe (< 20 years) you'll gain more going after people who are already aware they need to program

the new project is motivated by the question "if we only care about making Josh and I superhuman, what would that take and how far could we get?"

I think we've implicitly expanded that to "any sufficiently motivated person who already programs or wants to," but I think that's a reasonable place to start

There's a difference between building tools that will accelerate people to crazy heights and convincing a group of folks that flying is interesting to begin with.

Sounds like a fantastic goal. Curious if distributed systems and realtime applications are in scope. Same with graphics.

Yes, yes, and yes! 😄

Amazing 😄

Actually, depends on what you mean by realtime applications

in principle it would be fast enough to do hard realtime stuff, but I'm not particularly worried about synchronized clocks and hard budget cutoffs

Lol me neither. I’m thinking more IoT and games. Though robotics would be cool too

ah yeah, those would be fine. A flight controller? probably not?

it could ostensibly be extended in that direction if you want I guess

But a drone remote control 👍

yep

How far are you thinking on the graphics side?

all the way to shaders

one of our tests is if we can build a high fidelity game with it

I’m gonna cry

Lol

I’ve been wanting something like this for a long time

haha, well you don't know what it is yet so don't get too excited 😉

we're taking some interesting directions that I suspect will be a bit controversial

but we have a mounting list of evidence that's giving me more and more confidence

Between eve, the team, and the stated goals I feel like I have enough to get excited about 😉 but of course will suspend judgment. Any nuggets about the interesting directions?

a very high performance system that removes almost all the power you would have in a normal systems programming language is one example

you have no control of layout

because of that decision though, we can do utterly insane stuff

e.g. your whole program is laid out to optimize cache coherency based on an analysis of the access patterns actually present in the code

🤯

I was matching hand-written, cached optimized rust - but our code looks like a domain description rather than the horrid stuff you normally do.

Ok, I love both of those decisions

it was running in JS 😛

another side effect of that choice is that you can just grab a set of things and say "these live on the GPU"

to your distributed systems question, you can also say "these live in this postgres database" or "these live on a server"

We talked about the "world-scale computer" that we wanted to create with Eve. A big part of what I've been trying to figure out since is how we could actually have it.

I don't intend to run into performance issues again. 🙂

This all sounds so right, for 99% of use cases. Any rough feeling on a beta at this point? 1-2 years? I’m sure a lot will change as you go

not sure when we expand it out, but I'm expecting to show at least this crowd stuff in the next couple of months

Awesome. Well thanks for sharing some tidbits, and I’ll be keeping an eye out

👍 😄

The idea of looking at control flow first really resonates. On my project I trace all events and I've often found myself doing a blunt grep label trace as the first step of debugging a problem.

When I'm programming, I usually think about code in terms of both control flow and data flow. Most dataflow systems seem to only have one type of wire. Are there systems with different wires to represent different flows through the graph (eg blue wire for data flow, red wire for control flow)?

Max/MSP and Pure Data have one kind of wire for messages (discrete data), and another for audio signals ("continuous" data — not truly continuous, but discrete samples of a continuous function at a sufficiently high sampling rate).

Many visual programming tools will assign a type to their ports, and whenever a wire is connected to one such port, it can only be connected to other ports with a compatible type.

In my project Hest, wires are used both to convey data (and there may be different kinds of wire, with different appearance and properties and modes of interaction based on the type of data being conveyed), and to exist as "edges" in the sense of 2d vector / 3d graphics, so in that way they're both an aspect of the code and data literal.

Gorgeous

It seems quite similar to Capstone from ink & switch lab https://www.inkandswitch.com/capstone-manuscript.html

Actually looks like it's called Muse now https://www.inkandswitch.com/muse-studio-for-ideas.html

It does. It mentions the project from Ink&Switch as inspiration.

How can I download Muse? Can’t seem to find it on the App Store. Love love things like this. Also had a thought yesterday that perhaps coding and even innovations like visual programming are held back by the keyboard mouse input paradigm.

How different would code be written if it could be written with a pen?

I think this point was brought up when I was reading why APL failed - no standard characters. Free form writing with a pen might make that be less of a headache.

Chris Knott thanks for the links!

How different would code be written if it could be written with a pen?GRaIL comes to mind https://www.youtube.com/watch?v=QQhVQ1UG6aM

That’s pretty cool. Wonder it hasn’t been revisited and isn’t more mainstream

Does anyone have a Remarkable tablet? There is a fairly active dev community for it I think

Ope https://museapp.com/ looks like you sign up via email on their website

Yes, Muse'll just email you an invite, I tried it last month. It's more just a zoomable "canvas tree" whereas I'd personally like to see more backlinky stuff as in the Semilattice demo (🤞 it gets built). Muse is still super cool and will probably be my goto once they polish the UI and implement handrwiting search (it's the only reason I can find anything in GoodNotes).

Came here to post this, but you beat me to it! Nice.

The tl;dr is that they're running a contest with a $10k prize, where you design and submit a novel programming language (ideally an unconventional one).

They're also looking for judges, if you are interested in that. Current judges are some names you might recognize: Mary Rose Cook and Jordan Walke.

It would be interesting to hear what a novel language can look like!

New lisp dialect (not that novel but still) comes to my mind, maybe prolog-like or something more crazy 😁

I guess they talk about textual PLs only, right?

Yeah, I think it has to be a textual language. But I imagine you could make a spatial language like https://github.com/batman-nair/IRCIS

(Though maybe not with animated execution)

Wow, that’s impressive!

Maybe someone revisits APL

Replit ceo says it can be graphical? not sure if I read the tweet correctly: https://twitter.com/amasad/status/1285667493034565637

Exciting to see this jam, either way.

Displaying graphics is one thing. Making a nice GUI for graphical programming is a waaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

aaaaaaaaaaaaaaaa

[trim]

aaaaaaay different thing.

Is parsing all it helps with?

Steve Dekorte Not sure what you mean by "it"

Ope If you make an APL, the first thing it needs to do is 3D print the special keyboard you will need to use it 🤣 (Sorry, I couldn't resist making an APL joke. Those APL keyboards http://ageinghacker.net/hacks/apl-keyboard/apl-keyboard-1.jpg)

I was actually thinking more like using OCR instead to recognise custom characters. So you write it 🤓.

Actually, thinking about that idea more seriously, we have much better auto-complete these days (shout out to tab9 again lol). APL with a decent autocomplete widget might be interesting...

Corporate hackathons are more about benefiting the corporation with publicity and hype, which can only be created by the labor of a critical mass of entrants. But despite the incredible value of this publicity - and the allure of glamorous prizes- virtually none of the entrants get compensated.

In fact, it is an equation sorely in the corporations favor: the more entrants, the more hype, but since the prize pool is fixed, the cost never goes up.

I would recommend anyone considering joining any hackathon to think hard about the power dynamics at play, and the material value of your time spent participating, especially when a hackathon requires you to use the corporation's products(consider particularly that for a public publishing platform, the more usage a product gets the larger its user conversion pipeline becomes). This isn't a novel observation - much has been written about concerning the exploitative nature of hackathons.

And if as a potential participant you want the good-natured competition and dedicated time for hacking on a cool idea that a hackathon affords, it's always possible to organize your own jam, centered around and for the community, without any ulterior motives at play(EDIT: look at game jams like https://ldjam.com/ for a very successful model of how this can be done).

(And also, I think the word "grant" should be reserved for actual grants- not marketing contests. I like the idea of grants, and if a company wants to give grants, it should do so in a way that requires minimal effort by the participants, and focus on underrepresented people that badly need them)

I had similar thoughts but censored myself. Perhaps I shouldn't have.

Kartik Agaram thanks for expressing that. I really like what replit does, the sponsorship(esp. of the foc podcast) and community work they do, and they seem to have a lot of great people there, so I also felt pressure to hold my tongue*. But after thinking about this since when the announcement was posted, I decided I don't like seeing the community being exploited more than I don't like rustling feathers.

*to clarify: I meant this as in, I felt bad about criticizing an organization that I believe is doing quite a lot of good.

I think this contest is best for (eg) highschoolers, who don't yet have the network to throw their own jam, who (in theory) have a lot of free time, and who might need something to motivate them to take on a new challenge that'd be good for their portfolio even if they don't win. For folks in this community, we might stand to gain more from being a judge than being an entrant — maybe. And yeah, totally agree, some details of how Replit are structuring and promoting this contest are gross. But I think that's just capitalism — we here have the option to not participate in it, and I don't think that they do (what with investors to satisfy, etc.)

Love that you shared these thoughts, Yoshiki. Even though Replit are a benefactor of this community, their support should not shield them from criticism, especially criticism that we could all learn from that will help us do better if/when some of our projects are turned into businesses and we're faced with marketing challenges and the pressures of capitalism ourselves.

(And if Replit decide that they don't want to fund a community that reflects honestly and critically on what they do — though I expect Amjad et al would be fine with it — then there will be other ways to pay for the podcast, fear not)

Yeah, and to be clear, I think they're good people who can accept criticism.

Fwiw, Amjad has heard and responded to my criticism(albeit a version I shared that was much more concise to fit into a tweet):

https://twitter.com/amasad/status/1286092042179813376

It's easier to show up at some corporate jam and meet new people than to organize one yourself. You don't have to find the space, get the food, clean the place after, etc etc etc. Plus you'll likely meet people you don't know.

That said I don't personally get the idea of jams/hackathons that take place online. I can make something on my own at home. Saying "this weekend is abc jam! just upload your submission online!" doesn't motivate me all. All I want out of a jam is to spend time with actual people, make some new friends, see a new venue, that kind thing.

When I was a teenager, these sorts of contest-jams were fun as I learned 3D modelling. A good way to get some audience (and thus feedback) that my work wasn't yet good enough to earn on its own.

Yeah I mean, there is a value esp. in the physical events. And I don't want to discount the individual moral calculus people are making in their heads when they decide to participate, which I think is often quite sophisticated, and perhaps a lot of what I wrote is more of a reaction to my own past naivety toward these events(and I think, there is some unnecessary paternalism in my original post I should reflect on...).

My intuition is that the computing community in general would be healthier tho if there was a bigger focus on community-driven jams than corporate hackathons, but I need to do more research and thinking to substantiate this.

So what about a Future of Coding jam driven by the community and sponsored by... Replit? 😄

More seriously, there is something interesting in the Replit announce:

If languages are mediums of thought, and there are more and more people interested in expressing themselves digitally, shouldn't we expect to see many more types of languages?and later:

Programming languages are undoubtedly hard to make. They require a combination of design skills and technical chops. But for some reason, language designers are typically solo hackers. But what if we made programming languages easier to make? If we showed that it is possible for anyone, even those with limited experience, perhaps especially beginners, can create and experiment with languages? Wouldn’t we get a lot more exciting and fresh ideas?It is true that few people try to create language. And I guess it would be good it was different. As an example, I thought about trying to actually do it myself after years of programming. I guess I felt unlegitimate to do so before.

But, of course now I see it can be fun (and that you can even learn thing from it). In fact during workshops with kids sometime we start approaching programming by asking them to create programming languages: one kid plays the role of a robot that has to move on a landscape and an other kid writes down some "formal instructions" of her invention on a paper, and the robot-kid follows the instructions to reach a particular place of the landscape.

So I agree with Replit that it would be nice if creating languages was easier and more widespread.

i guee they are using openai api 🤔

Thanks for it!

Some thoughts on this matter from Hillel Wayne: https://buttondown.email/hillelwayne/archive/syntax-highlighting-is-a-waste-of-an-information/

This is the only pong-like demo I'm aware of: https://www.youtube.com/watch?v=Gy5m091fOTU

THAAAAAAAAAAAAAAAAAAAANK YYYYYYYYYYYYYOOOOOOOOOOOOOOOOOOOOOOOOOOOUUUUUUUU

Fuck these people's SEO. A friend of mine correctly guessed that it had "deep" in the name, but 45 minutes of googling of that kind got me bugger all

(Newcomer note: we don't slag people here, even if they're outside the community, since they might join in the future.)

I had a DM conversation with the creator of this demo and got some backstory you might be interested in. (Paraphrasing, so apologies to the creator if I get this wrong. Also, I believe they wish to remain slightly anonymous.) They originally built this demo in C++, inspired by Bret Victor. They made a video of it, posted it online. (That's why you couldn't find this via search — their "website" was only ever just a demo video on an otherwise empty, and now offline, landing page). As they continued to work on it, they realized that they didn't have a good way to scale up their approach to handle more complex cases, not without a tremendous amount of work they weren't prepared to do. Also, they saw what the folks at Luna (now https://twitter.com/enso_org) were doing, and felt that that was a much better approach — and that they actually had the funding necessary to make a go of building a novel visual programming environment.

Frankly, I disagree with the assessment. I think the DeepUI prototype was taking a far more interesting and potentially fruitful approach than Luna. To me, Luna is (was) a way of making an ML family language much nicer to work with, but it is not truly visual programming (and neither is Max/MSP, etc). Whereas, DeepUI actually is visual programming in the truest sense, and that, for my money, is a very exciting space just waiting to be explored. There absolutely are ways to scale this sort of a model up handle arbitrary complexity — but, I agree, it will take a tremendous amount of work. (Though, certainly, not as much work as has gone into gcc, JVM, or v8)

Was there ever a usable demo of this or just the video? Even in the pong demo there must be a load of logic not represented on screen

Just the video. I believe they continued hacking on the project in such a way that it no longer resembles the demo, and no longer runs, and is not in a publishable state.

Thank you for that Ivan, that is interesting!

I agree that DeepUI is more interesting than Luna, but I'm not surprised they couldn't figure out how to make it scale.

So far apart from Bret's work I am most impressed by Chalktalk, which I'm sure you know about. It has problems scaling too.

One of the reasons I was asking about DeepUI (and why I joined this slack) is that I am going to make a presentation on, surprise surprise, my own visual programming IDE ideas 😅 I hope to make it a bit more scalable... although I realize the odds are stacked against it

Cool! I hope you'll share some of your work in #feedback, or in #two-minute-week if you want to make short (2 minute) weekly updates as you go.

I'm aware of Chalktalk, but I haven't actually seen it. Do you have a good link?

Ahh I’m so glad you found the project @hamish todd! (I was the guy who replied to you from the enso twitter, spent a good while being like “wait did I miss someone making pong in Luna?!? and hunting around the interwebs!)

Heh thanks Joe. To be clear I think Luna is more scalable than DeepUI too 😄

Chalktalk: so the deal is that Ken Perlin has given the same presentation using chalktalk about 6 different times on youtube. He is a deep thinker on the philosophy of all this, almost as much as Bret and Alan, and his thinking around chalktalk is worth listening to; this 2017 presentation is as good as any https://www.youtube.com/watch?v=xuzrF_82z7U and this (timestamped) link shows you where it was in 2011 https://youtu.be/O9b-rtrcPEA?t=2707 showing a nice AR approach

• Business Model

Hosted SaaS and on-prem — cool? Whether this is worth paying for depends on what you'll use Enso for, so I guess this means they're going to (perhaps subtly) focus Enso on server-y stuff rather than GP programming. (Edit much later: I misread this as hosting the output of Enso, rather than hosting the tool Enso itself. Hosting the tool makes a lot more sense.)

Marketplace — this is my "if I can't think of anything better, copy Unity" idea for monetizing Hest. I hope I think of something better, because this feels like the sort of thing you should do after your tool becomes wildly popular. Otherwise, you end up with a bit of a ghost-town effect, and I've certainly been put off of participating in marketplaces because of that.

2. Variable Bindings = Lines

In other words, there are no variables. There's no way to put some data down, run away, come back later, and pick up the data where you left it. The data has to flow immediately from producer to consumer.

Perhaps they have some idea about a database or tuple space or some such in the background. But no mention of that yet.

Yeah, they're going to have to create some kind of value container for #2. You can't get very far without it and it's one of the worst parts of dataflow-centric systems. Part of the problem is that if all references need to be represented as lines, you get really nasty graphs the moment you have stateful containers like these.

Marketplace strategies pretty much always fail unless you're an incumbent

The way I've been thinking about #1 is paid SaaS for the tools with optional hosting (+ some very compelling reasons to use that hosting), but you can take the resulting artifact and run it wherever.

Any time your business model is hosting, you're now just an AWS reseller, and that feels like a business that can be shut down without warning at a moment's notice.

It worked for Heroku, but... there is already a graveyard here.

yeah, just hosting doesn't seem to work very well, but tool integrated hosting seems to workout fine. Webflow, bubble, retool, etc

Well, we'll see how many of those live past the end of their funding rounds. Webflow are at $70m, according to crunchbase, which is.. just mind-blowing to me. That's too much.

Anyway... hot takes!

3. Seems interesting, but there's nothing here about what this looks like. Do nodes with an error as part of the type signature turn red? Can you do any debugging on the visual side?

Moreover, the language was designed such that, even if you create syntax errors, all expressions not affected by the error will still work. That is that, if one of the nodes in your graph has a syntax error, the other nodes still function correctly. This sounds like an obvious requirement for a visual language, but is currently not implemented in any textual language.

That's cool, but that's how JavaScript works. You can have a function that throws an error or fails to parse, and code that's outside the stack of the thrown error or the scope of the syntax error (but in the same JS context) will continue to run unaffected. They probably meant to say "not implemented in any textual language ill with the type disease" :P

(Gotta be spicy or these aren't hot takes)

all of those companies are/were cashflow positive, fwiw

Ah, nice. I tried to verify that but couldn't find that info. Is this just something you knew offhand, or was there somewhere I could have looked?

know the founders for most of them, bubble was bootstrapped

so yeah, no secrets, just network 😞

I do wish there was more openness around business models and such, but I get why there usually isn't

interestingly enough, I've found indie game devs to be fairly open

watch enough GDC talks and you can get a good sense of outcomes in that space

Yeah, going back at least as far as 2D Boy being totally open about the financials of World Of Goo.

Hot take 4.

will provide you with a widget for that type

Good! Hopefully there's a way to define new widgets that are of an equal class to the built-in ones.

We have created a sophisticated > https://github.com/enso-org/ide/tree/main/src/rust/ensogl>  capable of displaying millions of data points in an interactive fashion, in a browser. This will allow Enso 2.0 to display much more data than any solution we know.

I don't care about how many "data points" it can draw unless I'm building a scatterplot. Can I work with 3D models? Vector graphics? Texture maps? Animations? Can I build user interfaces?

I always struggle a bit with statements like:

This will allow Enso 2.0 to display much more data than any solution we know.

Those kinds of sentences pretty much only open you up to being wrong, while providing no real value.

they make people want to prove you wrong and derail the actual point.

I love the idea of "just render it all in GL", but boy is that a big pill to swallow. There are so many things we take for granted that you have to do, and do correctly, to make something even remotely useful in the real-world.

Things we think are trivial, like text rendering, can take you years to get right and even there you'll likely be picking the lesser of a few evils.

As an example: font rendering in both unity and unreal still sucks and they have functionally infinite resources.

When we were trying to ship Visual Studio 2010, there were several people solely dedicated to font rendering in WPF. Fonts were still blurry in some circumstances all the way up to the last couple of months before going live. As I recall, they'd been working on it for almost 5 years at that point. 😕

This was not as spicy as I expected when I opened the thread 😂

Thanks for the takes Ivan Reese, appreciate your thoughts and insight! Some of these I can get you more info on tomorrow (it’s late here) - the error stuff for example, the visual part of that is just straight up not implemented at the moment, but there’s a vision for what that should look like.

Trying to strike the balance between being spicy, while still being constructive and not being mean :)

Those kinds of sentences pretty much only open you up to being wrong, while providing no real value.

Hahaha yes I totally agree, something we have to work on - after a certain amount of time talking to investors and funding bodies, I think it’s hard to turn off. I’ll try to be a better hype-net 😂

I particularly like his phrase: "systems literacy"

I https://www.lunchboxsessions.com in educational games, and this talk resonated a lot with me and my coworkers. It echoes a lot of things we've arrived at over the years. We're big on the idea of "design the gameplay first, then figure out what you can use that gameplay to teach" as opposed to the reverse. We also have a huge focus on systems-thinking, since we specialize in teaching troubleshooting skills. We also also strive to impart the "spirit of engineering", in addition to familiarity with particular technical matter.

One problem that we continually run into is that our biggest clients expect their educational software to fit a very specific, very familiar format. They buy educational software primarily for compliance reasons, not with an interest in helping people learn and do better at their jobs. (Worth noting: these clients are multinational companies, mostly focussed on resources, manufacturing, or transportation — business not known for being progressive.) They want multiple choice quizzes. They want slideshows. When we build things that learners genuinely enjoy using, we often have trouble getting higher-ups to deploy them across their org.

So I bristle at Blow's contention that there aren't people out there trying to make these sorts of games. There are! It's just not a guarantee that if we build it, they (customers) will come.

I work at one of those multinational corps interested only in slideshows and multiple choice quizzes for compliance sake, and "education" is probably the most pointless hours I spend all year (and that is, painfully, really saying something, compared to other useless tasks I trudge through)

I would have no idea how to break through this wall of set expectations, other than to produce something too good to ignore / that resets the marketplace.

That's a completely fair criticism! Especially in this community, people are obviously working on these problems.

In Blow's defense, he doesn't blame it all on the creators. He explicitly states how this kind of paradigm will never be bought by the current education establishment. It's the section where he compares the current education system to Cable packages from the 80's.

His suggestion is to ignore that market and go to Steam or the more general game markets instead. It's not a great solution honestly... You have to pay the bills, and Steam just can't compare to these big corporate contracts. The incentive structure is working against you hard here! It's a hard problem for sure.

I got into all of this via video games and am a big fan of Jonathan’s principled approach to his work. I personally believe there’s immense value in embedding expressive systems within a more ‘traditional’ game. For example, Minecraft taught many the basics of circuitry, Boolean logic, and even composability, modularity and systems design!

Ivan Reese for some reason, I’ve never actually seen what you work on. That’s so cool! I’m really curious what workflows you have for building the interactive animations. I noticed it’s all SVG. What’s the path from designer -> engineer look like?

(As an aside, I was watching the video and made it to “Explanations made simple powered by…” and my mind sadly autofilled with “AI”, and I was very happy when I got “interactive animations”.)

@Ray Imber "Ignore that market" — Totally, and that's basically what we've done. The https://www.lunchboxsessions.com I linked is something we built to run around the establishment and break down the back door.

(You probably know this, but I'll say it for the benefit of anyone reading who doesn't know...) Most companies have a Learning Management System (LMS) and expect to be able to buy modular slideshow+quiz content to plug into it, and then they assign that content to their employees whenever the mandated compliance schedule or some special circumstances (eg: a lawsuit) requires them to do so. To play in this market, you need to be B2B and by the book.

Our strategy for LunchBox Sessions is to force companies out of that mold. It's an iPhone-esq walled garden, that is meant to be discovered first by the employees who will be taking the training, who then tell their managers "This is really good, please buy this for us". Those managers then say to us, "Put your stuff in our corporate LMS", and we say "We can't". Occasionally, the company will relent and buy our material on our terms. The employees using the software love it, and we've won a customer for life. Often though, the company says "too bad" to their employees, and so we just have to wait for them to come around.

This "grassroots" / word-of-mouth approach to sales led to a lot of really interesting design choices at pretty much every level, all the way down to how we plan gameplay and educational objectives and such. Too much to go into here, but (tying back to the original post) there are definitely ways to improve the educational software market — they just, as Blow advocates, have to come mostly from outside the current practice.

Will Crichton The cutesy answer is that the path from designer -> engineer is not travelled by our content, but rather by our content creators. ;)

I'm the only formal programmer at our company, and I don't do any of the work on our content — I just build tools (and... all our web services, from visual design to database optimization, /curtsy/). When we hire a content creator, we hire only for art skills, and then train them how to program well enough to add logic and interactivity to their artwork — basic procedural code: variables, conditionals, calling functions, but not necessarily even arrays or maps. That training process takes about six months (most of it focused on domain-specific stuff), but they're usually able to build simple simulations pretty much immediately. We typically have 3-6 artists on the team. We'd happily train them much further in programming, but none of them have wanted to learn that yet (can't blame them.. which is why we're all here, right?)

Our current pipeline is:

• Draw in Adobe Animate (nee Flash) or Illustrator, with names given to various objects in the drawing.
• Export to SVG (which forces us to only use a safe subset of drawing features, sadly)
• Run a simple CLI tool to scaffold an interactive simulation atop our in-house https://github.com/cdig/svga/wiki (titled SVGA)
• Write some high-level CoffeeScript code to express the simulation logic, add controls, etc.
• Run a CLI command to deploy to our site. The final result is a single HTML file that works in any modern browser. Typical build size is about 200 KB, including the framework.

It takes one of our artists about a week to build https://www.lunchboxsessions.com/materials/deere-hitachi-excavators-shovels/hitachi-arm-regenerative-circuit-simulation from scratch, with a fairly even split of time between planning, research, design, art, and coding. According to cloc, that linked simulation is about 1700 lines of SVG art (after automatic optimizations), and 600 LOC of CoffeeScript simulation logic (where the amount framework boilerplate in that is probably less than 5%)

(If you're not familiar with it, the graphical symbols we use are fairly standard in https://duckduckgo.com/?q=iso+hydraulic+schematic, with a bit of our own spin to make them more amenable to animation).

That framework I keep referencing is about 4k LOC, and took probably 6 months to develop, spread out a fair bit.

Finally — the FoC project I'm building, https://futureofcoding.slack.com/archives/C0120A3L30R/p1586795860004500, is intended to take us away from writing text-based code at all, to let us build these simulations inside an Adobe/Autodesk-esq graphical environment. I've been working on Hest off-and-on for about 5 years now, and plan for it to be the main thing I work on for the next decade (aside from keeping our web services alive, of course)

This is getting off topic but couldn't you pitch it like a tool on your site where they learn the material, then a module on their LMS where they are tested/audited?

Chris Knott Yeah — sorry for monopolizing the thread. /tugs collar/

Some of our clients choose to do that as a compromise — they fill their LMS with empty content objects that just link to our site, and then we send them "score" data via an API. It sucks, because our idea of testing doesn't fit that mold. We're trying to get away from the notion of a "score" entirely.

I'm still watching this. Halfway through, this feels complementary to and as important as Bret Victor's work.

My go-to's (ha!) are knowledge anchors (current problems) and analogies (vision).

Could start from:

"Do you know why your printer/bluetooth/some website/whatever fail or sub perform all the time?", "What's up with restarting your laptop on every update?", "Why can't you count how many new instagram followers you had in the last month? And keep a rolling count", "Why can't you send a photo to your phone or friend in one touch?", "Could you disable some tv channels for your young kids on demand?"

But yeah I don't find it easy either. And also the technical level of the audience.

@Ricardo A. Medina these are good, I like the division into problems and analogies too. I wonder what other broad categories there are here. It may be a good use of my time to try and collect some of these and organise them a little in a shareable place

I'm still a little curious why there is no market for a flash like editor that generated HTML/JS apps.

Two things come to mind.

1) there is no way to distribute the creations as a single file like you could with flash and a swf file. It would be a minimum of 3 files, one HTML, one player file (probably too big), one data files. Where as with flash you generally had the player installed.

Unity has this issue. the smallest web app is 5-6meg just for the base engine.

2) pre smartphones there was effective only one platform. a desktop PC with a keyboard + mouse + landscape orientation screen. Now the major market is smartphones and in particular Apple has argrubly actively discouraged browser based games by not supporting the html standards that could make them a good experience.

Of course a flash like editor could export to a native app though maybe at that level it can't compete with unity.

I forget what it was called, but was work on this, in particular some people were making a format for distributing html5 games and animations, basically by zipping up the files

Gregg Tavares I think there is a market for that. It'd just cost a lot to build a modern equivalent of Flash (the authoring tool) that publishes efficiently to the web. Not amenable to a startup, and I don't think most existing companies have the culture for it.

This problem is what I've spent much of the last decade working on at my job, but at a small scale, focused just on making an alternative to Flash for our unique needs. We used to be a 100% Flash media shop, and now we still use Flash Animate to draw our artwork, but export to SVG and use CoffeeScript as a successor to ActionScript. The final animation is a self-contained HTML file, with about 50kb of framework code (that can be inlined or served by a CDN). I just posted a big comment about our production process, if you're https://futureofcoding.slack.com/archives/C5T9GPWFL/p1595443888155200?thread_ts=1595438091.136800&cid=C5T9GPWFL.

My FoC project is meant to be a successor to our current framework, so in that way it's meant to be a spiritual descendent of Flash. We're interested in making some money from it, too, so we're staring down that "why is there no market" question. Like our current framework, the plan is for there to be no player — just small, self-contained HTML files.

The plan is to compete with non-consumption, or perhaps to compete with "hire a multimedia team". Definitely looking at Unity as an example, but not going to swim in that part of the pool.

Actionscript was my first language. I miss how easily you could create and publish unique interfaces and games, and would like to see something like it again.

Also, here's a related GDC talk. It goes into early approaches to advertising / monetisation, preventing piracy, publishing platforms - super interesting.

https://youtu.be/65crLKNQR0E

1) there is no way to distribute the creations as a single file like you could with flash and a swf file. It would be a minimum of 3 files, one HTML, one player file (probably too big), one data files. Where as with flash you generally had the player installed.Isn't this what electron apps do?

It's the same problem as Unity. You get a single distribution format, but it bundles an entire damn web browser...

I think that's an own-goal on the part of Electron and Unity. If you want to produce media for the web, you should leverage the web platform — and it takes very little code to do that. Even ThreeJS seems stuffed to the gills with things that aren't needed by most projects (though, to be fair, I haven't seen what happens if you tree shake it).

Before I hijack the thread further (sorry 😛 ) I want to say, great article, I also grew up on Flash games, and I'm glad it's being given the historical curation and respect it deserves.

I think there is a market for that. It'd just cost a lot to build a modern equivalent of Flash (the authoring tool) that publishes efficiently to the web. Not amenable to a startup, and I don't think most existing companies have the culture for it.I feel like this discussion needs some clarification. We are talking about "Flash" as this monolithic thing, but it was made up of a few parts that made it successful.

• A well designed vector animation tool that was intuitive for artists (relatively speaking)
• An integrated scripting language
• A compact and widely accessible distribution format

Gregg Tavares in particular made this implication that distribution is intimately tied to the editor. Those are weird things to conflate together imo...

Adobe Animate CC, the modern Flash ancestor in fact has an export to JS/HTML5 option IIRC... I'm not sure how good it is though.

I do really like the combo for drawing graphics with the code editor (I suppose this is how unity works). Currently to get complex graphics in to webGL I need to draw it export to svg then convert the svg into a mesh - and then reading the code, you can't really see what it really "is". Maybe I should be just doing my project in unity 🤔

@Ray Imber

Adobe Animate CC, the modern Flash ancestor in fact has an export to JS/HTML5 option IIRC... I'm not sure how good it is though.

It's bad, or at least it was when I evaluated it. The output is bloated and doesn't offer much in the way of interoperability — less than even SWF content (which had an API you could tap into, and had good semantics around responsiveness, etc)

I had a feeling that was the case

Ryan King Unity doesn't have art tools in it. Hard to blame them, considering how good Maya/Modo/etc are, and how much insanely more complicated it'd be to put that stuff inside Unity itself.

It does have a scene you can assemble your externally-created graphic assets in, which is, I think, the modicum.

@Ray Imber

Gregg Tavares>  in particular made this implication that distribution is intimately tied to the editor. Those are weird things to conflate together imo...

I mean, they're conflated together by history — Flash was both of those things. A modern successor to Flash might also conflate those things. It's not necessary, but it might be beneficial. Look at Unreal — the engine and distribution platform (Epic Games Store) are technically separate, but they're designed to be mutually beneficial.

It's also sort of unavoidable to consider the output format when creating an authoring tool. And if your authoring tool is creating content for the web, you want the output format to be optimized for portability — a single file, or at least something that's easy to link to or easy to embed in a blog post. And something that anyone can view without a plugin. If you aren't thinking about that when designing an authoring tool, you're going to end up in a sad place.

Ivan Reese Yeah that's reasonable. I think it's more being able to click an object on the screen, and see my code inside that object showing it's behaviour. That fits better with how my brain works than files do.

I mean, they're conflated together by history

It's not necessary, but it might be beneficial.I think this is the more important point... Was Flash successful because these things were tied together, or is that just historical accident?

It's hard to compare with the Epic Games Store imo. It's too new.

If you look at Steam as a counter point, it's very successful as a distribution platform, and is very isolated from the content creation itself.

If you're interested, here's some ThreeJS alternatives I'm keeping an eye on. They're not flash but they're getting easier to code with at least.

https://github.com/oframe/ogl - a striped back webgl library closer to writing webGL but there's something nice and simple about it

https://github.com/protectwise/troika - a framework built on top of ThreeJS, it's declarative and a bit more dom-like. I'd like to see further abstractions like this.

@Ray Imber Yeah, you're right — Unreal/Epic is a bad example. Gregg Tavares isn't talking about distribution in the sense of "a store", he's talking about the file format. Flash had to create its own file format and player because there was no built-in functionality for multimedia on the web. There wasn't even audio — remember RealPlayer? There were other plugins they could have hooked into (Java Applets, say) but those sucked.

What we can have now (though nobody has done a good job of it) is an authoring tool that can spit out self-contained files that work anywhere (on the web), thanks to HTML and the web platform. So as long as your authoring tool produces nicely self-contained single HTML files, you're a lot closer to having that particular question answered, without needing a boil the ocean "our plugin is on 97% of PCs" solution.

So the distribution is "tied" to the editor only insofar as the editor needs to do just that — spit out files that people can actually use.

So the distribution is "tied" to the editor only insofar as the editor needs to do just that — spit out files that people can actually useIvan Reese I agree. Exactly. I basically agree with everything you said.

I think my problem with Gregg Tavares's original argument was that he was looking at it backwards from my view point.

The problem has nothing to do with the distribution format as a technology... It's not a technology problem, it's a social / market problem.

It's not that there is no market, it's like you said, there is a market on the consumer side. The problem is that there is no incentive on the business side to build it.

In that way, Flash was a product of it's time. Unfortunately the world is a different place now.

Maybe the modern web will produce a modern successor. All the parts are there... 😄

I hope this community will produce a modern successor. All the parts are here...

People have tried to recreate the magic of Flash (and are still trying.) There are a huge amount of JS engines out there both 2D and 3D. Most are engines only, they don't have any art tools (which goes back to my point about not conflating things together, at least being clear about the parts.)

Unity has a "web" option but it's a proprietary plugin. Basically everything bad about Flash, so I'm not going to count that.

Godot has an export to HTML5/JS option. IDK how good it is though (I assume it's probably not much better than Adobe Animate honestly...)

There are a few other less known engines that are trying:

https://gdevelop-app.com/

https://playcanvas.com/

Nothing has really taken off though.

There is some magic combination of features and social effects that made Flash so successful. What is the magic formula? 😛

I want to say all the parts are not there. You can't currently make a small single file embeddable multi-media experience with a single HTML file. You could encode your video/audio as dataURLs but the file will be HUGE. You could maybe generate some vector graphics animations in a single HTML file but not your average flash game. Further, flash's format was compat, arguably more than you'll get from HTML without some kind of player (so bloating your page). Other functionally you can't get from HTML in a single file is streaming. Flash could have a 2meg file but after download 50k it was running and downloading the rest in the background. Again to that in HTML requires a player and at least 2-3 files. The HTML itself, the player, either embedded or referenced, and then the data and the data has to be separate else you can't stream

Whether a single file format is important or not is debatable but it's certainly helps things go viral if they can easily be put anywhere by anyone? I suppose hosting your multi-file creation somewhere and letting people embed it would work that costs $$$. Maybe something like http://youflashtube.com and have the editor either online or export directly. I can imagine in some alternate reality that might be successful but it wouldn't be remotely the same as flash was if only because a hosted experience will be censored in various ways

As for unity, unity's web player has been long dead. Unity exports to HTML5/WASM for years now but the minimum size for hello world is 4meg which is basically Unity's engine so effectively it's as if every app is downloading the flash player. I know they are working on shrinking that significantly but I don't know how far along they are.

It's also not mobile friendly which which may or may not matter. If you're trying to make a profit by selling an app/game then you're happy with exporting a native app. If you're trying to make some viral experience that people share with just a link like they did with flash then not being mobile browser friendly is a non-starter

As for three.js, once you gzip it it's only 150k at the moment so barely a single image worth of data on most pages. It doesn't "tree shake" at all at the moment because if legacy code design but they are working on it. But three.js is not even a small amount of flash's functionality. It's just a 3d engine not an app/game engine.

Here is unity's plan for being able to export smaller HTML apps from Unity

https://unity.com/solutions/instant-games

And a live example

https://tiny.vision/demos/TinyRacing/Wasm/TinyRacing.html

It's still 4meg until the page shows anything at the moment.

Somewhat off-topic reply to Ryan King: if you enjoy ClojureScript, the Geom library has a much nicer API than ThreeJS. Here's a small example I build some years ago:

http://proscenium.rusher.com/geometer/

Source code:

https://github.com/jackrusher/geometer

Gregg Tavares

I want to say all the parts are not there.Remember, this is "Future of Code", not "Today of Code".

You are giving more examples of finished products that are not all there.

These are good examples of what is not working right now, but none of these examples show any fundamental missing pieces for why a new flash like platform can't work.

From a distribution perspective, I think all the technological pieces are there. Even before web assembly, I think the parts where there.

The death of flash is one of the big reasons web browsers started to grow the necessary features to replace flash(for better or worse). Web assembly might be the final piece, because it allows a single binary blob to be easily embedded and run. We just need to get better at compression (which is actively being worked on.)

Gregg Tavares My question for you is, if you think all the pieces are not there. What pieces do you think are missing? Not which product is missing, we wouldn't be having this discussion if a finished product already existed, but which technological or social pieces are missing?

I was thinking about this, and while it's very different in a lot of ways, I think the https://www.lexaloffle.com/pico-8.php community has succeeded with a similar spirit to the old Flash games communities, even if it is a bit more niche and smaller scale.

I said above I think being able to distribute in one file might be a reason things haven't happened. So that feature is missing. WASM doesn't work with one file. In fact typically it's min 3 files not including assets. One HTML file, one script with all the JS<->WASM glue. One wasm file.

Pico-8 has definitely has some of the vibe of flash games but it's a way higher barrier to entry than flash ever was. There's no doing anything in Pico-8 without code but there was plenty of flash with no or minimal code.

So I think 4 things are missing

• being able to distribute as a single file. not entirely sure this is important but feel it's possible it is important
• being able to stream the assets from that single file vs having to wait for the entire file to download. If 1 is important than so is this.
• working well on mobile in the browser (this is only semi-possible because Apple has so far refused to implement the needed APIs. In particular the fullscreen API so a webpage can go fullscreen (or close to it ) like a native app can and the orientation API so a webpage can force landscape view. Android Chrome/Firefox has had both of these for 6+ years and it hasn't ruined the browsing experience there.
• A well designed non-programmer friendly app for making things

I also feel it really is about the app. Unity and Unreal are not doing anything that game engines didn't do in 2003 but the fact that they package it all up so a non-programmer can build a Dear Esther type of game with no programming or even with a little programming is what made current explosion of games.

The same was true for flash IMO. It wasn't just that the tech existed. It's that there was an app that let people easily use tech to make stuff.

It could also be though the flash was just something that happened and it's time has passed. I found this site yesterday

http://poki.com/

It's got a ton of games that seem right out of the flash days except with higher-res art 😛

also https://itch.io is full indie games and many run in the browser, though likely desktop only unfortunately.

Just to add kindly precision (but I am not sure if this is really important for the discussion):

WASM doesn't work with one file. In fact typically it's min 3 files not including assets. One HTML file, one script with all the JS<->WASM glue. One wasm file.Technically, if needed one can bundle all this in one HTML file, with a script tag for the JavaScript / WASM glue, and with the WebAssembly binary content encoded in base64 somewhere in the script or in a custom HTML tag.

I forgot about http://itch.io! That is another great example of a modern Flash games like market! http://poke.com looks interesting as well.

but it's a way higher barrier to entry than flash ever was. There's no doing anything in Pico-8 without code but there was plenty of flash with no or minimal code.Good point. I can see that much of your argument is about barrier to entry and ease of use. 👍

Now about your 4 things. I'm going to disagree with you on a few points:

1. being able to distribute as a single file. not entirely sure this is important but feel it's possible it is importantI would like to point out that Flash was not that different.

In theory you could just "play a swf" file directly, but nobody did that. You still had to host a website with an HTML file (and probably a CSS file), and embed the swf file. And the user had to download or update their flash player on a regular basis.

In fact, nobody did that either! They uploaded their swf to Newgrounds, who did all the hosting work for them. The key here is the marketplace (Newgrounds).

That's not much different than the JS/WASM, HTML, CSS combo we have today. It's not really just "one file"...

An important part of the equation was that Newgrounds did a lot of that hosting work for you as a content creator.

There is no reason the same kind of thing couldn't be done for WASM or modern JS. In fact, that's what http://itch.io and Pico-8 are doing!

I would also like to point out that you can embedd assets in the WASM binary!

Modern assets like 3D models and bitmap textures are also very different from Vector based assets (I will talk more about this below).

The key is that there was a convenient way to get from the content creation tool to the hosting site easily. That's a really good point!

I just want to point out that there is nothing technologically preventing us from building something just as convenient today, it just hasn't been built yet...

being able to stream the assets from that single file vs having to wait for the entire file to download. If 1 is important than so is this.I am actually not sure what you mean by stream assets. It's been a long time since I programmed with flash (I did back in the day, but it was a looong time ago), but I'm pretty sure it didn't "stream" anything, at least not in the modern sense of sending bytes over a network a little bit at a time. Youtube didn't even exist until 2005, and when Youtube started, streaming technology was still pretty bad... If I'm wrong, please give me a source and correct me.

I think what you mean is compression. Flash was good at compression. It also had good "time to first interaction" latency. I totally agree that "time to first interaction" is super important and extremely under appreciated with modern computers. It's a big pet peeve of mine (and anyone who follows Jon Blow, Casey Muratori, and the handmade philosophy).,

I would like to point out the Flash assets were almost entirely vector based. Vectors compress extremely well! They are just control points for a Bezier curve equation.

The internet also had much less bandwidth during the height of Flash, so in general the assets were smaller.

I remember Flash animations that had a lot of bitmap assets instead of vectors. You got loading screens just as bad as today (worse in some cases).

The situation is actually better today for the end user from the tech standpoint. The internet has orders of magnitude more bandwidth, and the browser apis let us do actual proper streaming of data (through js download requests or websockets).

working well on mobile in the browser (this is only semi-possible because Apple has so far refused to implement the needed APIs. In particular the fullscreen API so a webpage can go fullscreen (or close to it ) like a native app can and the orientation API so a webpage can force landscape view. Android Chrome/Firefox has had both of these for 6+ years and it hasn't ruined the browsing experience there.

Yeah. Modern Apple sucks. I agree. lol

A well designed non-programmer friendly app for making things

It wasn't just that the tech existed. It's that there was an app that let people easily use tech to make stuff.I think you make a really good point here. I would add that it's not just about the content creation app, but also the hosting and distribution sites like Newgrounds. Newgrounds was a "proto-app-store" in many ways.

It could also be though the flash was just something that happened and it's time has passed.You might be right here. It seems like the big game engines like Unity, and the proprietary app stores, both on mobile and desktop, have taken over the hole left by Flash.

My TLDR; description of the "ideal system" we are describing looks like this:

• A content creation tool that is polished and intuitive for artists (normies? non-technical people?)
• with low barrier to entry scripting / programming interface
• A runtime that allows for a low "time to first use" latency metric and good compression (that might mean vector based, or some other easily compressed asset, or taking better advantage of modern data streaming technology)
• Is tied to some hosting platform. Some "app-store" or "marketplace" like system that is integrated with the creation tool to provide creators with low friction content distribution and hosting.

The technology to build all of this exists today imo, but:

Is there a place in the modern web (or modern computers in general if we include mobile devices) for such a platform?

Does it make sense to tie it to the web given the rise of mobile computing and politics like Apple's walled garden?

It's a bit chicken & egg, whilst platforms accelerated the growth of the flash, the content existed first. I imagine modern equivalents might be able to compile and publish to many platforms, eg. apple / google app stores, native desktop, and the web.

Perhaps animation is a nice gateway into programming. Tweening is a nice simple point-click exercise but the more complex behaviour you need, the deeper into programming you go.

Perhaps animation is a nice gateway into programming. Tweening is a nice simple point-click exercise but the more complex behaviour you need, the deeper into programming you go.☝ This right here is a key insight I think!

From a "FOC" point of view. Flash served as a "gentle intro to programming" gateway drug for a generation of programmers.

Flash was the 2000's equivalent of Basic for many people.

What made Flash a good intro to programming tool? I think you just answered that question 😄

@Ray Imber thought it would be interesting to make a thread about this https://futureofcoding.slack.com/archives/C5T9GPWFL/p1595624526464200

@Jack Rusher From your recent posts, perhaps you have some insight? 🙂

The ideal model is a user defined n-ary tree… or well, a list 🙂

My long-term project is a tree editor that does exactly that - purely transforms the data.

You have a tree T, you write a transform function f(t)->g, you got a projected tree of G now, but inside I keep them connected via references - the model isn’t “G”, the model is “f(t)->G”. This way, you grow your codebase from the model outwards, expanding it into each category you need (sql schema, model, viewmodel) via transformations, or by creating new trees that reference stuff from any of the existing trees. That way, you mutate your original model and all the references get the mutation transformed to their own projection.

I keep it all inside a large model called Lattice, that is a basically a huge graph that connects over multiple dimensions (each transformation adds a new dimension to the graph)

Didn’t yet come to edges as mutation permissions, since I feel that isn’t a part of the pure model, but of a “projection”.

Your "transform function" is the same as "derived state", a database "view", and a "projection", right? We have a lot of names for it. That's an example of a programmatically-maintained information structure as I called it earlier. What about static structures? How would you encode a text document, or a (static) spreadsheet, and how do you derive meaning from the encoding? Spreadsheets in particular do not have a clean representation as a list or a tree. You'd have to do some shoehorning.

What do you mean by "information"?

In the most general form, this is the core question of ontology, which philosophers have been wrestling with for millenia. Slightly more restricted, you can look at GOFAI approaches to this problem, like cyc, which have largely failed to bear the kind of revolutionary fruit promised.

How general are you trying to be? What are you trying to achieve?

This is a fairly deep topic for which a simple answer is impossible. A few fingers pointing toward the moon:

• It would be best for such a system to store the data in a structure that "changes" through accretion rather than mutation (http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.102.8253&rep=rep1&type=pdf, much emulated since). If every modification point is essentially a snapshot, you get time travel and other useful semantics for free (Datomic is a recent database that uses this approach).
• Up-front schemas are brittle over time. The less you can get away with saying about structure, the better off you will be. (A special case of the more general advice to "design for change").
• The approach whereby everything is a logical assertion (i.e. triples, sometimes called "Subject/Predicate/Object", sometimes "Entity/Attribute/Value", depending on the culture of the speaker) is the most flexible I've been able to come up with yet. This underlies RDF (and TripleStores generally), and is currently in vogue among those who have encountered datalog, mostly via Datomic.
• It's very useful to be able to specify additional automatic behavior over triples via some kind of metadata, in the best case also specified in triples. (RDF via OWL, not Datamic.)

I have many differences of taste with the place where RDF ultimately went, but there's loads of good stuff in there too. You can find out more about the history of the "data as assertions" approach by reading up on https://en.wikipedia.org/wiki/Knowledge_representation_and_reasoning (as distinct from data representation), https://en.wikipedia.org/wiki/Frame_(artificial_intelligence)#Frame_language, and so on. This stuff is one of the threads that leads to object orientation, though much less emphasized in a modern context than the strand motivated by simulation.

The "immutable tree on disk" stuff comes from generalizing database journals, which for me starts in the 80s with https://en.wikipedia.org/wiki/Log-structured_file_system. Today there's loads of work on immutable logs for all sorts of applications, especially with the recent popularity of cryptocurrencies.

Wow this thread has already brought out so many fellow travellers who I didn't know were exploring the same terrain as I am! The history of my own project over decades is intimately tied to my search for the "perfect" data and data transformation approach, in search of the "purest" model of programming - which is one that is most cognitively aligned. Aligned to the cognition of a .. ahem .. "average human", so probably less to that of the "average programmer"!

So for me it is important to step back and look at common patterns of cognition - how we organise our perception of reality. One approach is to read Roget's Thesaurus! All the important cognitive concepts are there, organised in a specific idiosyncratic way. There are many other "meta-ontology" approaches, as have already been listed here.

I could start listing out what I believe are essential "reality modelling primitives" as you already have done yourself. For me, where I am right now in my journey, that is basically: identity of things, properties of things, relations between things.

At a more abstract level, you have symbols of value, order of those symbols and their aggregations, bags/sets of symbols. You have named properties that can have values.

So, as you can probably deduce, I've settled on something that looks like objects that have unique identity, which are bags of properties in the form "property name: property value". Property values are just symbols (text really) or ordered lists of symbols but can also be links to other objects by identity - relations between objects.

One issue that really bugs me is that I can't extract a single representation of order! There is order in my properties (I don't need that but I want it). There is order in my lists of property values, and there is order in the direction of the link pointers.

If you want to make it attractive to humans it needs to be built out of concepts that humans have "hardware support" for.

Sometimes there's a tendency to seek "elegant" solutions which use the minimum possible number of concepts, whereas I think it's often more natural to expand the number of concepts, even if introduces "redundancy".

For example, you can describe a bijection between XML and JSON, that would involve translating JSON lists into a something you can describe in XML such as ["one", "two", "three"] becomes <list-item index="0">one</list-item><list-item index="1">two</list-item>... or something.

You could say that XML is a more elegant way to represent information, because it is based basically only on the concepts of Containment, Naming and Properties. But because I believe that humans have "hardware support" for many more concepts, such as Sequences, I think that JSON is actually easier and more natural (for humans).

So basically my only input would be to be sceptical of mathematical beauty if the format is intended to store information relating to the human world.

There is a list of things that have been observed in every human culture (Donald Brown's Human Universals) and it is surprisingly long. Unfortunately it doesn't have much on the way of numbers (only the numbers 1 and 2) as some cultures are almost completely innumerate. It does have quite a few logical operations like Not, Or etc

S.M Mukarram Nainar By "information" I just mean data with associated meaning. I refrained from using the word "data" because it's a machine-oriented word and I don't want people to start talking about conventional data structures and pointers. I want to focus on the human interface, not the machine representation.

I'm not too concerned with standardizing ontologies or AI right now. I'm more concerned with allowing humans to group a bunch of symbols together by hand or programmatically in a way that makes sense for their own understanding. I'm designing a programming language, but unlike almost everyone in the last few decades, I'm not taking "arrays" or "objects" or "algebraic data types" or "databases" or "relations" for granted: I'm trying to figure out, from complete scratch, what information structures are easiest for people to think with when they are integrated into a complex system (e.g. a distributed app). I'm trying to be as general as I can, but my domain is "programming", taken broadly.

I want a model for information that allows users to encode information like "Bob has a car with 4 wheels" in the form that is most useful to their specific application. In a company database the car might be an immutable fact with an associated date and source. You might then tell me that the right model for encoding information about cars is the relational model. However, in a video game the car might be an entity with a position that changes over time, and so we need to be able to talk about the idea of change and the distinction between descriptions (where the description itself doesn't "change" as time passes) and references (where the entities referred to can change).

The question of "how do I model information" may seem trivial (or "solved") at first glance, but the moment I ask that you don't treat conventional data structures as axioms ("we already figured it out, just mix these existing things together and follow best practices"), it becomes incredibly deep.

@Jack Rusher I've liked the idea of "accretion, not mutation" in the past. But I'm worried that making it a requirement in a programming system is going to lead to space consumption problems for certain use cases. How do you get around that, if you aren't restricting your application domain and user base?

I agree that it's wise to limit unnecessary structure: scenarios where a relationship is specified with some kind of directionality/asymmetry that isn't inherent. I've been calling this "incidental order", and it's one of my biggest concerns. I've been inspired by Datalog, though I've come to a conclusion that mere relations (including as triples) aren't expressive/intuitive/"easy" enough as the sole information structure for for general-purpose programming*. I think it's part of the answer though. Do you believe otherwise?

• I probably wasn't explicit that I want a model for information that can be used ubiquitously throughout a computation, not just as a "final output".

And thank you for the links! I'm going to chase them up later.

Duncan Cragg Careful! 🙂 The age old practice of modeling human mental processes with first order logics has not gone very well. Even something as seemingly straightforward as object identity is quite fraught in practice: https://en.wikipedia.org/wiki/Ship_of_Theseus

Duncan Cragg

where I am right now in my journey, that is basically: identity of things, properties of things, relations between things.How do you distinguish between properties and relations? Is the fact that I'm "holding my tea cup" a property of myself, or am I just in a relationship with my tea cup?

At a more abstract level, you have symbols of value, order of those symbols and their aggregations, bags/sets of symbols.What is a "symbol of value"? I don't quite understand that term. And you seem to recognise both sets and bags? I've always found those to be in contention with one another, and so I've (for the meantime) concluded that if a user is in a scenario where they want a bag (which should be rare), they can simulate it with a set quite easily. The converse is not possible, though.

So, as you can probably deduce, I've settled on something that looks like objects that have unique identity, which are bags of propertiesI've currently settled on two notions that together can act like a conventional object. The first is something I'm tentatively calling a picture, which is a set with an exterior label (I call it the topic) that contains symbols and/or further pictures (these are called subtopics). These can be used to form the "descriptions" I was hinting at earlier; they are immutable value types. If you want a picture to be mutable (and thus behave like an object), you can stick it in a workspace, which is the unit of mutability. Workspaces are identified (and thus referenced, and queried) by the aforementioned symbols. A workspace's state evolves by a rulebook (a set of rules), which updates a workspace's picture in response to events. I plan to base the semantics of rules on logic programming. The end result will be something like "nested Datalog", though there is no literature that matches the model I described above.

Picture topics replace the need to have a special notion of a statically-specified "attribute", because the topic can itself be any value. This should hopefully make meta-programming possible, but at the very least, I hope it leads to a very minimalist (and intuitive) semantics.

All of the above is subject to change, of course! Given my current track record, I wouldn't be surprised if I find some major flaw in this kind of architecture.

@Nick Smith In terms of storage, the natural thing to do in a log-structured environment is to have a "cleaner" that purges no longer referenced old versions subject to some policy. In the paper I linked, we built the system to purge whatever was no longer linked. In another system one might have a policy of "versions older than a year to which no one holds a dangling pointer". In either case, I would strongly encourage you not to consider practical matters of performance or storage efficiency at this stage. Go a bit mad, follow untrodden paths, find weird things!

As for expressivity/ease of triples, one would expect a system that can construct other representations on top of triples for ease of use. What that looks like in your domain is a very open question. 🙂

@Jack Rusher I'm definitely good at going mad! It's my perpetual state of being 🙂. Thank you. I'm going to keep pondering on all of this!

@Jack Rusher which paper are you talking about exactly? I was on the verge of implementing a cleaner, “Interdimensional black hole” so to say that would purge empty references in the background, but couldnt decide on when to run it.

Duncan Cragg

One issue that really bugs me is that I can't extract a single representation of order!If you make any breakthroughs on this, let me know. I'm still trying to figure out how to represent sequences (lists, priority queues) and trees (e.g. family trees, or any other domain-relevant tree structure) in a non-hierarchical manner, such that users can perform arbitrary traversals (in any direction) and queries. My big constraint here is that I want to ensure traversals will always terminate, so I need there to be some measure of progress associated with the traversal rules. Another challenge is multi-dimensional ordered structures, for example, a spreadsheet! How can one describe a traversal across a spreadsheet? Every cell in a spreadsheet is ordered in two dimensions! My current line of investigation is to determine whether all kinds of ordering can be exposed as numberings, and sequence manipulation is manipulation of those numberings, i.e. a list might look like {(c,1), (a,2), (t,3)} and a priority queue might look like {(c, -12), (a, 7), (t, 34)}. I'm thinking of going deep and figuring out if lists and trees can be treated as spatial data sets and manipulated geometrically.

How do you distinguish between properties and relations? Is the fact that I'm "holding my tea cup" a property of myself, or am I just in a relationship with my tea cup?it's a property of you, the link to the cup, and a property of the cup, back-linking to you, since both of you care about the holding! Other links may be one-way: the puddle has a link to the sun that's warming it and evaporating it, but the sun couldn't care less about the puddle

That seems like a bit of an arbitrary distinction though, right? How do you formalize who "cares" about a relationship? How does it affect how a user models their application?

What is a "symbol of value"? I don't quite understand that term. And you seem to recognise both > sets>  and > bags> ? I've always found those to be in contention with one another, and so I've (for the meantime) concluded that if a user is in a scenario where they want a bag (which should be rare), they can simulate it with a set quite easily. The converse is not possible, though."3.2" "up" "red".

set/bag: just recognition that ordered lists sometimes don't need order, but have to be laid out somehow in space, which means the order is incidental. I don't think this is a problem though.

"who cares": it's up to you as the modeller

This thread is piling up and I need lunch.. 😄

What are your measures of success? To be programmable by techies or normals?

@Jack Rusher I take the approach of aggressively GCing old states/versions: if you want to model history, you do that explicitly yourself

On order (e.g. spreadsheets): that's a whole thread in itself

I'm getting overloaded. Time for lunch

@Ian Rumac This http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.102.8253&rep=rep1&type=pdf describes one such implementation. We used multiple threads of execution. One of the benefits of the overall architecture is that multiple readers can continue running without any locking while writes are streaming in.

So ... fuelled up now.. 😄

Spreadsheets: well, tables, well... again it depends on the semantics of the table: for example, a list of objects can form a table with the common property labels as either row or col headers, and all the property values in the grid. If you have a genuine case of 2D tabular data - e.g., an f(x,y), then the order you select (x then y or y then x) only matters for optimisation

funny, can't think of an example of f(x,y) as a table, where col headers and row headers are either both symbolic or both ordinal .. can you?

(a 2D lookup table where row and col symbols are values not "property names")

If you make any breakthroughs on this, let me know. I'm still trying to figure out how to represent sequences (lists, priority queues) and trees (e.g. family trees, or any other > domain-relevant>  tree structure) in a > non-hierarchical > manner, such that users can perform arbitrary traversals (in any direction) and queries. My big constraint here is that > I want to ensure traversals will always terminate> , so I need there to be some measure of progress associated with the traversal rules.why non-hierarchical? how does that help with traversal termination?

I've currently settled on two notions that together can act like a conventional object. The first is something I'm tentatively calling a > picture> , which is a > set>  with an exterior label (I call it the > topic> ) that contains > symbols>  and/or further pictures (these are called > subtopics> ). These can be used to form the "descriptions" I was hinting at earlier; they are immutable value types. If you want a picture to be mutable (and thus behave like an object), you can stick it in a > workspace> , which is the unit of mutability. Workspaces are identified (and thus referenced, and queried) by the aforementioned symbols. A workspace's state evolves by a > rulebook>  (a set of > rules> ), which updates a workspace's picture in response to > events> . I plan to base the semantics of rules on logic programming. The end result will be something like "nested Datalog", though there is no literature that matches the model I described above.

Picture topics replace the need to have a special notion of a statically-specified "attribute", because the topic can itself be any value. This should hopefully make meta-programming possible, but at the very least, I hope it leads to a very minimalist (and intuitive) semantics.

This is what prompted my question above: who are your target audience? Is this going to be intuitive to everyone, if it's non-techies you're after?

(iow maybe you could draw a diagram with examples! 😄 )

@Nick Smith "Has anyone..." asked in the place where a good fraction of us have. 😉 After playing in this space for twenty years, I'm not a believer in an ideal model. What is ideal will depend on the task at hand. What is better is having systems that you can readily shift between.

You know you're in bad programming place when a conceptually straightforward shift proves difficult. On realizing that instead of having one of these things, you want to be able to shift between three different kinds. which will now require six months of developement work.

Also today via Hacker News, I was pointed to Terry Tao talking about mathematical notation, which is of course a model for the underlying notions. https://mathoverflow.net/questions/366070/what-are-the-benefits-of-writing-vector-inner-products-as-langle-u-v-rangle/366118#366118

What are your measures of success? To be programmable by techies or normals?Definitely "normal people". Unusual folks like us can focus on developing the infrastructure that implements humane programming systems. Of course there's still going to be concepts that need to be learned and practiced for a person to develop competence.

Is this going to be intuitive to everyone, if it's non-techies you're after?Once learned from effective teaching materials, and with the aid of effective visualisations, yes, I hope it will be intuitive. I actually think I can map it to everyday representations like nested dot points and tables. But first I have to figure out the semantics I want.

Duncan Cragg

funny, can't think of an example of f(x,y) as a tableThe data that people store in a spreadsheet is usually tabular, but the spreadsheet itself, as a graphical artifact, is most certainly a function f(x,y) from row/column number to datum. Navigating a spreadsheet using a keyboard requires locating successor/predecessor cells within each axis. So a spreadsheet is perhaps better termed a grid, which one could argue is the generalization of a sequence to 2D.

why non-hierarchical? how does that help with traversal termination?Those are orthogonal, but the reason for eschewing hierarchy is to avoid a situation where a user specifies a data structure, and later realises that they can't express the query they want because the data is "ordered" in the wrong way. Then they change the structure or add a new one. Hierarchical data structuring always leads to this pain.

Been thinking about this problem for the last year or so, and I’ve ended up somewhere I didn’t intend so this may not be an answer you’ll like.

I don’t think there is any single representation that has all the desirable properties you’re looking for, but not only do I think that’s okay, I think we can do even better by moving up a level and instead look for a representation that can itself represent the more desirable representations. If we take a hypergraph or metagraph, we can use it to represent sets, lists, trees and acyclic graphs, etc, etc. And thus can embed structures with desirable properties if we accept a base representation that has few if any of the properties we would like.

We're gonna need some examples to chew on. 😄

I’ll pull out some references from my research library when I’m home, but in the meantime I’ll put a couple things forward.

There is a partial order to the generalisation of structures such that hypergraphs [1] are at the top, a subset of hypergraphs form a directed graph, a subset of directed graphs form an undirected graph, and this pattern follows down to trees, sets, lists, and so on.

There are many possible semantics for these structures and I’ve yet to see much work here as most of it is towards proving certain properties of these structures. Category theory provides useful properties but certainly doesn’t translate directly to nice semantics for an actual end-user system.

1. This is not quite true, as metagraphs are more general though not yet well understood or defined. And also typed https://rcor.me/papers/typed-graph-theory.pdf. (Sorry for all the Greek!)

I’d add that I’m not proposing that this is the best way to interact with a system, and there are places where natural representations do not always fit in a machine-oriented way, such as functions, n-dimensional arrays, etc. But in these places there are natural representations that can be used in part to interact with them. I.e. representations of grammars for strings (not the data type, just ordered sets of symbols) along with logics, type systems, etc given an appropriate semantics and grammar.

@Orion Reed Idk, I'm not really being sold on the hypergraph thing yet. I've never seen an information model based on manipulating graphs that feels "good" and "intuitive" for general-purpose applications, and if that's not your point, then I'm not sure what value you're proposing they have. Mathematical formalization takes a backseat to user experience in my world.

Also, I can actually model hypergraphs quite easily in the information model I briefly described above (labelled nested sets). I'm not sure why I'd want to, though.

@Nick Smith he's saying that if you use hypergraphs as your canonical representation, all other structures are reduced to views and mutation rules over that hypergraph. This gets you the property that @William Taysom was talking about that it is trivial to go from one representation to another. It also gives you the flexibility to create "good" and "intuitive" interfaces on top without changing the underlying model.

@Nick Smith I agree, mathematical formalisations should take a backseat to user experience a lot of the time, and also agree manipulating graphs doesn’t feel good for general purpose applications, at least not yet. My point (which by all means you can ignore!) is that it may be useful to build the information model at a slightly higher level than your planned use-case, which may help de-emphasise finding a perfect representation which you may wish to tweak later.

I think there’s lots of merit in sticking to your intuition and discovering the emergent properties of your system [1]

I bring up these points mostly as exploratory, and to help support the idea of separating the model from the internal workings of computers or current computer systems.

1. many great ideas happened this way, Unison for example discovered many great consequences only after deciding their basic principles of content-addressed immutable code.

Chris Granger yes that’s exactly it, well put

if your goal is to go after end users and also be truly general purpose, I think you'll likely end up on a hypergraph. People do not naturally think in strict structures, they think in very loose, just in time structure. As several people have said in here, that means you need to flow seamlessly between representations and allow for partial/incomplete ones to still be useful and to link to completely different views that color things in a bit more.

This is what I've been thinking about for the last several years, and haven't gotten all that far. One of my goals is to provide an intermediate format for converting/proxying between systems speaking different languages (immediately commercially useful and helps overcome network effects). I think my most interesting idea at the moment is that data is a program (under a suitably restricted interpreter), that outputs itself, or parts of itself identified by an argument (e.g. treating a hash map as a function from keys to values). This also helps with one of my long-standing goals to ease mixing of manually and procedurally generated data.

I thought about hypergraphs for a while, albeit when I was a lot younger and stupider, and my conclusion was that I'd rather go with a fundamental structure that makes it easy to implement hypergraphs. To me that looks more like a logic database or triple (tuple?) store.

@Andrew F that first goal of yours is one I’ve been pursuing for a while for the exact same reasons! Have you made progress you’d be willing to share?

@Orion Reed haha, not really. I've spent the last several years building up ideas and tearing them down when I realize they're not general enough. I am confident at this point that the data store needs to be an active participant to support things like API translation proxies, and that solving the problem of Naming Things is important (tracking identities referenced in different ways in different representations). Ask me again in a year. :D

Have you already read the Functorial Data Migration peoole's work?

@Andrew F I hadn’t but will check it out, it looks compelling. I’m assuming https://arxiv.org/abs/1009.1166 is the work you’re referring to?

@Orion Reed yep. I think they've built a company on their ideas, too. IIRC they still had to do some weird hacks that made me think it needed more work to be nice to work with in the real world, but definitely interesting. I need to read them more closely at some point.

Oh, thanks to you guys I now know that what I was doing is a hypegraph 😄 IIRC my hypergraphy was basically a store of mostly maps of <ID, Value> and <Type/Dimension/GraphID, <ID, NodeData>> . Guess I was calling it wrong the whole time 🤔

I had to look up hypergraph, but I'm still confused about why it's not just the same modelling approach as .. JSON?

Or really just relations.

Duncan Cragg Relational models impose some more strict structure over your data in the form of tables. In graphs and hypergraphs the vertices/nodes and edges/lines are first class citizens.

relations in 6th normal form are a good practical representation for hypergraphs

Amazing thread. Been slow churning on this for years with nothing to show for it. Anyway, recently found this paper which may be of interest - tries to define a 'conceptual graph' that represents info of how the user sees it, without the storage representation details: https://pdfs.semanticscholar.org/2ae6/ac8fc13710d9c086c0e5cb952eef52c9b3cd.pdf

@Nick Smith I’ve been thinking about similar things for a while.

What I see now is (obviously?) graphs are the only, truly extendable & fractal like structures.

Fractal like: which also means the minimum we can do with the “Graph Information” on a the “simplest level” is also the maximum we can do with it on “higher levels”. With that in mind I feel like the direction to is to build “Views” on the top of the “Graph Information” - which Views themselves are going to be Graph based entities. Also, we can say these Views would act like filters to reduce the noise of “all information” and deliver the proper context.

So after that, my answer is: just simple, pure Graphs, but a new question arises: how to build Views on them - even on multiple levels…

I just don’t see ANY other idea which can’t be translated to that approach. It seems to me it’s the alpha and omega.

It’s not accidental RDF node-edge-node triplets and stuff like that emerged and ontology is full of that.

this thread is amazing. i’ve been working on this since i started hoarding popular science magazines in middle school and wanted a way to organize them lmao. lately i’ve become convinced that whatever structure you settle on is going to have to be expected to be in an “incomplete” state, and that there’s some kind of dual relationship between data and computation that needs to be leveraged (thinking about issues like caches and garbage collection as a UX feature rather than an infrastructure bug). i’m working on a really stupid version of this stuff in json and will be posting in #feedback and #two-minute-week, and i would be delighted if any of the people in this thread ruthlessly criticize where i end up!

When Chris Granger says "6th normal form," it more or less means "all hash tables" with array keys as needed and does a great job of capturing the dependencies of the form for every and , you can have a ___. Correct me if I'm way off base.

Yep! You can also think of it as triples where the entity can be a composite key. This gets you to true atomic units of data, reducing any further would cause information to be lost. The important thing that this captures is properties on hyperedges - e.g. something like marriage where the key would be two people and maybe the value would be the date or number of people in attendance.

I think my "nested labelled sets" conception is just a particular manifestation of a 6NF DB, with the only difference being it has a notion of scope and modularity that allows you to easily shuffle chunks of data around. Chris Granger this is like your scope-limiting bag "database" notion in Eve (I re-checked the name), except you can nest them inside each other.

yeah, we ended up calling them databases. We regretted adding them and removed them right after 0.2. The problem was that it became very confusing to figure out where things were and should be. Namespaced tags/attributes ended up working much better for us.

You have to have a boundary at some point though right? If your PL supports distributed apps you don't want to accidentally query some fact from a server in some Japanese village somewhere. At what point do you hit a "bucket" of data?

you would bring those in as namespaced tags, so #foo vs #japanese-village/foo

And every query must draw facts from a specific namespace?

you can arbitrarily union them together within a query, you can also bind them to some common tag and just query that tag

Isn't that the same as an Eve 0.2 "database" if you add the ability to make the database a variable? That would emulate the latter capability you mentioned anyway.

I guess I'm trying to understand this problem you mentioned:

it became very confusing to figure out where things were and should beI don't quite understand how this could be a serious problem. I'm no fan of hierarchical organisation in file systems, but a little bit of hierarchy is good for modularity, which you really need in any distributed system.

We should probably split this off so as to not derail the conversation further. The difference is largely in how people approach the two mechanisms. One is just a more specific name, the other is an actual place. We, ourselves, pretty consistently made mistakes about which place to query and write into, but we didn't experience the same problems with more specific names. Part of the difference is also how you selected a database in Eve (declaring it at the search/bind level) which made unions pretty awkward. That one's not at all fundamental and you could've chosen something else, but I suspect it would look a lot like namespaced tags if you did.

It's easy to forget the database name, or forget to change it when you copy a block over, for example. And the bugs that result from that are pretty hard to understand without some nicer tooling.

I think "namespaced tags" and buckets/databases only diverge in meaning in the presence of mutation (changing values). I'll discuss that in this thread (once I think it over further): https://futureofcoding.slack.com/archives/C5T9GPWFL/p1595571049299600

Yes, @Nick Smith, that's a good point: you can't edit db views. You can't edit the value in a formula cell. So a bucket that can evolve may look the same as a query result, but the latter only evolves because one of the former does. Same with tagged data: the "bucket" of items with that tag depends on items being tagged, and can't be edited or evolve independent of that. Whereas a directory of items, a bucket, can.

From an end user perspective, the question is whether the item or a collection of items is the primary focus. In some modelling tasks, it's important to model a first class bucket of items, e.g. the actual people standing in an actual room, rather than having a big swimming pool of people and picking out those who are in the room to be tagged accordingly.

@Nick Smith Also don’t forget you can replace data on edges easily by introducing a new, intermediate node. (N1)--[E]--(N2) might become (N1)--(E)--(N2) , then you have a bipartite graph to work with.

@Pezo - Zoltan Peto Tbh I'm not really keen on thinking about graphs as a foundational concept. Graphs are a solution looking for a problem. You can use a graph to model a set of relationships, but a graph is not itself a set of relationships. I think we should talk about "entities" and "pointers" between them as a potential use case for graphs rather than talking about graphs first and figuring out their applicability thereafter. Otherwise our thought patterns will never escape from our preconceptions of what a graph is and can be.

@Nick Smith I agree for most of us we should not go from graphs -> use-cases and want to add: “Not every problem is a graph problem” which is evidently true. There’s also the useful truth that almost any imaginable problem can be translated into some sort of graph. (Space efficiency, practical implementation aside)

I often think in sets, partial orders, trees connected to lists inside of graphs next to some other structure, etcetera. It’s certainly a messy business and we’d miss a lot of perfectly sensible thinking if we forced people to first translate their thoughts into graphs. But I guess that’s what the computer is there for 😉

Chris Granger @William Taysom does that mean that clojure’s EDN is a natural fit for 6NF or am i missing something?

I cannot read your mind Garth Goldwater. Are you talking about https://github.com/edn-format/edn? What kind of connection are you imagining?

yeah, that’s the one—i just mean that it has support for array keys, for example. is having sets point to other sets the minimum requirement for eg serializing a hypergraph or are there other capabilities a data format would need for comparably powerful semantics

You should add a link to your post!

@Tyler Adams title 1 sounds like a title from pre 2018, and title 2 sounds like a title from post 2018 (for better or worse) 😛 it's interesting that kind of phrasing totally developed out of clickbait marketing but seeing it in increasingly reputable news services now too...

Of course, https://codefaster.substack.com/p/jq-features-considered-harmful . Yeah, a good title seems to matter almost more than a good article these days...

People don’t get Rx / eventStreams 😜 & FRP

that's so 2018, devs can only collectively swoon over one thing at a time, now it's CRDTs

It was CRDTs in 2010, too ;)

Or maybe, like the hipster who burnt their mouth on the pizza, I was into CRDTs before they were cool.

You probably were all about FRP in 2009 - so tell us about 2028 Ivan

Still, I think this is a real issue preventing progress. Sorry. :< 😄

Christopher Galtenberg ...earlier in this thread.... ;) https://futureofcoding.slack.com/archives/C5T9GPWFL/p1591042419317500?thread_ts=1591041589.315800&cid=C5T9GPWFL

Kartik Agaram Hah, you are reading my mind! These are my top priorities as well!

I’d add that, which is remained unsolved to me in the past years, and maybe this is the main driver behind all of my research:

Be ready to change/delete any part of the code any time!

Mixing it with your 2nd (controlling evo) and 4th (controlling deps) point:

avoid broken abstractions

• AKA -

don’t write (or evolve to!) code which forces you to delete more (prod+test) code than neccesary (in case of new requirements)

• AKA -

eliminate God Objects (don’t even let them birth!)

-AKA -

Minimal surface: don’t make available any data in any given scope you don’t use => if you do, you are creating a God object and you have to REIMPLEMENT the stuff after deleting the nighmare and all of its connections.

Sure it is an obvious idea to do that, but hard/impossible(!!!) to do without the correct level of abstractions (I think anything “below eventStreams”).

To me it seems like we might need some level of redundancy to get there, but I’d pay that price if I can make sure the freaking God object and spaghetti bs goes away…

This is just a deep intuition at this time, but I’ll update on that later.

Modularity (separating things into separate "wholes") is not necessarily related to there being any hierarchy. I think a lot of hierarchy comes into trying to shove like things into a "common base" and then subclass. But on that matter I'll just say: composition over inheritance, interfaces over subclasses, labels over a tree. "A city is not a tree". Some things fit multiple categories, or can play multiple roles, and that does not slice cleanly into exclusive buckets

Traditional approaches to modularity certainly induce hierarchy. In a typical program you have "modules" containing "functions" containing nested functions, loops, or other kinds of scopes. And that has nothing to do with OOP!

I'm discovering how complicated a question this is. I'm not even ready to provide a response myself yet! I'll respond a bit later 🙂

Anyway, breaking things into parts does NOT mean you have modularity -- less likely if there's a hierarchy involved.

Modularity is about how "whole" those pieces are: Do they make sense in isolation, or are they highly dependent / interconnected with each other?

I'll add more in a bit, I have to step away

Each module should be it's own world -- not a fragment of a larger picture. However, it may be an inhabitant of a larger world, or may have inhabitants of its own. But it should have no knowledge of the outside world, or of the inside world of its inhabitants. It may know if the interface-boundary of its inhabitants, but without their knowledge.

So for example, a Car may have an engine and doors and wheels, and might have behaviors (methods) like drive forward, turn left, etc. But "pick up the kids at noon" is NOT part of a Car, but part of a bigger scenario that may contain a car. That scenario would know that the car can drive and turn, etc., but it's naive to how the car actually does it.

Sometimes there's the illusion of modularity in software, where a larger behavior is handed off from one thing to the next; but that's not modularity, because you pull one thing out of the pipeline, and you've broken that overall behavior. Generally speaking, you want "vertical" slicing, not horizontal. So instead of A hands off to B hands off to C (which slices what should be one while picture into fragments), you instead have the whole flow exist as one whole thing, and it may call down into A, B, and C. But for example, the code that updates your database should not "know" that it's supposed to do that after the user clicks a button.

One clear indicator of broken modularity is when something tries to "happen at the right time" (rather than just letting the outer environment call into it as needed). This can look like event listeners, AOP, pipelines, or components communicating indirectly with each other through some "mediator" or "command" pattern. Those things can have their place, but if it's to split apart a larger whole, rather than to incorporate multiple unrelated whole things that don't need to work together, than that's not modularity: you're going to have to reverse engineer everything and trace the cause and effect all over the place, to really understand what's happening in that code, and you're opening the door for race conditions and unexpected breakages when things change. The whole point of modularity is to make it easy to understand and change things.

... I've thought a lot about this before :)

You can see a program as a hierarchy of modules and scopes, or you can see it as a network of scopes calling other scopes. Which is more important? The hierarchy doesn't feel like the important piece here.

On a different note, there's a common idiom of decomposing programs into layers (MVC, app/lib/kernel, etc.) That's arguably a hierarchical organization even though there's no containment. The network is a DAG, and it has a definite 'grain' to it. This feels like an important lens. Arguably here we already have tags, in the form of providers and so on.

Some things fit multiple categories, or can play multiple roles, and that does not slice cleanly into exclusive buckets@Dan Cook Yes, I agree. There's a need to be able to categorise something without allocating it to an exclusive location (a bucket).

Each module [...] may be an inhabitant of a larger world, or may have inhabitants of its own. But it should have no knowledge of the outside world, or of the inside world of its inhabitants.How do you identity whether a module has inappropriate "knowledge" of something external? You must be talking about Objects specifically, right? Classic information hiding concerns etc. Your answers all seem to be in the context of OOP and method calls.

Generally speaking, you want "vertical" slicing, not horizontalI'm not really clear what distinction you're trying to make there. Are you arguing for call stacks (RPC) over message-passing and/or events? Call stacks are definitely banned from my own design. They're irredeemable in a distributed system.

you're going to have to reverse engineer everything and trace the cause and effect all over the place, to really understand what's happening in that code, and you're opening the door for race conditions and unexpected breakages when things changeThese are problems induced by poor language and tool designs; they're not really an immediate consequence of having an event system. I agree that if you have some kind of event system, then it has to be designed to play a carefully thought-out role in a larger architecture and in a way that can't be abused or "used wrong".

I feel like your overall point is a condemnation of existing software architectures and design patterns, rather than a denouncement (or advocation) of any particular abstract programming primitives. I don't know what else to say to that. I'm certainly not about to defend the status quo of software engineering!

Kartik Agaram You're talking about code, but what about information? Are you saying that a piece of information (e.g. an employee record, or the state of a video game) should be represented as a network of scopes? How do I address that as a self-contained unit?

Data needs to be modularised too. Not just code!

This reminds me of A City Is Not A Tree, where he says a city should be modelled as a semilattice. In particular it has some discussion similar to above about when things are truly "separate" or not

The classic paper https://www.cc.gatech.edu/classes/AY2013/cs7601_spring/papers/Simon-Complexity.pdf has a lot to say about hierarchy, and where it comes from. It's hard to not see hierarchies everywhere, although it isn’t really clear if that's because they are everywhere, or if it's because we just like to see them everywhere. (Hint: it's us. Our brains love containment as a foundational structure for sense-making.)

In the abstract, if making a "module" is taking together N nodes in a graph such that internal connections are isolated from ones that go thru the module boundary, then not allowing hierarchy at all will limit the "arity reduction", since either there will be a ton of modules, or modules will contain a ton of nodes. Hierarchy will help with the cognitive complexity of the graph, IF the average amount of edges is low or very clustered. The problem with hierarchy is what happens when a new edge gets introduced, and the system doesn't allow re-running the modularization heuristics from zero 🙂

One thing I'll say it's that it often makes sense to have hierarchy of the data model, and you'll almost always have a hierarchal execution model (e.g. data composition and function composition). But it's a mistake to shoe-horn one into the other. That's the mistake I see most often in OO programs: This part of the use case touches this data (class), so that class owns that slice of the behavior, e.g. Car.DriveToTheStore() (or maybe maybe MapNavigator.DriveToTheStorePart1() calls Car.DriveToThrStorePart2(), etc.)

@Dan Cook to illustrate with another example, paper.cut() this for me is a wrong way to model stuff. Somethings are just data and can’t do anything. They should be acted upon instead. It should be cut(paper).

Multimethods are useful here.

Ope - Exactly. I'd say that any proper module are just x, and only contain functionality (if any) for acting on that x. That x can be simple data, or a whole system -- so long as it's a whole system (which can contain other entities that are also whole in the same way). A module does not act on its neighbor or outside world, only on its contents aka "itself".

@Nick Smith - That's what I mean by "vertical" slices (e.g. "down" = my constituents) versus "horizontal" (e.g. I'm the middle part, and my neighbors are the left and right parts, of some bigger whole, and none of us can be understood in isolation).

I saw an ad once, of a boy walking through his house commanding IOT objects to do random things: "Fridge, turn on the kitchen lights. Toaster, read my email. TV, set an alarm on my phone". Or something like that.

I am 100% sure these map to different processors in your brain. The exclusivity of a bucket allows you to leverage your Location powers. As things like the Mind Palace technique, and the trick of returning to the place you lost your train of thought, show, leveraging our sense of physical location is very powerful.

Tags are also brilliant, but they leverage a different thing, which is our natural ability to categorise and stereotype. This is mainly useful for describing queries and commands.

I remember having an argument about 15 years ago with somebody who was saying that filesystems should be rewritten to be based only on tags, as it's more general. They made the (good) point that it's absurd to distinguish between /usr/photos/wedding/ and /usr/wedding/photos/. I still struggle to explain why I hate the idea so gutturally, but I do. I can't relax until I can say "that's there's, that's here" etc.

Basically the ideal system for me is built on a bedrock of buckets, but with the ability to arbitrary query it based on properties, including arbitrary user defined tags. Something's location, i.e. it's filepath can be considered an implicit tag but it should be recognised as fundamentally different.

I am actually making a system like this; basically a Lucene+Tika powered add-on for a filesystem. Impossible to know how universal my preferences are 🤷

There was a good bit written about this distinction in the early 00s, trying to tease out the differences using the terms taxonomy and folksonomy. Some of that work might be of interest to your concerns.

Chris Knott Do kangaroos go in /animals/australian or /animals/marsupial? Is a hot dog a sandwich?

I think the point is, it doesn't matter, there's value in it going Somewhere if that allows you to make use of a sense of location.

Where does a spork go in a kitchen? I bet it varies by person, but I bet all of those people would remember where they put it, in their own kitchen. In someone else's kitchen they would have to start querying and searching, for sure.

I recently sorted out my Lego so been thinking about this a lot 😅

Chris Knott So in the "models for information" thread, Chris Granger was talking about how he prefers the idea of "namespaced tags", which look a lot like file paths, so you can write something like /usr/photo/wedding/. The distinction from exclusive buckets though, is that you can reference a single datum under multiple different paths! How does that idea feel to you?

@Jack Rusher Those terms are leading me to some interesting resources. Thank you!

@Nick Smith You mention a lot of different things as part of the “bucket” category, so it’s a little difficult to come up with a single, universal answer to what the differences are.

Mathematically, a set and a tag can both be just a binary relation without any explicit properties assumed. Then they could be the same, behaviorally.

A list implies some sort of order on multiple relations of that kind, which a tag doesn’t (although you’d always have to order elements tagged with the same tag in some way to display them).

Tables and databases add more assumptions on the relations they model. Are they reflexive? Transitive (which you need to model hierarchy)? Etc.

Tags probably feel more flexible because they don’t assume anything about the relation they model other than that there exists one (perhaps if it’s one- or two-way?), while “buckets”, especially when seen as containers, do assume that these relations have additional properties which makes them less “flexible”.

At the end of the day though, the “purity” of tags is diluted by practical concerns, e.g. how to display all elements tagged with the same tag, and then you’re forced to ensure the relations have certain properties, and suddenly you’re back in container-land, just that the same elements can now also be in several containers at once. And then you run into questions of identity: if I change that element in one category, does it change everywhere, or does it just change in the current context (was it just a copy)?

Also, not hard-coding any properties of these relations basically just shifts the burden on the user; they still need to remember what a tag means to them and keeping different kinds of meaning apart. That’s when they suddenly re-introduce containment with hacks like namespaces or paths.

Technically, modeling fewer assumptions on relations gives you more flexibility in exchange for performance, as the system can’t make certain assumptions (e.g. order) which could be used for optimizations (e.g. indices).

Thankfully, the way the system seems to work from a user’s perspective doesn’t necessarily have to be the way it’s implemented. That probably is what good abstractions are about — they’re not just hiding something, they’re hiding a dramatically different structure that you wouldn’t expect from how the system behaves — and hopefully the way it behaves is easier to understand than how it actually works.

Stefan Lesser I think we're going off into the weeds a bit here. I'm (personally) not interested in talking about specific data structures or data models in this thread. That kind of discussion is best reserved for the thread I initiated yesterday. I listed a couple of examples only to hint at how the abstract notion of a "bucket" can manifest.

“buckets”, especially when seen as containers, do assume that these relations have additional properties which makes them less “flexible”.I think the actual interface that users are presented with is going to be the main driver of assumptions about what a bucket means. My definition of "bucket" here means simply an exclusive location that a datum is considered to "live" within. That's extremely broad, and my hope for this thread is that we can work out the relevance and utility of buckets in comparison to tags.

And then you run into questions of identity: if I change that element in one category, does it change everywhere, or does it just change in the current context (was it just a copy)?I actually believe this to be the only difference between buckets and tags. In the presence of mutation, the two concepts may behave differently, and without mutation, they are equivalent. I've not yet had time to develop detailed reasoning as to why this might be the case. I'll hopefully post something here in the next 24h.

Set ~= Tag isn’t it?

As for List / Table: they have an extra layer of connection via integers (list 1D, table 2D), but you could replace this “mesh” with adding proper next/prev information (tags) everywhere. Integers themselves can be modeled! You should check out Idris!

list of [1,2,3] VS

set of numbers(1,3,2) + nexts( (1,2) (2,3) ) + prevs( (2,1), (3,2) ) + the automation which adds these pieces of information always on change.

To me, naturally, if we would like to be really really precise we would only use Sets + adding out extra layers of information to the models by ourselves, but Integers are so useful and make us so capable to hack around with a degree of certaincy without fluff. However, using integers exposes logic you can use to reason about your code.

On the other hand, building on Integers means you are exposed to “vulnerability”, because you don’t use the “real interfaces” Integers themselves really do have behind. (eg. successor aka “next”/ successor^-1 aka “prev”)

We're off-topic again! I'm going to delete the mentions of those concrete data structures from the question. They are not really what I was trying to talk about at all here.

Ah. Ok. Then my answer is: nothing. 😄 But maybe I don’t get the idea.

I added an extra note to the question clarifying the conceptual difference.

I think the key distinguisher is the exclusivity. This is what maps to our natural understanding of location and identity. An object can only be in one place. If otherwise identical objects are in different locations we perceive them as copies. I think this persists even if mutations on one affect the other. We perceive the "other object" as being magically affected "at a distance", we can't perceive it as one object with two locations. The notion than an object can only be in one location at a time, overpowers the related notion that objects can easily change their location and persist their identity.

This is not quite the same as containment. The features of physical containment are transitivity and anti-reflectivity. If A contains B, then B cannot contain A. If A contains B, and B contains C, then C is also inside A (indirectly).

A filesystem obeys these physical properties, but something like Python's list does not obey either. This is why users get confused by lists that contain each other, and objects being changed when they are in two lists. It is also why symlinks can be so confusing. (You thought you were deleting a copy? Too bad sucker!)

Let’s go off-topic to the other side then… :-)

When you say “interface” I hear “how users understand it”. Then we can talk about my favorite topic: image schemas. These are cognitive patterns that we all use to structure our understanding.

A “bucket” is an instance of the container schema. Things are either in it or not, and it has a boundary (which we might or might not be able to describe precisely). That means that the container is of a different quality than the elements it contains.

A “tag” is an instance of the link schema. It just means there is something that connects the things at the two ends like a rope (imagine the force you feel when two items are tied together with a rope; and it is such physicality that gives meaning to the abstract concept). The two items connected to each other don’t automatically have different “status” like in the container version, although they could have through other schemas in effect simultaneously.

We can cope with the same things being in different containers at once, usually through frames, which are larger contexts of experience. E.g. when you say “cut the flesh” and you are in a restaurant, you evoke completely different images than when you’re a surgeon in a hospital. And it even works out ok on the surgeon’s night out to the steak house. ;-)

Now if you use a tag to signify membership to a group, e.g. all blog articles tagged with “technology”, then it’s used much more like a container and likely understood based on the container schema.

There are also more complex image schemas that could be relevant, for instance whole-part, which is a little like a container, but where the elements don’t just have to be present, but also need to be arranged in a certain configuration for the whole to emerge. E.g. if you disassemble your car and still have all the parts, it’s not really a car anymore. (That’s also behind the layers example Kartik made in the other thread.)

Or take the center-periphery schema. This is how we understand gradual or fuzzy relations. Some things are “core” or central, and if you change them the thing is no longer the same thing, e,g. a tree where you cut the trunk is still a tree, but it’s not the same tree anymore. Other things are peripheral and can change, but if they do the thing still stays the same. E.g. if the tree loses all leads in winter it’s still the same tree. Oh, did you have a haircut recently? No problem, still you.

The difference between “bucket” and “tag” can be as simple as container vs. link, depending on how you use it.

If the identity of an object changes when it’s modified in different places, does it mean it wasn’t supposed to be the same thing in the first place? This is at the core(!) of programming issues around value vs. reference types and affects your system design. What parts of the elements tagged or put into buckets can change without making them a different thing?

I think I disagree that tagging is the same as linking. Tagging to me is a kind of a categorizing/grouping. It's a type of non-exclusive container. Then I would put "bucket" as a separate image schema of exclusive container, which to me feels like a v important distinction. Mistaking a non-exclusive container for an exclusive container is surely to root of those spooky action at a distance gotchas.

deep in the weeds here, but i think that the core of the ambiguity here is about identity vs value. in my head a value is something you can “get to” by eg a query or a computation (paths, links), whereas an identity refers to more of a physical object you might find in a place (something you can mutate). if you change a path or a computation, you get a different value. if you change “a thing”, the values update (properties of that thing update? language is hard). that kind of implies that a paths and computations are “things” that “give you” values, which strikes me as both exciting and scary (too meta?)

if that’s the case, buckets are things that contain things whereas tags are things that return or evaluate to values (lists of things). i think?

i think the system needs to address both copy & paste and linking but it needs to do so in a closed way (algebraically)

I’ve been really enjoying this podcast. I don’t always understand what Ryan is saying but it is interesting and relaxing 😌

Ivan Reese I think the layered state stuff speaks to some of your questions about variables

(just updated because I managed to post the old link)

Perhaps paradigm, exceptions, or expressive power? (As measured by some formal heuristic, such as statement ratio or line ratio, often compared to C)

I don’t really like my contributions above so perhaps I can ask a useful question. Could you further define axes, in this context. Should they be formal properties? Should the options be mutually exclusive?

Size? There’s a lot of languages, i.e. Lua, where a major feature they espouse is being small in size and embeddable, for example

I mean something concrete that’s a property of the language not just what it affords. Like you can do functional programming in the broad sense in python for example but Haskell affords it better so I want to keep paradigms out since the programmer can decide to use different paradigms. I remember something Phillip Wadler said(I think he is the one who said it) that Haskell is the best imperative language.

like a friend pointed out Memory management - Garbage collection. Though I wonder if garbage collection is going a bit into the implementation vs. what the language actually is.

Pointers or no pointers. Mutable variables

scope and name resolution!

https://www.famicol.in/language_checklist.html

(There needs to be a :sigh: reaction)

Considering generics, one of the axes could be: how dynamic is polymorphism, and where is it handled? (Is it just based on data types, or are there protocols, or templates, or generics, etc)

Also... how large is the type-level part of the language? Is it just primitive hints like Int / String, can you define new types, or is it basically a whole other execution context (a la Idris)?

Another axis (or, rather, a space): what details of the underlying stack do they reveal, redefine, or obscure?

How are they meant to be evaluated? Effectful statements in order? Some sort of tree with branches in parallel? Some sort of search through a possibility space? No notion of evaluation at all? (Not all programming languages are meant to "run" on a "computer")

(Many of these points won't lie on a line unless you sort of... rank them from "totally chill and normal and boring" to "very eccentric, don't invite to the office", which I think is a wonderful way to rank ideas in computer science)

Instead of "mutable variables", I prefer 'pure functions and referentially transparent expressions' vs 'shared mutable state'. Curv is in the former category, but it supports mutable local variables and a while loop. Curv wants to be a better imperative language than Haskell.

In my experience:

• automatically garbage collected vs. not
• statically vs. dynamically typed
• inheritance-oriented vs. composition-orientedThe three most useful high-level axes for comparing languages.

The method by which garbage on the heap is collected affects what language primitives are available, what is efficient, and how you write code. There are multiple choices: tracing gc (most languages but originally Lisp), C, Rust, Swift, etc.

I can confirm regarding design documents (we called them Tech Design Docs - TDD - not the best name since it is confused with test driven development).

Regarding documenting different options, usually we would consult the TDD or ask somebody, which is certainly not very approachable.

And if there was a clearly better alternative to implement something, usually we would describe it in TODO comment next to the implementation.

However, I wonder how often somebody really asks (why wasn't this done in this way), from my experience, I haven't seen that happening very often, at least not at the degree where reviewing all the previously considered options would be useful. But it does happen sometimes, and then TDDs usually worked out (but with lot of digging, figuring out the latest one and so on).

Just thinking out loud, I might on the other hand find it useful if they were more available, but it is hard to imagine right now.

Yeah, I'm not aware of existing solutions, but this is a problem I've often felt and tried to hack my way towards.

On my projects I use a system of layers so that I can keep track of past versions with fewer features. And continue to keep them working in CI: http://akkartik.name/post/wart-layers. Layers form a linear sequence, and I've often considered introducing branches. But I've never found a strong concrete use case and clean way to do this.

One easy approach may be branches as you mentioned coupled with links to those branches (on GitHub or wherever) sprinkled in a few key places that the branch modifies. I like links 🙂 (https://futureofcoding.slack.com/archives/C5T9GPWFL/p1592285060323600?thread_ts=1592252831.319600&cid=C5T9GPWFL)

Here's another attempt in an old project of mine, creating a subdirectory called 'alternatives' with clones of the top-level at specific points: https://github.com/akkartik/wart/tree/master/alternatives/lite. And each of them has a subdirectory called 'orig' with the baseline to compare against. Like I said, totally hacky.

I've been thinking more recently of a code project as a set of knowledge in a domain or to solve a problem. I wish this knowledge could not only be on the final decisions but also on wrong decisions and the paths not chosen

It sounds like you want to distribute branches (makes me think of multiplicative distributivity, anyway) down to the source file or function level, or something like that? That seems like an interesting idea. The fundamental obstacle in my view is that a lot of those roads not followed will, by nature, affect the design of large swathes of the project, like whether certain files exist at all. It might be that the best thing is to add annotations, uh, somewhere, about where those branches' existence should be surfaced in the VCS or code browsing UI. E.g., somehow when you're looking at the router for your REST API, there's a button that shows the branch where you tried gRPC.

You're right that many alternatives may require pulling in more changes. But not all. There's lots of value in simpler scenarios.

For example, here's a self-contained alternative in a code comment that I've found useful for 10+ years.

https://github.com/arclanguage/anarki/blob/0913288ec1477f2598e8095a2b5e9b14eb97dc4e/arc.arc#L662

Good point (and nice example). It might be easy to just branch on the size and scope of the diff when deciding whether to show it inline. Less than 20 lines and all in one file? Just show it, otherwise put it behind a button.

I learnt the basics via https://store.steampowered.com/app/383730/RPG_Maker_2000/. You can use it to make little gameboy-like RPG games with point n' click coding (I wanted to make my own Pokemon obviously).

At first if I wanted something different to happen I just duplicated the map, so if you entered the forest, and went back to town you'd end up in an entirely new but identical map with characters saying new things. I had 100s of identical maps but the great thing about the RPG Maker is you could download and see inside other people games.

So I downloaded someone else's game and they only had one copy of each map and instead they used these things called switches (boolean values) to make different things happen after particular events. Soon my games had 1000s of switches. Eventually, I downloaded another game and it had barely any switches instead it had a single storyline variable that would increment by one every time something significant happened.

From there I learnt to use variables in more complex ways, and whilst I only pursued programming as a career fairly recently, I've used a basic level of programming with almost everything I do, and that's stuck with me as a foundation ever since.

When I was a kid, we would get the "Things You Never Knew Existed" catalog in the mail. For those not familiar, it was full of mostly useless "as seen on TV" type products. I loved looking at all the weird stuff in that magazine...

I somehow convinced my parents to order two things from that magazine. First was a deck of Pokemon cards. Second was "Interplay's learn to program BASIC on CD-ROM"

I am a product of late 90's edu-tainment 🤣

I found a youtube video of "learn to program BASIC" for your horror and entertainment:

https://www.youtube.com/watch?v=uBYz9syhNAA

@Ray Imber this aesthetic is clearly what FoC projects are missing

So technically I was taught QBasic in secondary school but I don’t remember any of it.

Then first year in uni picked up Java but never got to the point where I felt competent so I dropped it.

Then I found Python In year 3 I think and I loved it! Compared to C, which was another language I dabbled in because I wanted to write embedded systems; It had/has better UX. With C, I guess I was way too sloppy and would get frustrated by the segmentation faults + it was really low level while python had this robust standard library.

To get proficient, my gateway drug was competitive programming - Hackerrank in particular. It was/is fun coming up with solution and it appealed to my math/competitive side a lot. Was a side thing and planned to write embedded systems but here I am 🙂

html, then copypasting javascript without knowing what I was doing (20 years ago), then c

cheating at warcraft 1 bychanging the .ini files to finish the campaign because i was too young to understand how to be even marginally good at it lol

Ryan King that's exactly how I started coding 😛 I got my hands on a burned copy of Macromedia Flash 5 and started with bouncing ball animations. I remember using a lot of free tutorials. At some point I learned to trigger frame changes with buttons, and that's all I needed to start making cursor mazes where you need to avoid the walls and moving obstacles. All of the obstacles were just animated buttons that you weren't allowed to hover over.

That's all possible without having to write any ActionScript, which I did eventually learn, but I think it was a steep learning curve from there on.

I wish I could remember exactly what features of the editor/model made things accessible for me at 13. But I'd guess it had to do with having a traditional graphics editor with the capability of attaching code/behavior to objects on the canvas.

It was QBasic for me. At some point I discovered that Microsoft Office came with VBA which was fun because I could make buttons and stuff.

Ope maybe we should be teaching programming in a way that fits into Bartle's taxonomy of player types ( although I'm a bit iffy on psychological taxonomies )

@Zach Potter I'm no scientist but that's theory validated for me! 🎉

As a kid I was briefly exposed to programming through gamemaker and a community college java class. But it never really stuck.

As a college freshman I took intro to programming (also in java 😭) but at the time I didnt really have a reason to want to learn to program so it never really stuck.

I started programming seriously my senior year. I became motivated to actually learn to program after I learned about people I felt were doing interesting things with computers (Bret Victor, Paul Graham, _why, David Nolen, Rich Hickey). I also felt motivated to learn because I was graduating from an unknown liberal arts school with a low gpa 😅.

I suspect for most people they need to first understand how they can use programming to do things that are meaningful to them (make games, music, money, “I want to be like x person”, learn things, etc.) in order to have the discipline to learn. I think starting with a domain specific tool for doing meaningful things like a game maker and allowing users to “open the hood” with programming is a good way to scaffold learning.

100% - if they taught me how to play the Mighty Morphin' Power Ranges theme instead of hot cross buns on the recorder in kindergarden, I imagine I'd at least have some musical talent right now.

I saw a computer for the first time in my life at age 18, in my sophomore year of undergrad. However, my dad was always sure that it was the field for me (even though he had no computer knowledge either). So he bought me a computer book when I was in IX or X grade -- on dBASE III Plus. And I read it a few times over the next few years. In my freshman year I was declared CS but had no CS courses, and we weren't permitted yet into that holy of holies, the computer lab. But I could access computer books in the library. I read the K&R C book and various others that year (including one that had an Alice in Wonderland theme throughout; I no longer remember the title, but I really enjoyed that one). But still without access to a computer.

I was pretty good at Math and Physics in high school, but it was pretty clear by the end of my sophomore year that computers were the thing for me: I never fell asleep in front of a computer the way I fell asleep over my textbooks for other subjects. And the feeling of flow was intensely pleasurable. In my junior year there was a contest to build an AI for a boardgame, and I won it by implementing A* search. So it seems clear that I must have had a few hundred hours of practice in that first year.

If I recall well, I first program in Basic with a Comodore 64 in a school small workshop when I was a kid (maybe 12). Then also at school and with some friend's computer I continue Basic on computers like Sinclair ZX81, MO5 and TO7. I don't recall well all what I have created that time, just one thing: some program related to the 1986 FIFA World Cup (at that time I loved soccer) to show teams and results or something like that with some graphic part. I recall using some grid paper to figure out the pixels for the logo as I had to encode it pixel by pixel, there was no drawing tool at that time.

Later I owned my first computer an Atari 520 STF, in fact co-own because it was the family computer. I was amazed by the possibilities of the computer and in particular there was MIDI input/output which allows to use the computer for music and also to play network games (by connecting two Atari with MIDI). I uses mostly for games of course. But I also continued programming with Basic for some little programs every now and then.

During all that period it was in fact a bit frustrating, because several time I wanted to develop games, but it was very complicated at that time, and I have started some but never really finished one. But, in fact programming was my thing, so I was happy to create things whatever it was.

Ryan King first time I am hearing of that taxonomy. (love how I get exposed to random stuff like this here) 🙂

Kartik Agaram This reminds me a little bit of my friend Karsten Schmidt's initial experience. He did see a computer, but there was only one for all the kids at his school in what was then East Germany, so he did all of his programming using paper and a (IIRC handwritten) copy of the manual. 🙂 He went on to do all sorts of good work: http://postspectacular.com

I started with basic on the commodore 64 quickly followed by assembly language on both c64 (1985/86) and later amiga. Then pascal, c, delphi, c#, javascript, actionscript3, objective-c, php and knowadays mainly c# and typescript. And I've just started with Rust. Mostly I've learned all of it by myself by reading books/internet, trying things out/building applications and I had formal education here in the Netherlands as well ("Hogere Informatica" in Eindhoven in the beginning of the 90's).

Kartik Agaram's story makes me appreciate that I was extremely privileged to have access to a computer at a young age. It's easy to forget that it is a privilege (even as computer access is becoming more ubiquitous with younger generations).

Anecdotally, in my experience, programmers tend to be slightly more auto-didactic than the general population. This thread seems to confirm that. It's an interesting correlation. I'm curious about the connections there. 🤔

I started with basic on 8bit computers like Atari 800, TRS-80, Commodore 64, Apple II etc.... I learned by experimenting and typing in programs from various magazines and having to debug the typos out. I can't really imagine that would be a good way to learn in 2020 tho.

To the original question though about Flash. My gut tells me it's easier and more interesting to take more complete systems and modify. I'm thinking VBA meets Unity (I'm sure this exists, maybe Roblox?). The point is if I have a simple game and I just double click an object in that game and add 1-3 lines of code to change something, I feel like that would be a more approachable way to start than the start from nothing way.

There was a failed kickstarter trying to teach coding in a game (not the same thing as above but similar)

https://www.google.com/search?q=code+hero+kickstarter

The distinction I'm trying to make from Unity itself (and maybe this not an important distinction, not sure) is that it seems more intuitive to just double click the object you want to edit and get presented with a function to fill out the same way VBA did for forms. That is not how it works in Unity. In Unity you add script components which give you a blank script and I think requires a lot more knowledge what to put in that script where was if it was designed for learning maybe there would be more direction or more or simpler and easily discoverable places to add small snippets of code to change things.

I'm another one that came in through gaming, wanting to make games. My first actual programming was something called DarkBASIC. A sort of BASIC with DirectX bolted on. I don't think it was popular outside of the UK, but it used to come on the cover disc of PC Gamer. Probably picked that up around 13yo-14yo. Basically self-taught by editing example source code and seeing what it did. I remember when I first installed DarkBASIC and ran it, I thought it hadn't booted properly. I was like "It's just a blank screen? It's a text editor...? How do I make games". I literally closed it and opened it again several times 😀.

Before that (~11yo) I used to make games in MS Publisher that involved moving images around with the mouse (drag-dropping). Would have worked much better with physical paper, but I wanted it to be a "computer game". Before that (~8yo) I used to make mazes on paper which had sort of RPG-like elements e.g. a corridor with spikes along the side ("You have to go fast along this bit!!"). So I definitely think the motivation was to tied to kind of world-building creativity.

I separately loved puzzles and later, maths, and only a long time after learning to program did I realise there was a thing "computer science" that involved maths-like puzzles and thinking.

Dunno if this is a weird thing so say or not but if there's any women out there lurking I'd be really interested in hearing how you got into coding

Scott Anderson I'm still waiting too. 🙂

My background started in sociolinguistics, financial auditing and economics. I came to Future of Coding after founding a tech company on a mission to reduce emissions by 50% for average Canadians, by leveraging gaps in the real-estate tech market and careful UX design. So this conversation is interesting to me.

It's important to remember how much the phenomena of surveillance capitalism and platform capitalism are not new, just the terms are.

One could argue that culture always has been the emergent "data essence" of a society, which different players (eg. Religion or the army) have used for their own ends. Viewing data and culture as a similar thing is helpful, because it allows us to apply known methods from culture conflicts to the world of surveillance. For example, parallels between decentralized cultures and decentralized data.

With Platform capitalism, how different is it really from what a market has always been? Every farmers market that is owned by a third party is effectively a two-sided marketplace that the landlord owns. 1,000 years ago the landlord had to guess if the merchant was making more money, and then could be gauged a bit more, now it's just digital and simple.

Digitization hasn't actually created the emergence of a new phenomenon, it's moreso increased the scale at which old phenomenon can operate. The world has become more flat. Everyone can be in every market simultaneously, even before we start employing barter bots. Even tinder allows me to be swept upon by matches 500k away after Covid-19....

Up to here there are 3 basic points I have:

1. Data and culture are the same thing

2. Platforms are just digital markets with increased information access by the platform owner

3. The number of players in all games we play in the digital world, has grown exponentially

So how do we enact systemic change for the good?

Firstly, I think it's inevitable. That's a whole other discussion, but it is not hard to see the cosmic arc of history's bent towards justice, or the innate human goodness that happens https://www.theguardian.com/books/2020/may/09/the-real-lord-of-the-flies-what-happened-when-six-boys-were-shipwrecked-for-15-months.

Inevitability aside, I then ask, "is it getting better for me." And that's the selfish side. So the way I make it better for me is by playing the game to accelerate the inevitable process. The diamonds are in the delta.

The three basic points guide the methodology within our organization to do that.

• Avoid building a global brand, build decentralized data aligned to cultural boundaries
• Reduce the market's existence, eg. optimize for buying eggs at my neighbours backyard coop
• Align all our metrics and system optimization on the one constraint a flat world hasn't made disappear: neighbourhoods. We all live somewhere. Design success in one neighbourhood, and scale it to neighbourhoods. Ignore other metrics.

Along the way, I'm sure we'll get to test drive some pretty nifty tech solutions. 😜

@Tim Lipp great points, I would point out there’s a healthy debate about just how new platform capitalism is, there doesn’t seem to be a useful consensus yet. A good friend of mine is currently writing a paper on that exact question.

One of the best https://thebaffler.com/latest/capitalisms-new-clothes-morozov of the https://en.wikipedia.org/wiki/The_Age_of_Surveillance_Capitalism conception of surveillance capitalism by Evgeny Morozov addresses some of your points, though they do also recognise the model as important but incomplete.

To me, your second point seems incredibly important and easy to understate

2. Platforms are just digital markets with increased information access by the platform ownerI think there is danger in taking this to be something that’s been around forever, the increase in information available to the platform owners can be a monopolising force that we may not ever be able to escape incrementally. I’d argue we need more than just better company leaders if we want to change this, replacing Tim Cook with a generous anti-monopoly utopian thinker will only go so far.

Another concept defined within surveillance capitalism and its follow-up refinements is that of digital others i.e. your facebook profile + the many bits of associated metadata.

It’s been argued quite effectively that highly monopolistic power has been formed by tying producers and consumers together into one entity. This is evident on social media platforms such as Facebook, where your very consumption and interaction with the platform as a consumer is, to Facebook, a valuable production of resources in the form of data.

Hmm, I'm not sure I understand the danger in drawing comparisons to historical analogs, but your point about digital others brings up another interesting thread about the implications of when bots pass the Turing test.

Why is it dangerous to view platform markets as just markets with more information? A small example happened 500 years ago with the invention of the receipt, it gave increased certainty for a transaction, but also added risk that information could be used against the parties involved.

Different cultures have different marriage norms. Some are polyamorous, some are monogomous, some even practice polyandry. If we were to equate Tim Cook to the perfect man, and profitable transactions to sex, are there healthy upper limits to profits as there are healthy upper limits to sex? Even if I have 20 lives, they can only make me so happy.

Perhaps the dilemma with profits is that the upper limits follow a much bigger power law than with other "wins" such as sex, and we haven't yet evolved an understanding of what the counterforce is. At the very least one exists in the reality that Tim Cook will probably get bored one day. Perhaps we could speed up the process by creating an even more interesting game to play.

Thank you @Orion Reed for trying to create such an interesting discussion. Currently I’m on vacation for the next two weeks and I only have my phone with me but your inquiry really resonates with me. So I’m now challenged to condense the experience and knowledge of my last 20 years into some meaningful and tangible for you and all others interested. I will try my best ...

For quite a long time I was very much focused on (computer) technology but in the last two years that has changed. At first - like many in this community , I think? - I thought that we “just need better tools” to approach the problems we have capitalism being one manifestation of them. So functional programming and other technical design techniques attracted me. Eventually leading me to Ethereum - RChain and the Web3 movement in general. But soon these projects revealed even more clear the problems we have when interacting with technology.

The book from Yuval Noah harari reveal 3 Fundamental orders related to the unification of mankind: 1. Empire 2. Religion 3. Money

The idea of money being a tangible even moldable cultural/technical artefact , is a huge step forward. The blockchain space and the web3 movement as a technical implementation of this idea is a important step forward.

If this particular step could teach us something then it is: culture is a huge thing and because it is so huge and powerful it is also only slow to change. Nonetheless trying to engage or to create a more conscious way of culture shaping , creating a new meta perspective to reflect about society institutions and other cultural artefacts is a worthy goal in my opinion.

Ok so much for meta perspectives and their value. Now what could coding still contribute? I think we live in auch complex and contradicting times that sense making is paramount. In pursing auch thing I really like the concept of a Zettelkasten and I could see how quite small and cheap efforts from this community could help improve these ideas.

I always found it so interesting that nobody (as far as I noticed) noticed the giant gaps or paradox effects of capitalism stated like this:

Funding! (Again rifing on the money order but the other 2 are also very interesting)

So many people here show their projects but how are they evaluated?

If really useful tool could be created or Incubates here then why wasn’t there any effort to connect to existing web3 efforts or to local communities, like people who actually could need computing services ?

I mean this moves at least into two directions: one - there are people starving to use tools in a sovereign way but they can’t because most todays tools do not allow for sovereign use . Two - In some regards many people here have already tapped into the cognitive science potential but it’s still unstructured and undirected - no new habits or actual cultural practices besides : text chat, twitter (social media) , YouTube ...have been created.

But change as you @Orion Reed are asking for it , change always needs persisting habits. It needs to include the whole society not just programmers (inclusion discussion starting also but IMO still to few)

That’s all still quite vague but I expand and give examples from Zettelkasten and other experience gladly if you can find spots that resonate with you

Just a very quick note here: I tried a couple of times to introduce the web3 (just as a first step) as in interesting community to connect and extend to but this Medium this slack channel didn’t allow me to clear the problems or getting people interested in it in a meaningful way.

Just another note: meaningful thinking about how meaning mould are the systems we use and build, in the general channel here are over 1200 people and two (including me) replied - views could be also interesting. Twitter and other social media too, people with 700 followers get 1 RT or 1 reply when posting something. How meaning is that ? Should it be? Could it be?

Book recommendation for those interested in the evolution of society: Hanzi Freinacht’s « The Listening Society » and « Nordic Ideology ». On capitalism, he thinks (and I agree) that it will stay around but fade from center stage as a principle for the organization of society as material goods become less of a focus.

👍 his meta-modernism gives nice perspectives

Also agree with this in that thread https://twitter.com/terronk/status/1286774558314250240

I agree with this but as a very visual thinker I'm always a little conflicted. A small formula field can take an aeroplane cockpit worth of graphic interface and compress it down to a tiny box. But try to do too much with the formula field and it can easily get too complicated in its own right.

I recently got a look at a how a team at a Fortune 500 company was using Excel. They’re using it as at full custom complex app + database that any user can modify. What seemed remarkable to me is how poorly spreadsheets are currently designed for this use while also being better designed for this use than perhaps any other option. I used to be surprised when I heard about companies running major internal app off spreadsheets. Now I’m surprised everything doesn’t already run off something very much like them.

All the Twitter thread is very interesting. Thanks for sharing!

it hasn’t to my knowledge. a connection between trees and matrices is (as i understand it) critical to the efficient parallel compilation of the co-dfns subset of APL: https://youtu.be/lc4IjR1iJTg

The Brainfuck inspired minimal VM is more interesting than the language

Agreed, I like the minimal VM too; not sure about the portability claims, since it requires rust. I made a similarly minimalist VM for some early music language experiments I did; but really just because I was trying to do the easiest thing possible; not striving for a minimal VM per-se.