what is a pixel if not bits?

Presumably everything would have implicit location, and also be considered relative to other information

Any physical characteristics available for free, like color and brightness, would be metadata that could take on vast new connotations

We'd presume all information was in a larger frame and context, and that itself had characteristics important to each pixel itself (whether the 'picture' was coherent or stochastic)

To clarify terms, by pixel you mean three arrays, one each for RGB, and a fourth containing X/Y co-ordinates on a 2D grid (which could be a screen, or presumably could be abstracted?) Or other things?

I’m not sure how other primitive things necessary to make this work in the fashion described, like neighbour calculations, would be crafted in pixel-terms, or if the “instead of bits” interpretation is a bit too limiting.

If you’re interested in computing on a spatial grid, in which cells can contain color information, though, you might find agent-based modeling frameworks an interesting place to start.

A lot of our basic example sims in HASH are around things like color passing: hash.ai/models?query=color&sort=relevance&page=1

Some docs at hash.ai/docs/simulation if you want to have a play. It’s all free, and mostly open

Don't all bits have a context through which they are interpreted? Pixels are no different and fall under this perspective, they are after all just bits. I just don't see how CS theory would work if bits were replaced with pixels, or how computation in silicon would work, because in the end, the bits are really an abstraction over the physics run in hardware

@Tony Worm hence why either could serve as a base representation tool. however, although there’s no such thing as a purely ‘abstract’ bit, it sure feels that way based on how i’ve experienced CS. i think it’d be harder to hide the underlying physics (and other implicit contexts) with pixels.

Christopher Galtenberg haha speaking my language

@David Wilkinson (HASH) leaving the definition open ended, since there are many ways to approach it. but if pixels were the fundamental unit of information, i imagine the definition would have something to do with samples over space & time, with rgb sensors/displays as a special case. basically, in real systems, tensors (and tensor fields) are more fundamental than disembodied bits, and pixels (in the usual sense) carry a lot of that intuition for free. definitely a fan of the direction of HASH!

I consider a dynamic medium to be any medium for expressing ideas that can respond to actions or changes made by the recipient of the message.

This includes but is not limited to face-to-face conversations, spreadsheets, video games, text-based games, and most forms of simulation that the user can control or direct, such as physics or chemistry or biology simulations.

There are many of them.

You interact with them using whatever input and output devices the computer and the simulation support, including but not limited to keyboard, mouse, video, microphone, light pen, touchscreen, camera, game controller, VR headset, and position sensors.

It mediates any idea that one human wishes to communicate to another, and is especially useful for ideas that involve systems that change over time. For example, spreadsheets are often used to simulate and communicate information about projected earnings and expenses. One person creates a spreadsheet that embodies a certain model of how the organization works and certain predictions about what is likely to happen. They then give it to another person, who can explore the model and see what it predicts if different choices are made or different events occur in the near future.

What type of literacy it supports depends on the medium. Spreadsheets have probably done a lot to improve financial literacy. Video games have helped many people to understand how virtual combat systems work. (Whether or not that is useful information to know is beyond the scope of this comment.) Physics simulations help people to better understand Newtonian physics in an intuitive way, and to make fewer Aristotelian assumptions.

Ultimately, once we have a dynamic medium that anyone can use to express their ideas, meaning a personal dynamic medium, it may support deeper thinking about processes and systems in general.

Personally, although I don't think it will happen in our lifetime, I like to hope that it will ultimately make us smarter all the way around and help us to understand and solve our really big problems, like poverty, pollution, corruption, and war.

“Mojo of the thing is simulation ” - Alan Kay

@Personal Dynamic Media Interesting...

What do you mean by "the ideas can respond to interactions of the recipient"? Does that mean that eg. the spreadsheet is the idea, or does that mean that there is an agent/mechanism that updates the ideas/facts? Would this agent itself be a kind of medium?

In the case of a spreadsheet, the spreadsheet software is the medium.

The idea is whatever you are encoding in a particular spreadsheet workbook using the medium. For example, your model of how finances work at your company and how they will be affected by your projections for income and expenses.

Responding to the recipient means after you give me your workbook containing your model and your projections, I can make changes and observe the results. I can ask "what if" questions by checking to see what happens if our earnings end up being 30% lower than you project, for example.

@Personal Dynamic Media Would a video qualify as a dynamic medium? Certainly a spreadsheet has a richer interaction scheme; can one somehow measure the richness of interactions provided by the medium? For example, a spreadsheet that allows to "compute backwards" (set the result and observe the possible compatible input configurations) could be richer than a regular spreadsheet. Is there then a "maximally rich" dynamic medium?

Would dataflow fit your requirement list? en.wikipedia.org/wiki/Dataflow

Timely Dataflow: dl.acm.org/doi/abs/10.1145/2517349.2522738

Diferential Dataflow: link.springer.com/content/pdf/10.1007/978-3-662-46678-0_5.pdf

More "concrete": materialize.com/life-in-differential-dataflow

Things to look up: Adapton, Self adjusting computation, dataflow, Salsa (rust), Reactive programming/systems

glimmer is another such system, though more focused on the UI side

Jane street has a library called incremental based on works by Umut Acar.

blog.janestreet.com/introducing-incremental

ccs.neu.edu/home/amal/papers/impselfadj.pdf

I've found dissertations to be the best place to learn hard technical things. They are long. But almost always assume less prior knowledge and give you lots of references.

Thank you all for all the pointers so far. I have a lot to look through 🙂

dagger.io can do this, built on BuildKit and CUE (cue/flow)

auto dependency detection and knows when inputs have changed and something needs to be rerun / recalced

It's a lot more than a devkit for CICD

scattered-thoughts.net/writing/an-opinionated-map-of-incremental-and-streaming-systems

microsoft.com/en-us/research/uploads/prod/2018/03/build-systems.pdf

FRP: conal.net/papers/frp.html

Branching off FRP, i would recommend a javascript libray like @vue/reactivity, solidjs, ect.

import {ref, computed} from ‘@vue/reactivity’



const count = ref(0)

const doubleCount = computed(() => count.value * 2)



console.log(count.value, doubleCount.value) // 0, 1



count.value++



console.log(count.value, doubleCount.value) // 1, 2

observablehq.com notebooks are functional reactive graphs that do the correct thing when code changes (like spreadhseets). The runtime is open source: github.com/observablehq/runtime. An explanation of the execution is here: observablehq.com/@observablehq/how-observable-runs

Partial evaluation may also be relevant.

Maya uses a Directed Acyclic Graph, I guess this is one implementation of DataFlow. In Maya’s system, nodes are invalidated when data is pushed onto connections on the node, and the dirty state is propagated to all dependent nodes. When connections are read, nodes are computed up the chain until the connection can be validated. Maya has an interesting design where nodes can also be part of a hierarchy (used to evaluate a scene graph for geometry transforms, etc.). Since I read about this ‘first’, a lot of my graph designs use a similar approach; it’s the first thing that made sense to me.

Thank you all! I'm going to study Jane Street’s Incremental next, and I also found the “map” by Jamie Brandon very helpful.

My context is I want to prototype a system for storing DSL models in the form of ASTs and doing transformations on them (type checking, interpretation, etc.). I don’t think I will find an existing implementation I could directly reuse, but I hope to borrow ideas.

Adapton looks like what I’m looking for, too.

I love this post: lord.io/spreadsheets

The "further reading" section at the bottom also has great pointers.