[March 26th, 2023 5:24 AM] fp: So, I'm an EUP person at heart, and this ChatGPT thing has obviously got me thinking all over again about what programming would look like to a non-technical person. At heart, I feel it should be like they're "casting spells" over reality (or virtual reality). This tips into the area of cognitive modelling: how close the physical manifestation needs to be to be able to be abstracted up to a satisfying cognitive model that matches the human's intention. In other words, you cast a spell "make that banana green!" and it comes back a lurid dayglo green, that would be a cognitive dissonance because really, you'd expect to simply get a very unripe-looking banana. What are the elements of this formalised spell-casting, this "programming system"? You have objects (banana, this one, not all ones), attributes (green, the correct one!), a sense of time or evolution (went from yellow to green). You start to get into Roget's Thesaurus land: what are the key concepts for describing the world, our human world?

Anyway, just a splat of the stuff buzzing around my head right now. Thoughts?

[March 24th, 2023 12:17 AM] wtaysom: Friends, I don't know what to make of developments in AI these days. Having worked on dialog systems in the aughts and having loosely followed developments since (I recall preparing a talk around 2010 which left me pretty enthusiastic about ML applications in contrast to the App-and-Facebookification of "tech" — that was on time horizon of a few years, which ended up being a decade plus), every day I check in on Twitter I see more exciting stuff than I can possibly process. I was just writing someone yesterday about how in six months time, we'll have LLMs acting as the front-end to knowledge bases and rigorous computational systems, and then we'll need to focus on getting the human, AI, and formal model all on the same page.

As has already been noted in <#C5U3SEW6A|linking-together> today, my estimate was off by roughly six months. Consider, "I've developed a lot of plugin systems, and the OpenAI ChatGPT plugin interface might be the damn craziest and most impressive approach I've ever seen in computing in my entire life. For those who aren't aware: you write an OpenAPI manifest for your API, use human language descriptions for everything, and that's it. You let the model figure out how to auth, chain calls, process data in between, format it for viewing, etc. There's absolutely zero glue code" https://twitter.com/mitchellh/status/1638967450510458882.

If you can tolerate his prose, Stephen Wolfram has a long post https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its-wolfram-superpowers/. The "Wolfram Language as the Language for Human-AI Collaboration" section is most relevant to Future of Coding. What do these developments mean for the Future of Coding? And how are you all holding up? Me? I can hardly process what's happening, let alone what to do about it.

[March 27th, 2023 12:30 AM] jarno.montonen: On the heels of the "LLMs and the future of programming" discussion (https://futureofcoding.slack.com/archives/C5T9GPWFL/p1679642239661619, https://futureofcoding.slack.com/archives/C5T9GPWFL/p1679892669316079), I'd like to start a more concentrated discussion around their effect on Future of Coding projects. There was already some sentiment that LLMs are going to kill FoC projects. Some yes, but certainly not all. So what kind of FoC projects LLMs will not kill?

[March 26th, 2023 9:51 PM] nmsmith65: Here's my perspective on LLMs and the future of programming.

I don't believe that the introduction of LLMs that can write code is going to obviate programming. And I don't believe that it is now pointless to develop new programming languages. Instead, I think LLMs are going to make programming and FoC research better, by automating one of the least interesting parts of programming: fiddling with the minutiae of syntax, language constructs, and libraries.

I think programmers will still have plenty of work to do. The profession is not doomed. But to justify this, we have to take a step back and consider all of the activities involved in programming.

Firstly, what is a "program"? A program is nothing more than: • A formal specification of the behaviour of an interactive system • ...that computer hardware can execute (after translating it into machine code). To emphasise this, I will use the term "formal spec" in place of "program" for the remainder of this discussion.

GPT-4 can understand formal specs, and also everyday English. Thus, if we can describe the functionality of a system in everyday English, GPT-4 can (attempt to) translate it into a formal spec. But writing the formal spec is just one activity of programming.

Altogether, programming (or perhaps "software development") involves several activities: 1. Determining what functionality the system being developed "should" have. This is done either by talking with relevant stakeholders (e.g. the future users), or by directly observing deficiencies with their current practices. 2. Expressing that functionality as a formal specification, i.e. "coding". 3. Verifying that the specification correctly implements all of the functionality of step 1. This includes practices such as reading and reviewing the specification, as well as testing the software. 4. Validating that the implemented functionality addresses the stakeholder's problems. 5. Repeating the first 4 steps until the stakeholders are satisfied with what has been developed. Here's my hypothesis: In the next 10 years, LLMs might radically reduce the amount of work required for step 2, but only step 2.

Steps 1 and 4 are very human-centered, and thus can't be automated away — at least until we are at the point where we have an omnipresent AGI that observes all human practices and automatically develops solutions to improve them.

Similarly, step 3 will not be automated any time soon, because: • The plain English descriptions that we give to LLMs will often be ambiguous, underspecified, and maybe even inconsistent. Thus the LLMs will have to make educated guesses at what we mean. (Even if they are able to ask clarifying questions, there will always be some choices that are automatically made for us.) • LLMs will occasionally get confused or misinterpret what we say, even if we are clear and careful. We will not have infallible AIs any time soon. So let's assume that LLMs can automate most of step 2. What does this mean for those of us developing tools and technologies to improve programming? Is our work obsolete now? Will the AI researchers and AI startups be taking the reigns?

I don't think so! There is still a huge opportunity to develop tools that address step 3, at the very least. (Steps 1 and 4 are harder to address with technology.)

In particular, step 3 involves the task of reading source code. When an LLM spits out 1000 lines of JavaScript, how do you know that the code implements the functionality that you wanted? You have to verify that it does, and for large programs, that will be an enormous amount of work!

As we all know, no amount of testing can prove that a program is correct. Thus, we cannot verify AI-generated programs just by using them. Maybe the program has a subtle bug, such as a buffer overflow, that might only be triggered 5 years after the program is deployed. Or less insidiously: maybe the program just doesn't handle certain edge-cases in the way you would like it to. Either way, a human should probably read through the entire program with a keen eye, to check that all of the logic makes sense.

There's clearly an opportunity for FoC researchers here: we can make languages and tools that make reading and verifying the behaviour of programs easier! Some examples: • We can design programming languages that are vastly easier to read than traditional languages. How might we do that? Well, "higher-level" languages are likely easier to read, since they are likely to be more concise and focus on the end-user functionality. So work on higher-level programming models will continue to be valuable. To complement this, we can (and IMO, we should) invent new syntaxes that are closer to plain English, such that the specifications that LLMs produce are accessible to a wider audience. • We can design programming languages where it is harder to write erroneous programs. For example, we can design programming languages that cannot crash or hang (i.e. Turing-incomplete languages), but which are still general-purpose. This reduces the kinds of errors that a human needs to consider as they verify a program. • We can design better tools for reading and interrogating source code. (For example, better IDE support for navigating and understanding the structure of large codebases.) • We can design better tools for exploring the space of behaviours of a running program. (Perhaps similar to the tools discussed in Bret Victor's <http://worrydream.com/#!2/LadderOfAbstraction|"Ladder of Abstraction"> essay.) Overall, I think the future is bright! I'm going to continue my own PL research project (a very high-level language) with as much vigor as ever.

📝 Pause Giant AI Experiments: An Open Letter - Future of Life Institute

📝 GPT’s Very Inhuman Mind | NOEMA

🎥 CLAIRify: Instruction Guided Task Programming with Verifier-Assisted Iterative Prompting