You are viewing archived messages.
Go here to search the history.

Paul Tarvydas 2023-03-28 22:22:30

Paul Tarvydas has joined the channel

Jonathan Edwards 2023-03-28 22:24:34

Jonathan Edwards has joined the channel

Jude Rayan 2023-03-28 22:25:13

Jude Rayan has joined the channel

Nick Smith 2023-03-28 22:32:06

Nick Smith has joined the channel

Duncan Cragg 2023-03-28 22:33:37

Duncan Cragg has joined the channel

Harrison Shoebridge 2023-03-28 22:38:39

Harrison Shoebridge has joined the channel

Robin Allison 2023-03-28 22:39:29

Robin Allison has joined the channel

Duncan Cragg 2023-03-28 22:40:07

[March 26th, 2023 5:24 AM] fp: So, I'm an EUP person at heart, and this ChatGPT thing has obviously got me thinking all over again about what programming would look like to a non-technical person. At heart, I feel it should be like they're "casting spells" over reality (or virtual reality). This tips into the area of cognitive modelling: how close the physical manifestation needs to be to be able to be abstracted up to a satisfying cognitive model that matches the human's intention. In other words, you cast a spell "make that banana green!" and it comes back a lurid dayglo green, that would be a cognitive dissonance because really, you'd expect to simply get a very unripe-looking banana. What are the elements of this formalised spell-casting, this "programming system"? You have objects (banana, this one, not all ones), attributes (green, the correct one!), a sense of time or evolution (went from yellow to green). You start to get into Roget's Thesaurus land: what are the key concepts for describing the world, our human world?

Anyway, just a splat of the stuff buzzing around my head right now. Thoughts?

[March 24th, 2023 12:17 AM] wtaysom: Friends, I don't know what to make of developments in AI these days. Having worked on dialog systems in the aughts and having loosely followed developments since (I recall preparing a talk around 2010 which left me pretty enthusiastic about ML applications in contrast to the App-and-Facebookification of "tech" — that was on time horizon of a few years, which ended up being a decade plus), every day I check in on Twitter I see more exciting stuff than I can possibly process. I was just writing someone yesterday about how in six months time, we'll have LLMs acting as the front-end to knowledge bases and rigorous computational systems, and then we'll need to focus on getting the human, AI, and formal model all on the same page.

As has already been noted in <#C5U3SEW6A|linking-together> today, my estimate was off by roughly six months. Consider, "I've developed a lot of plugin systems, and the OpenAI ChatGPT plugin interface might be the damn craziest and most impressive approach I've ever seen in computing in my entire life. For those who aren't aware: you write an OpenAPI manifest for your API, use human language descriptions for everything, and that's it. You let the model figure out how to auth, chain calls, process data in between, format it for viewing, etc. There's absolutely zero glue code"

If you can tolerate his prose, Stephen Wolfram has a long post The "Wolfram Language as the Language for Human-AI Collaboration" section is most relevant to Future of Coding. What do these developments mean for the Future of Coding? And how are you all holding up? Me? I can hardly process what's happening, let alone what to do about it.

[March 27th, 2023 12:30 AM] jarno.montonen: On the heels of the "LLMs and the future of programming" discussion (,, I'd like to start a more concentrated discussion around their effect on Future of Coding projects. There was already some sentiment that LLMs are going to kill FoC projects. Some yes, but certainly not all. So what kind of FoC projects LLMs will not kill?

[March 26th, 2023 9:51 PM] nmsmith65: Here's my perspective on LLMs and the future of programming.

I don't believe that the introduction of LLMs that can write code is going to obviate programming. And I don't believe that it is now pointless to develop new programming languages. Instead, I think LLMs are going to make programming and FoC research better, by automating one of the least interesting parts of programming: fiddling with the minutiae of syntax, language constructs, and libraries.

I think programmers will still have plenty of work to do. The profession is not doomed. But to justify this, we have to take a step back and consider all of the activities involved in programming.

Firstly, what is a "program"? A program is nothing more than: • A formal specification of the behaviour of an interactive system • ...that computer hardware can execute (after translating it into machine code). To emphasise this, I will use the term "formal spec" in place of "program" for the remainder of this discussion.

GPT-4 can understand formal specs, and also everyday English. Thus, if we can describe the functionality of a system in everyday English, GPT-4 can (attempt to) translate it into a formal spec. But writing the formal spec is just one activity of programming.

Altogether, programming (or perhaps "software development") involves several activities: 1. Determining what functionality the system being developed "should" have. This is done either by talking with relevant stakeholders (e.g. the future users), or by directly observing deficiencies with their current practices. 2. Expressing that functionality as a formal specification, i.e. "coding". 3. Verifying that the specification correctly implements all of the functionality of step 1. This includes practices such as reading and reviewing the specification, as well as testing the software. 4. Validating that the implemented functionality addresses the stakeholder's problems. 5. Repeating the first 4 steps until the stakeholders are satisfied with what has been developed. Here's my hypothesis: In the next 10 years, LLMs might radically reduce the amount of work required for step 2, but only step 2.

Steps 1 and 4 are very human-centered, and thus can't be automated away — at least until we are at the point where we have an omnipresent AGI that observes all human practices and automatically develops solutions to improve them.

Similarly, step 3 will not be automated any time soon, because: • The plain English descriptions that we give to LLMs will often be ambiguous, underspecified, and maybe even inconsistent. Thus the LLMs will have to make educated guesses at what we mean. (Even if they are able to ask clarifying questions, there will always be some choices that are automatically made for us.) • LLMs will occasionally get confused or misinterpret what we say, even if we are clear and careful. We will not have infallible AIs any time soon. So let's assume that LLMs can automate most of step 2. What does this mean for those of us developing tools and technologies to improve programming? Is our work obsolete now? Will the AI researchers and AI startups be taking the reigns?

I don't think so! There is still a huge opportunity to develop tools that address step 3, at the very least. (Steps 1 and 4 are harder to address with technology.)

In particular, step 3 involves the task of reading source code. When an LLM spits out 1000 lines of JavaScript, how do you know that the code implements the functionality that you wanted? You have to verify that it does, and for large programs, that will be an enormous amount of work!

As we all know, no amount of testing can prove that a program is correct. Thus, we cannot verify AI-generated programs just by using them. Maybe the program has a subtle bug, such as a buffer overflow, that might only be triggered 5 years after the program is deployed. Or less insidiously: maybe the program just doesn't handle certain edge-cases in the way you would like it to. Either way, a human should probably read through the entire program with a keen eye, to check that all of the logic makes sense.

There's clearly an opportunity for FoC researchers here: we can make languages and tools that make reading and verifying the behaviour of programs easier! Some examples: • We can design programming languages that are vastly easier to read than traditional languages. How might we do that? Well, "higher-level" languages are likely easier to read, since they are likely to be more concise and focus on the end-user functionality. So work on higher-level programming models will continue to be valuable. To complement this, we can (and IMO, we should) invent new syntaxes that are closer to plain English, such that the specifications that LLMs produce are accessible to a wider audience. • We can design programming languages where it is harder to write erroneous programs. For example, we can design programming languages that cannot crash or hang (i.e. Turing-incomplete languages), but which are still general-purpose. This reduces the kinds of errors that a human needs to consider as they verify a program. • We can design better tools for reading and interrogating source code. (For example, better IDE support for navigating and understanding the structure of large codebases.) • We can design better tools for exploring the space of behaviours of a running program. (Perhaps similar to the tools discussed in Bret Victor's <!2/LadderOfAbstraction|"Ladder of Abstraction"> essay.) Overall, I think the future is bright! I'm going to continue my own PL research project (a very high-level language) with as much vigor as ever.

Duncan Cragg 2023-03-28 22:41:57

Oh! Did I put mine first? Completely unintentional obvs.

John Krasnay 2023-03-28 23:36:23

John Krasnay has joined the channel

Chris Granger 2023-03-28 23:43:29

Chris Granger has joined the channel

Amelia Wattenberger 2023-03-28 23:50:20

Amelia Wattenberger has joined the channel

Katie Bell 2023-03-28 23:54:14

Katie Bell has joined the channel

Lu Wilson 2023-03-28 23:56:14

Lu Wilson has joined the channel

Denny Vrandečić 2023-03-28 23:59:52

Denny Vrandečić has joined the channel

Trevor Stenson 2023-03-29 00:29:28

Trevor Stenson has joined the channel

Ruben Duque 2023-03-29 00:37:27

Ruben Duque has joined the channel

Jarno Montonen 2023-03-29 08:10:39

ChatGPT (4) and DALL-E seem to be very slow atm, almost unusable. What if the demand for these services grows so fast that the semiconductor industry can't keep up? Could it be that AI won't take over the world within the next couple years as either there won't be enough computational power to run all the queries, or the prices will be hiked by a lot to balance the demand for the available computation power?

Paul Tarvydas 2023-03-29 08:35:47

just goofing around...

pt: say a pithy comment about commenting about artificial intelligence

ChatGPT: Commenting on artificial intelligence is like teaching a fish to swim - we’re all still figuring it out.

pt: When will ChatGPT be trained only on code generated by ChatGPT?

ChatGPT: “...

pt: What is in-breeding?

ChatGPT: ... it can also increase the expression of deleterious or harmful recessive alleles, leading to reduced fitness or genetic disorders. ...

pt: Who said “you’re fired”?

ChatGPT: “...

pt: Who said “just goofing around”?

ChatGPT: <crickets - generating for many minutes - answer did not include ‘22 Minutes’>

Nilesh Trivedi 2023-03-29 13:45:29

Found this tool to use LLMs on command-line with pre-define roles/prompts etc:

Here are some sample roles/prompts:

Jonathan Edwards 2023-03-29 14:15:17
Duncan Cragg 2023-03-29 15:48:04

What do you think about these, Jonathan?

Jonathan Edwards 2023-03-29 18:06:09

I tend to agree with Tyler

Konrad Hinsen 2023-03-30 16:35:03

It's hard to believe that none of the celebrities who signed this petition know the basics of game theory - which is all it takes to see that this appeal cannot work.

Jonathan Edwards 2023-03-30 23:21:23

Jimmy Miller what do you make of section 4 of

Srini K 2023-03-31 01:04:24

emphasis on “Artificial” from “Artificial Intelligence”

Duncan Cragg 2023-03-31 11:07:46

Can't believe the internet is still arguing over whether ChatGPT is just "glorified autocomplete" 🙄

Jimmy Miller 2023-03-31 17:16:10

It's a well written article. But there isn't much of an argument there. Can you define knowledge, understanding, intention, etc behavioristically? Well the article seems to assert you can, but why think that is the case?

The evidence we seem to be given in the article is two fold 1) Look at ChatGPT doing all these things, how can you deny it understands? 2) We can take the intentional stance towards the system and it works remarkably well.

I'm going to assume the article doesn't believe its presenting an argument, because if so, its a rather lackluster one. Philosophers have already talked about systems like ChatGPT well before they existed. Searle's Chinese room argument asks the exact question being raised, can a computer program be behavioristically identical to a human while not understanding?

Various people fall on various sides of this argument. But I don't really think ChatGPT has really any bearing on it. Of course that doesn't stop philosophers like David Chalmers from embarrassing themselves by making silly statements.

Joshua Horowitz 2023-04-01 02:10:15

I found the article (or section 4 at least) quite useful. Not because it made an argument about what understanding etc. are in a philosophical sense (I sort of care about that stuff, but not really?), but because it helped me make sense of deflationary sleights of hand in the discourse.

When Bender et al. call GPT a “stochastic parrot” or Chiang calls it a “blurry JPEG of the web”, they are not making a Chinese-room argument about what understanding really is. They are making an argument about what GPT is capable of , based on its make-up, an argument which seems to be demonstrably flawed. The “deepity” concept helped me tidy up this dynamic. As far as I’m concerned, the rest of the article may be making a misstep by making it sound like this is about deep questions of philosophy.

Jimmy Miller 2023-04-01 02:31:22

I need to spend more time reading the Stochastic Parrot paper, but from my reading they are not at questioning the output of GPT style models. They seem to think they are very capable of outputting fluent speech but lack things necessary for understanding.

Text generated by an LM is not grounded in communicative intent, any model of the world, or any model of the reader's state of mind. It can't have been, because the training data never included sharing thoughts with a listener, nor does the machine have the ability to do that. This can seem counter-intuitive given the increasingly fluent qualities of automatically generated text, but we have to account for the fact that our perception of natural language text, regardless of how it was generated, is mediated by our own linguistic competence and our predisposition to interpret communicative acts as conveying coherent meaning and intent, whether or not they do [89, 140]. The problem is, if one side of the communication does not have meaning, then the comprehension of the implicit meaning is an illusion arising from our singular human understanding of language (independent of the model).

I'm not saying I necessarily agree with everything here. Just that this does seem to be the argument they are making. The deepity point was a bit off imo. From the article:

The other meaning is the one that suggests that ChatGPT has no understanding of communicative intent, so when you ask it a question, it can only respond correctly in limited cases where it has seen the question, or else give awkward ill-fitting answers. But in this sense, ChatGPT is obviously not a stochastic parrot. You can ask it all sorts of subtle things, questions it has never seen before and which cannot be answered without understanding.

Then there are some transcriptions demonstrating understanding of "communicative intent". But that's to misunderstand the point the paper was making, not to refute it. Of course GPT can talk about communicative intent. The point being made in the paper is that the ability to talk about communicative intent doesn't entail that it has communicative intent or pays attention to our communicative intent.

William Taysom 2023-04-03 08:51:19


[Text generated by an LM] can't have been [grounded in communicative intent], because the training data never included sharing thoughts with a listener

Does this ring at all true to you guys? I think of the "the training data" as being things people have written online, which almost always involves "sharing thoughts with a listener."

I take the fact that ChatGPT gives sensible answers in a way that previous GPTs did not as evidence that the model can at least preserve intent and perhaps recombine so as to generate new intent.

Jonathan Edwards 2023-03-30 23:22:04

Invokes Ryle and Dennett

Gustavo Goretkin 2023-03-31 01:27:25

Example of using LLM and classical verifier (a compiler for some DSL) to boost an LLM's code generation: