You are viewing archived messages.
Go here to search the history.

Tak Tran 2023-03-26 20:46:00

I’ve been playing around with hand tracking, inspired by Bret Victor’s Future of Interaction Design article from ages back, and it’s come together in this little toy I made, combining fluid simulation: Magic Hands 🙌 (blog post)

Very much a starting point, but quite keen to see what else can be done with hand tracking in the browser - also keen to hear more examples of hand tracking work 🙂

Joshua Horowitz 2023-03-28 00:07:40

That’s fun! I’m curious about how you were inspired by Bret Victor’s article. IIUC, that article is a rant against touchscreens as “Pictures Under Glass”, in favor of giving hands back tactility and physicality. By itself, your experiment is even less tactile & physical than a touchscreen, right? Are you planning on moving in a more tangible direction?

Paul Tarvydas 2023-03-28 02:49:35

edit; works with Chrome on my MacBook ; seems to want my open palm and not my closed fist or back of my hand - very interesting ; and Firefox ; different on Safari

Tak Tran 2023-03-28 07:26:08

Thanks for checking again Paul Tarvydas - I’m still debugging the issue on safari. The open palm/back of hand detection is deliberate. I wanted to create the effect of a delay of nothing happening when you raise your hands, but then showing the visuals when your hand opens - like 🙌. It means you can control when you want the visuals to show, even while your hands are on screen. This may change in future, but still figuring out what I want to do next with this.

Paul Tarvydas 2023-03-28 14:13:00

datapoint: on iPhone8, swipe works but camera doesn’t (Safari and Chrome)

Tak Tran 2023-03-28 21:46:56

Joshua Horowitz sorry, missed your original msg in my notifications, but you are right, that the demo doesn’t make it more tactile, but I’d argue that it is more physical, as you are required to move more of your body. I also like the augmented-ness of it, where you can see yourself and the glowing fluids overlayed, as if it’s more of an embodied experience vs it being just your finger on touch devices (or more disconnected, as a mouse experience). That’s where it feels more theatrical to me.

Tangible is def an area I’m very interested in, and I want to explore some gestures that might use some natural feedback mechanics eg, if you tap your finger on your thumb you get tactile feedback for free. I rmb seeing Google Soli doing it ages ago, but it doesn’t seem like they followed through with it - dunno why. Part of my point with Magic Hands is to stimulate more ideas for getting away from standard keyboard, mouse and touch interactions, as Bret was trying to encourage in his article. Definitely a lot more work to be done there, but it’s a good time to play with the tech 🙂

Tak Tran 2023-03-28 21:50:13

I also think part of the theatre metaphor I’m embracing is “fake it till you make it”. Just fake the experience a bit, by guiding people where you want them to go, if the tech isn’t up to it yet. Puppetry isn’t exactly super hi-tech. but you can create a really engaging experience from the magic of the puppet master.

Tak Tran 2023-03-26 20:46:00

I’ve been playing around with hand tracking, inspired by Bret Victor’s Future of Interaction Design article from ages back, and it’s come together in this little toy I made, combining fluid simulation: Magic Hands 🙌 (blog post)

Very much a starting point, but quite keen to see what else can be done with hand tracking in the browser - also keen to hear more examples of hand tracking work 🙂

Joshua Horowitz 2023-03-28 00:07:40

That’s fun! I’m curious about how you were inspired by Bret Victor’s article. IIUC, that article is a rant against touchscreens as “Pictures Under Glass”, in favor of giving hands back tactility and physicality. By itself, your experiment is even less tactile & physical than a touchscreen, right? Are you planning on moving in a more tangible direction?

Paul Tarvydas 2023-03-28 02:49:35

edit; works with Chrome on my MacBook ; seems to want my open palm and not my closed fist or back of my hand - very interesting ; and Firefox ; different on Safari

Tak Tran 2023-03-28 07:26:08

Thanks for checking again Paul Tarvydas - I’m still debugging the issue on safari. The open palm/back of hand detection is deliberate. I wanted to create the effect of a delay of nothing happening when you raise your hands, but then showing the visuals when your hand opens - like 🙌. It means you can control when you want the visuals to show, even while your hands are on screen. This may change in future, but still figuring out what I want to do next with this.

Paul Tarvydas 2023-03-28 14:13:00

datapoint: on iPhone8, swipe works but camera doesn’t (Safari and Chrome)

Tak Tran 2023-03-28 21:46:56

Joshua Horowitz sorry, missed your original msg in my notifications, but you are right, that the demo doesn’t make it more tactile, but I’d argue that it is more physical, as you are required to move more of your body. I also like the augmented-ness of it, where you can see yourself and the glowing fluids overlayed, as if it’s more of an embodied experience vs it being just your finger on touch devices (or more disconnected, as a mouse experience). That’s where it feels more theatrical to me.

Tangible is def an area I’m very interested in, and I want to explore some gestures that might use some natural feedback mechanics eg, if you tap your finger on your thumb you get tactile feedback for free. I rmb seeing Google Soli doing it ages ago, but it doesn’t seem like they followed through with it - dunno why. Part of my point with Magic Hands is to stimulate more ideas for getting away from standard keyboard, mouse and touch interactions, as Bret was trying to encourage in his article. Definitely a lot more work to be done there, but it’s a good time to play with the tech 🙂

Tak Tran 2023-03-28 21:50:13

I also think part of the theatre metaphor I’m embracing is “fake it till you make it”. Just fake the experience a bit, by guiding people where you want them to go, if the tech isn’t up to it yet. Puppetry isn’t exactly super hi-tech. but you can create a really engaging experience from the magic of the puppet master.

Paul Biggar 2023-03-28 15:06:52

Though this is an announcement of new direction for Darklang, I spend quite a bit of time looking at what AI means for programming languages blog.darklang.com/gpt

Jarno Montonen 2023-03-28 15:43:59

Paul Biggar I was wondering, are you going to have a parsed textual version of darklang that ChatGPT would generate or are you trying to get it to generate some serialized format of your AST directly?

Kartik Agaram 2023-03-28 16:29:59

Paul Biggar and others, we'd love to have you on this thread: 💬 #of-end-user-programming@2023-03-27T20:40:36.443Z

Paul Biggar 2023-03-28 18:18:41

@Jarno Montonen Right now we extended the F# parser, but we might be getting close to needing our own parser

Matt Webb 2023-03-29 19:45:40

Hullo! Would anybody be able to look at my new website about my AI Clock and simply tell me whether it works? I’ve had a report that it is being labeled as “unsafe” and I would like to resolve this before sharing the link publicly… Thank you!

poem.town/clock

Matt Webb 2023-03-29 19:46:27

This is what it is supposed to look like

📷 image.png

Mariano Guerra 2023-03-29 19:51:26

works here, firefox nightly on ubuntu

Mariano Guerra 2023-03-29 19:52:07

also on safari on macos

Mariano Guerra 2023-03-29 19:52:39

and chromium on ubuntu

Matt Webb 2023-03-29 19:55:07

Thank you, I really appreciate it!

Joakim Ahnfelt-Rønne 2023-03-29 19:56:03

Works on iOS Safari

Duncan Cragg 2023-03-29 20:43:12

All good on Android Chrome

Matt Webb 2023-03-29 20:43:59

This is such a relief! I shared it with a media person earlier and they said it triggered the “unsafe site” warning on their corp machine

Duncan Cragg 2023-03-29 20:44:31

"media person" 🙄

Denny Vrandečić 2023-03-29 21:06:01

works fine in Chrome on Mac

Christian Gill 2023-03-30 05:21:52

Works on iOS + Chrome

Matt Webb 2023-03-30 07:48:22

Duncan Cragg lol I might get a single line mention in a newspaper article, but I know better than to say anything until it actually happens

Matt Webb 2023-03-30 07:49:09

Thank you everyone! Based on this reassurance I published my first Substack update about the clock:

aiclock.substack.com/p/update-1-makin-plans

Duncan Cragg 2023-03-30 08:47:45

If you can make the rhymes even cheesier and more awkward you'd definitely be onto a winner. Well, with my family at least!

Ivan Reese 2023-04-01 17:39:15

Future of Coding • Episode 63

Ben Moseley & Peter Marks • Out of the Tar Pit

𒂶 futureofcoding.org/episodes/063

Out of the Tar Pit is in the grand pantheon of great papers, beloved the world over, with just so much influence . The resurgence of Functional Programming over the past decade owes its very existence to the Tar Pit’s snarling takedown of mutable state, championed by Hickey & The Cloj-Co. Many a budding computational philosophizer — both of yours truly counted among them — have been led onward to the late great Bro86 by this paper’s borrow of his essence and accident . But is the paper actually good? Like, really — is it that good? Does it hold up to the blinding light of hindsight that 2023 offers? Is this episode actually an April Fools joke, or is it a serious episode that Ivan just delayed by a few weeks because of life circumstances and his own incoherent sense of humour? I can’t tell.

Apologies in advance. Next time, we’re going back to our usual format to discuss Intercal.

Jason Morris 2023-04-01 22:14:27

If you do programming by generating a LLM to do a task, you cannot use informal reasoning to understand the behaviour of your system. You can't even use formal reasoning. The run-state of the code is so far removed from what you wrote, you can't look at the source code and say "oh, here's where I made it racist.". That is a programming that informal reasoning doesn't work for. It requires that you are building systems that build systems, of course, and that it is the system two turtles down you are concerned with understanding. But still, I think it fits?

Jason Morris 2023-04-01 22:34:24

☝That is my answer the challenge #1. Now challenge #2. In Blawx, it would be trivial to take a test, and generalize the inputs of that test by making them abducible, and run the query again. The answer to that second query would be a set of models, with constraints on unground variables, in which the query holds. Essentially, a description of all inputs that would have made your test pass. But of course, there is no reason to have the extra step of starting with a grounded test. You can go straight to the most general query, and say "give me all the inputs for which the following assertion holds." But technically, I think that's an example of #2. The system could answer "that test passes, because it is an example of this model, anything in that model would work. Also, here are all the other models that work." It's also the reason I'm really excited about its potential as a tool for generating tests that can be used to validate the legal reasoning of other systems.

Ivan Reese 2023-04-02 02:35:54

Great answers!

Jason Morris 2023-04-02 05:03:59

Great episode! I wonder if being bad at reasoning over specific examples drives one toward philosophy, which attempts to find broader truths. A weakness that creates a corresponding superpower?

Paul Tarvydas 2023-04-02 15:58:49
Ivan Reese 2023-04-02 18:16:33

On slide 5 you have "waterflow workflow" — is that a typo? Did you mean "waterfall"?

Paul Tarvydas 2023-04-02 23:45:14

Yes, that’s a typo. I meant “waterfall”. Thanks for catching that. I’ll see if I can remember how to regenerate that file….

Konrad Hinsen 2023-04-03 08:20:07

Interesting, thanks for sharing! A lot of overlap with my own thoughts/projects over the last few years.

My own perspective is a bit different, as I am coming from scientific research rather than software development. As you hint at in a few places, research has always been done with failure in mind. As was scientific computing in its early days, when machines and software were tailor-made for us.

Then the commodification of hardware and software happened, and scientists became increasingly pressured to adopt these commodity tools, and with them the attitudes of professional software engineers. In parallel, scientists were forced to adapt to project-based funding schemes that punish (visible or admitted) failure.

I have bookmarked your slides for another read with an in-depth study of the references. More later...

greg kavanagh 2023-04-02 17:44:29
greg kavanagh 2023-04-02 17:45:15

I’m working on a tool for text.

Paul Tarvydas 2023-04-03 02:20:14

Love it.

I agree that a progress indicator is necessary.

IMM, text toollng is 2 parts

Creating and editing text

Displaying text

This tool does (2) very well. Youtube, hip-hop, songwriting all indicate that the audience wants less text and compressed content.

My favourite for (1) is Keynote (MacOS, probably Powerpoint, too, but, I don’t know). Keynote is a REPL for content creation. You edit text, Keynote changes the font size on the fly and you get immediate feedback on whether your text has too many words in it. Keynote, though, then tangles (1) and (2) together resulting in bloatware (images of slides), instead of separating the two aspects.

The best analysis of textual prose that I’ve seen is Pat Pattison’s work. There are ways to write text that result in more interesting text. Pattison analyzes and narrows it down. Pattison, ostensibly, talks about songwriting. Songwriting is about writing compressed text, how to punch it up and how to make it memorable.

greg kavanagh 2023-04-02 17:45:54

This is some output but it also seems apt considering the current zeitgeist. (Warning contains some adult content)

Duncan Cragg 2023-04-02 17:55:09

I like the tap book thing, but there's one thing missing which I'm hoping you could add: most people would probably appreciate some kind of progress indicator, or at the very least page numbers "of" (e.g. page 10 of 200) I find it hard to commit to reading without knowing how long it's going to take. Otherwise, great start to a book, as far as I've read! 👍

Duncan Cragg 2023-04-02 17:55:49

Are you going to start a thread on the URL token patent thing, also?

Duncan Cragg 2023-04-02 17:55:57

I'd be interested in that

greg kavanagh 2023-04-02 17:56:23

Of course… I would feel the same. Thanks for the feedback.. it’s a short story btw.. 10mins

greg kavanagh 2023-04-02 18:00:37

I will start a thread about it. I’’m going to put together a tap story about it and post it. But mainly the idea is that you could exchange some value for a story or any digital artifact but because it’s so abstract I had to build a publish platform first to prove the concept.

greg kavanagh 2023-04-02 18:05:32

The object.network page would suit the format.

Duncan Cragg 2023-04-02 18:10:11

I'll consider that! 👍

greg kavanagh 2023-04-02 18:15:14

It’s a fascinating idea, I want one of the smartwatches. The books are all written in a DSL so they could easily be objects.

Duncan Cragg 2023-04-02 18:50:51

Yes I have text object type which is basically a single paragraph. I'll make you a personalised watch, my first customer!

😊👍

Kartik Agaram 2023-04-02 20:19:56

Great story! The medium managed to suck me in without the usual trappings I'm accustomed to (social proof, liner notes, reviews).