You are viewing archived messages.
Go here to search the history.

Sophie Smithburg 2021-11-16 16:33:13

🐦 sophs wants to make equitable tech infra: tl;dr: the unnecessary concentration of power, intellectual property (and the resulting workflows it enables) kind of necessarily creates what we currently call a tragedy of the commons...

Tweet Thumbnail

Kartik Agaram 2021-11-16 17:21:04

I was ranting about the tragedy of the commons for several years before I started working on my current projects in a hard-to-articulate attempt to address them. So the problem really resonates. However, I think the problem is harder than you make it seem, and have trouble following the leaps of logic/intuition between your tweets. A few responses:

  • People can cause the tragedy of the commons both by action and inaction. Rent-seeking covers the former, but the term was originally coined to cover the latter. A shared bit of grazing land that nobody owns (so no rent to seek) and therefore nobody is incentivized to fix. So I'm curious to hear more elaboration on why you think less ownership will lead to better maintenance of commons.
  • The tragedy of the commons is intimately tied to game theory and the prisoner's dilemma. One way to rephrase "manage commons sustainably" is "keep people from defecting." I hope that phrasing makes it obvious that this is a really hard problem.
  • Lots of solutions that seem to "solve" the tragedy of the commons rely on social enforcement mechanisms that only work in small groups. So, "like, don't" at scale is an unsolved problem of social engineering.
  • One kind of commons that is consistently under-appreciated is abstract idea spaces, like IP as you mentioned. Or think about the way every site started pushing for websites to black out for a day for some cause or other until one day it stopped working. Abstract commons are even harder to protect from tragedies because it's much harder to make a case that they're a commons, and so much harder to create the widespread will to enforce penalties for defection.
  • Putting it all together, the rules for deciding what constitutes rent seeking are an abstract commons. It's hard to imagine getting a lot of social credit/kudos/whuffie for better policing, and so less effort goes into it. In this way the problem becomes circular.

My best response to these problems right now is to help people grow more sophisticated in their consumption choices. There's already a blooming awareness, for example, that the software you choose to support has societal implications. We're becoming less individualistic and more collectively-minded there and in a lot of other similar areas. I just want to accelerate that shift as much as possible.

Sophie Smithburg 2021-11-17 11:47:45

asking questions about my original thread may have been a more productive starting point given you start off this five point thing with "However, I think the problem is harder than you make it seem, and have trouble following the leaps of logic/intuition between your tweets."

let's start from the top: the idea that the tragedy of the commons was meant to address inaction as opposed to rent seeking, and that this in any way vindicates it. mostly: empirically we don't see tragedies of the commons. a nobel prize has been awarded for this research. elinor ostrom.

she found primarily that groups tend to succeed at this in varying conditions modulo specifically the two aspects i mentioned, rent seeking and exogenous shocks. in other words, the tragedy of the commons is a nice idea, but it's pretty much exactly that: an idea.

re it being analogous to the prisoners dilemma, yes, i'm aware. ostrom's work is based directly off of that insight and trying to apply game theoretic methods to this more generally social sciences problem.

re rent seeking redefining the commons: just no. rent seeking is just rent seeking, or a particular sort of extractive behavior. we needn't complicate that. extractive has a well enough defined sense that we can just stick to that.

the commons is the common sense set of things we think it is. whether or not the things in it are physical, they are sticky in the sense of hyperobjects, and this gives them all the necessary attributes of physicality to sensibly reset our thinking back to normal.

Sophie Smithburg 2021-11-17 12:01:46

also: i'm not actually anti property re how we manage commons. i'm just suspicious of current attitudes/mindsets re property. property rights are a standard mechanism for managing commons, among others.

Kartik Agaram 2021-11-17 14:26:46

Sorry, I tried hard not to sound aggressive, but I guess I'm not there yet. I'll go read Elinor Ostrom.

Timothy Johnson 2021-11-17 14:38:07

I think I'm missing some context for this thread. Can you give an example of a "tragedy of the commons" situation that you have in mind, and how you would prevent it?

Sophie Smithburg 2021-11-17 16:30:53

Kartik Agaram i haven't read her work in great depth myself either, i was just summarizing what i know so far and wanted to let myself be chill re citations and stuff to get the idea out at all.

Sophie Smithburg 2021-11-17 16:35:59

@Timothy Johnson tech usability/infrastructure/tooling is a tragedy of the commons; we deeply underinvest in blatantly obvious next steps bcz they don't seek enough rents.

how we address it: directly fund the creation of common value tooling rather than leaving it totally to corporations minimal benevolence. protecting the inclusive nature of institutions and tools etc in the commons; fighting specifically to ensure the commons don't end up enclosed.

Arvind Thyagarajan 2021-11-19 04:08:59

george monbiot has a nice phrasing for one part of this idea: "private sufficiency, public luxury" -- or in its more challenging form, "public luxury for all, or private luxury for some"

Sophie Smithburg 2021-11-16 16:33:27

(it's a thread, that's just the last tweet)

Breck Yunits 2021-11-18 22:24:13

Does anyone know examples (or name of the pattern) of markup formats where instead of writing format directives inline <b>like this</b> you do it out-of-the-line like:

text writing format directives inline like this

 bold like this
Andrew F 2021-11-18 23:16:12

I'm not sure I understand your example. Is the line bold like this a directive to bold the substring "like this" in the previous line?

Breck Yunits 2021-11-18 23:17:53

Yes. Of the form [bold|italic|underline|...] [text string to match]

Breck Yunits 2021-11-18 23:19:52

The text is in blue, the format directive in purple, and the search string in yellow:

Breck Yunits 2021-11-18 23:21:06

So this example would compile to html like <p>writing format directives inline <b>like this</b></p>

Shalabh 2021-11-19 05:59:55

codexeditor on Twitter calls them standoff properties.

Mariano Guerra 2021-11-19 09:11:07

roff/troff/nroff?

.nf

.ll 4.0i

.in 2.0i

101 Main Street

Morristown, NJ  07960

15 March, 1997

.sp 1i

.in 0

Dear Sir,



.fi

.ti 0.25i

I just wanted to drop you a note to thank you for spending the

time to give me a tour of your facilities. I found the experience

both educational and enjoyable. I hope that we can work together

to produce a product we can sell.
Kartik Agaram 2021-11-21 19:52:52

Immortal programs vs crash-only programs

Immortal programs: http://steve-yegge.blogspot.com/2007/01/pinocchio-problem.html

Crash-only programs: https://en.wikipedia.org/wiki/Crash-only_software

In brief, immortal programs try to never, ever reboot. Crash-only programs are designed to always be able to recover gracefully from a reboot. There's a fundamental tension here, and I'm starting to realize I'm very definitely on one side of it. I like a neat desk, and am compulsively closing things (terminals, browser tabs, browser sessions) when I'm done with them. I prefer text editors to IDEs, vim to emacs, unix as my IDE rather than slime. I'd always thought of these as subjective opinions that were just down to my personality and past experience. But, upon reflection, I want to make a stronger case that "my side" is superior.

  • Focusing on recovering from reboots makes you better at simulating immortality. Restarts can in principle become instantaneous. Focusing on never rebooting makes you worse at recovering from crashes.
  • It's easy for immortal programs to end up in situations that are difficult to reproduce. I spent some time recently programming with Tudor Girba's Glamorous Toolkit. Modern Smalltalk uncomfortably straddles the image and git repo worlds. The way you work is to make changes to your running image until you have something you like, then go back and package up a slice of your image into a git repository to publish. If you make mistakes, others can have trouble reproducing the behavior you created in your image. Testing if you did it right necessarily requires rebooting the image.

Putting these reasons together, immortal systems are more forbidding to newcomers. Crashing becomes a traumatic event, one newcomers are not used to, something beginner tutorials don't cover. When things don't work, it's more challenging to ask for help. Creating and sharing reproducible test cases requires crash-recovery skills.

Rereading the Pinocchio post now, I notice that there's actually no concrete benefits stated for long-lived programs. All there is are (compelling) analogies. A counter-analogy: an immortal program is like a spaceship. Once launched you're in a little bubble, stuck with whoever you happened to start out with. A crash-only program is like a little stone rolling down a hillside, gathering other stones until it turns into an avalanche.

As I said above, I'm biased because of my experiences. I'm curious to hear from others with more experience of immortal programs. Am I understating the benefits, overstating the drawbacks?

Kartik Agaram 2021-11-21 20:36:30

That's a good point. I think Erlang might be the ultimate no-compromises crash-only system, the way Smalltalk is the ultimate no-compromises immortal system.

Jamie Brandon 2021-11-21 20:41:56

I think this is the key to the success of the database / application server separation. You put all of your long-lived state into some immortal process with a very controlled data model and put all the scary weird stuff into a crash-only process.

Jamie Brandon 2021-11-21 20:43:37

Maybe the real benefit from the immortal systems that yegge described is not that you can avoid restarting them, but that they are forced to have lots of tools for inspection / modification. If you have something like https://ourmachinery.com/post/the-story-behind-the-truth-designing-a-data-model/ instead you can get the same benefits while also being able to do clean restarts.

Jamie Brandon 2021-11-21 20:46:39

Similarly, the problem I had with smalltalk is not that state lives in the image, but that the state is smeared all over the place and built out of pointers and mutable variables. Having all your state and code in eg sqlite or couchdb is essentially the same idea but is much easier to inspect and understand.

Kartik Agaram 2021-11-21 20:59:40

There's definitely value in reducing the blast radius of crashes, but that feels like an orthogonal axis. I love how the DB is separate when I restart my web app, but it doesn't help me when I need to restart my DB. Is there any stateful system today that has a decent story for restarting without downtime?

Now I wonder what a storage system would look like that was designed from the ground up to be crash-only..

Andrew F 2021-11-22 01:05:38

I definitely lean toward crash-only. The only truly immortal program is one that runs on a machine that never has power failures, angry users with hammers, etc.

I also agree that in principle, requiring reboots is a sign of flawed software. I froth at the mouth a little bit every time I have to "turn it off and on again". I think the only real tension between those two ideals is the one related to users being unfamiliar with the process of rebooting, and I'm pretty sure that can be surmounted. I can't think of any reason you wouldn't try for both. Maybe think of it as "immortal unless you pull the plug, which must be allowed".

I don't have the patience I once did for Stevey Blog Rants, so I skimmed his post, especially the middle. However, I think the parts that he thinks should be immortal (what he calls software as opposed to the rigidly defined software-as-hardware he says static types create), that soft stuff should be thought of as user-created content that is interpreted by relatively stateless infrastructure. From that perspective, it's clear that content should be immortal (and portable, inspectable, etc), while the infrastructure can be started up or killed whenever. Obviously there's some caching involved, maybe even including JITed user code, but I think this architecture could basically get the nice properties of both immortality and crash-only.

George Campbell 2021-11-22 02:16:40

There are tons of distributed databases that handle partial failure all the time. I work with Cassandra. One of the interesting things we’ve done is to use AWS’s virtual block storage (EBS) to swap versions by reattaching storage to the new instances.

Breck Yunits 2021-11-22 03:47:58

I'm 99.9% biased toward crash-reboot from my empirical dataset. I'd also say that in biology cell-division seems like a crash reboot.

Chris Knott 2021-11-22 07:01:22

This is why I like git - it can't really "crash" as such because it isn't running 99.9% of the time. Highly stateful system, but all the state is either on disk (or very short lived).

The equivalent of "crashing" in systems like this is getting a bad config, so it won't run at all. This is incredibly annoying to deal with. So I'm not sure there's actually a philosophical difference here, other than "be mindful of unanticipated states". Having a crashy system probably just makes you think about state more explicitly.

The main difference I guess is that a "crashed" (i.e. corrupted) git repo is an automatic "memory dump" to which your file browser and text editor act like an already attached debugger.

William Taysom 2021-11-22 07:28:15

There really are important differences between long-lived and short-lived parts of a system. To a first approximation it's alls just data and transformations thereof, but how manage the tradeoffs of robustness, performance, mutability, and recovery differ.

Paul Tarvydas 2021-11-21 20:51:38

I agree with your perspective, but I think that the Happy Path culture is more insidious than imagined. It deeply affects our tools and thought processes. Here are some of my thoughts... Failure Drive Design.