You are viewing archived messages.
Go here to search the history.

Srini Kadamati 2021-06-28 15:27:41

Does anyone know of any tools that let you ‘trace’ and understand every function, variable, + state / values in a Python web app / project? I’ve found tools for logging and full on observability in production / monitoring but I want to be able to “do a few things” in the UI and then understand the Python code that was run. I’m also aware of step debuggers and breakpoints but is there anything that just captures the entire state of the Python app and lets me understand it as it changes over the span of 30 seconds. I”m well aware that this will likely generate a TON of data thats painful to understand, but I’ll cross that bridge then!

My motivation here is to understand a daunting code base for an open source project.

Harry Brundage 2021-06-28 15:49:41

i think rr is one of the main tools that does this, it does it not for python specifically but pretty much any program at a low level by plugging into gdb

Harry Brundage 2021-06-28 15:50:09

there's an exciting wrapper around it called Pernosco that doesn't have python support yet but that I think looks super awesome https://pernos.co/

Harry Brundage 2021-06-28 15:50:16

the keyword they use is "omniscient debugger"

Srini Kadamati 2021-06-28 16:11:57

oh sweet I will play with this, thanks Harry! And also thanks for the hat tip w.r.t. the word to search for “omniscient debugger”. I kept running into the wrong articles before

Wilhelm Klopp 2021-06-28 20:11:08

Srini Kadamati this is almost exactly what I'm building with https://kolo.app

Caveats are that it's alpha-quality and Django + VSCode only right now

🔗 Kolo for Django

Wilhelm Klopp 2021-06-28 20:18:48

What's the daunting open source project out of curiosity?

Srini Kadamati 2021-06-28 21:18:59

neat!

Srini Kadamati 2021-06-28 21:19:39

Apache Superset, which I work on at work (but more on the community, advocacy, content, docs side than pure engineering)

🔗 Welcome

🔗 apache/superset

Srini Kadamati 2021-06-28 21:19:53

its a relatively complex web app, but heck I even want to start by mapping out / internalizing the backend first!

Srini Kadamati 2021-06-28 21:22:52

@Wilhelm Klopp just saw that you also work on Simple Poll, that’s neat! My former coworker (Mikael) works with you 🙂

William Taysom 2021-06-29 13:20:59

I feel you. Every now and then I do a janky one-off in Ruby using its TracePoint API, which never becomes more than a one-off mainly because you cannot easily trace object modifications.

For code understanding, step debuggers would be more useful if you could step backwards. Stepping forward often has the problem of setup_a_thing.do_a_thing where setup_a_thing is long, convoluted, uninteresting, and do_a_thing mostly only looks up something obscure that comes at the end of setup_a_thing.

So it would be best to jump into setup_a_thing then cross jump back to where @that_store gets setup.

A few years (years, yikes!) we prototyped a nifty way to deal with traces in VR. Basically, the trace was a super long ribbon of events, and then you had automatic layout that would fold bits to be next to each other. For example, every time an object is referenced. This actually actually allows you to see the object graph where edges represent communication patterns.

Sverrir Thorgeirsson 2021-06-29 14:34:44

Python Tutor is an interesting tool and could be a source of inspiration for those who are working on similar tools.

Example program

Wilhelm Klopp 2021-06-29 15:01:50

@William Taysom that VR project sounds awesome haha – Was it ever made public / do you have a link to it?

William Taysom 2021-06-30 13:41:38

Alas, no. Been wanting to get back to it.