In a slow couple of weeks, I've been mostly working on fixes and stability of the text editor. In the video I cover some ongoing work to improve UI/Widgets. On the Synthesizer side, I've been working on performance issues and continuing my implementation of the Ixi music language; I'll do an update on that when it is in a better state!
There's a longer video about Zep here; it gives a more general overview. I made it to give visitors to the github page a better idea of how it works: https://youtu.be/T_Kn9VzD3RE
Both videos are really cool.
You mentioned unit tests. How do you write tests for an app with so much interaction?
Thanks 🙂 Take a look here for unit test examples (particularly the bottom part of the file): https://github.com/cmaughan/zep/blob/master/src/tests/mode_vim.test.cpp
Essentially, each unit test here is setting up the text buffer, then running a keyboard sequence against it, and testing the output. Since I test every editor command, it is really easy to refactor things because I have a high feeling of confidence that I didn't break stuff (and usually minor changes break at least one or two tests!)
That's for Vim. I have tests for 'Standard' mode (like notepad, etc.); but TBH that needs fleshing out a bit more since I am more focused on Vim mode.
The harder things to test are window management, etc. I have a Regression command which I used to test splits and tabs, for example. Checking other higher level things like file save/load, etc. are things I need to add tests for. But I'm a great believer in using testing when it makes sense, but not going overboard. This particular project is a great example of how unit tests are essential, but I don't always use them.
Most interesting. How do the tests verify things after performing operations on the live editor? Are they looking at specific locations on the screen or something like that?
No, I think that's where testing would have to resort to visual diffs of screenshots. That kind of testing is much harder, and becomes tedious as you 'remaster' your screenshots when the visual look changes.
My tests here are effectively validating that the character buffer is changed correctly. I'm not confirming that the displayed pixels are the same. But if I needed that, I'd probably have to automate screen grabbing and compare, or use an external tool.
(I'd probably build such functionality into the ZRegress command I use above, and have a mode where it captured screenshots and a mode where it compared them). But it's a question of priorities.... I'd have given up ages ago if I didn't have the command tests; because they are what keep the editor functioning correctly after my changes. And I religiously add a new test when I find an edit problem; I like that it becomes more robust over time.
I recently added support for UTF8 (still not fully tested/finished). In order to do it I had to change many fundamental things, including making an iterator that could walk the editor in utf8 'code points' (i.e. 1-4 bytes per char). I just hacked in all the changes fearlessly and fixed it until the tests passed 😉 I doubt I would have attempted it without that security blanket.
I see. Yeah, I agree you don't need to compare pixels here. I have a different issue: if you're driving the screen when you only care about the in-memory buffer, that feels "smelly" to me. Your tests are running code they don't need, which I worry will make them more brittle over time. I'll share a demo soon that shows an alternative, so that this is more than just another random internet comment.
Oh, I'm not driving the screen while running the command tests; at least not the way you imply with the picture displaying each test. The tests are just run from the command line. The video above is the higher level regression test for windows. The command tests are just running in the background (in fact in Visual C they are running automatically after every compile, in the background).
Oh. Can you elaborate on what exactly is being checked in the higher-level regression test pictured?
I do have the concept of a 'Dummy Display' for the command tests. This is because Zep wraps its lines 'terminal' style. And some vim commands behave differently on wrapped lines. So there is a dummy display which represents the visual area that the buffer is being displayed on.
Yeah, so the ZRegress video is basically running a sequence of commands to create window and tab splits. Since managing splits I found tricky. It does a lot of random splits and makes sure that everything is internally self consistent. I showed it really to demonstrate how I test higher level things (or would approach them). Really, the video is showing 'Integration Testing' vs 'Unit Testing'. But in all fairness I have mostly unit tests currently.
I'm still curious what exactly the integration tests check on window and tab splits. If they're driving the screen, are they checking screen pixels? Or just that the program doesn't crash?
Yes, mostly that it doesn't crash. At one point I had a bug where I'd get an empty split, or a hanging tab with nothing in it; which in the design is not allowed and will inevitably lead to a crash eventually, or at the least some confusing behavior. But it was hard to reproduce. So I wrote this test. I intend to make it do more things; perhaps even testing screen output at some point, but only if I really need it.
Could you point me at the dummy display in your codebase? I'm curious to compare and contrast with how I do it 🙂
Oh, one more thought - the window.cpp code is the code I'm least happy with; I'm actually refactoring it slowly. It does all the nasty work of displaying the wrapped buffer, and handling display of all the markers, syntax, etc.
Part of the reason for that is, surprise, that it is hard to test. Part of it is that a lot of complexity exists in that domain.
Managing the buffer is somewhat easier. The bottom level structure is a Gap Buffer which has its own unit tests. Then a command layer is the only thing that can modify the buffer, etc. But window/display stuff is hard. And easy to break 🙂
That's kinda the focus of my stuff: to make the hard stuff easy at as low a level as possible.
To give context, Zep creates 2 concrete ZepDisplay classes; one for ImGui (OpenGL) and one for Qt. These 5 functions are what's required to draw the editor on a new display
The Null is just returning a dummy text size which is '1' unit per char requested.
In truth there are 2 abstractions; the Display abstraction is for rendering, but there is also an Editor abstraction; that's to handle system-specific keyboard input.