My mistake of formalism

by Coranna Howard in Self

I've been working exclusively on Hord and Onsang this month. These projects are roughly designed in the same way as the rest of my 2009–2014 projects are (in which they occupy only 2013–2014). I only really grew into thoughtful programming in 2009, as I started using C++, but this isn't to say the projects of that period are well designed. Years of working in the same style doesn't show one much; there's nothing to contrast against, and fools like myself generally don't seek out alternative approaches unless they see the existing problems. The naivety of a budding programmer certainly offers no help.

Luckily, I've had a couple of big contrasting designs & methodologies since 2014. Most of this was digesting material from talks, blogs, & papers, and, thankfully, rational thinking. I wasn't especially attached to my approaches, but that alone wouldn't have been enough to force me to work & think differently. It took a willingness to explore, even if that meant leaving behind a lot of work I'd already done. I owe much to bitsquid, whose blog make up the majority of the material. All of this shed light on the issues I didn't know I had — issues that drive deeper than the code itself.

The executive initiative came with togo near the end of June. At the beginning, I was taking a dive into the deep end on theory alone, without having experienced the values and intricacies of a data-oriented design, with a semantic and syntactic style far different from my existing projects. I began to appreciate it more over time, especially as I kept going back to tinker on Hord & Onsang. The efficiency of compilation and editing alone had sold me (and, somewhat unsurprisingly, the systematic structure I valued), especially in comparison to my earlier projects. Compiling togo is faster than compiling my smaller projects by factors of 3 to 7. Even though the majority of my projects are relatively small (including togo), these factors are significant.

It probably goes without saying, but even a minor delay between editing and execution is a major hindrance to iteration and testing — the less friction, the better. An important factor of togo is that it does not depend on other projects with poor design, whereas most of the others do. In header libraries like duct++ and AM‽, any change requires all of its dependers to be recompiled! This compounding is what makes them worse, even if they were individually as fast to compile as togo.

So, going back to Hord and Onsang, where my mistake was made… Apart from simply being excruciatingly slow to compile, the over-reliance on what I'd call ‘structural formalism’ has doomed them (Hord directly, and Onsang as an instigating bystander). Being a bookend of sorts to my designs up to 2013, I had (unfortunately) decided Hord needed to be strongly specified and very strict. There's some value in this, in that I was determined to make sure there were no discrepancies and everything was sufficiently documented, but in the end none of that mattered, and it only served to harm the end purpose of Onsang.

This mode of thinking tended towards premature optimization and, more importantly, eternal black holes of design. This mode of thinking lead to the ghastly inventions of ceformat, murk, and duct++'s Gnarly Report (it's literally right there in the name!), all of which I immensely regret. This mode of thinking lead to “in theory this should work; can't wait to try it out!” (which I was saying for two self-destructive years) instead of “in practice, this works, because it's actually running right there in the terminal” (i.e., functionality, which is where all of that effort on formalism should have been directed).

The inception that lead to this was the idea that I could learn about tech that I'd not seriously engaged before. How in the world did that lead to formalism? I think it comes down to a few things (though, as always, there's a whole swamp of bile responsible): a desire for systemic structure that I had ever since I started using C++, some kind of semantic contract to back it up, and the Googlemachine. You see, the Googlemachine decided in August 2012 that my GitHub profile met some obscure criteria, and I had received a pre-pre-interview solicitation from one of their recruiters. I didn't make the cut, and the recruiter decided to mention they'd come back to it two years later. Here the motions begin.

They wanted me for the Site Reliability Engineering (SRE) team, which primarily prides low-level knowledge of *nix, networking, and That Monster, Big Data. Naturally, I had none of this, but I was eager to learn, so I decided to finally remake my log keeper (which is essentially a UI wrapped around a tree of spreadsheets) in January 2013 and learn along the way. Networking had to be part of that, so Hord was designed with messaging and distributed data storage. All of that is now gone. One by one, these little bits I tacked on to “learn something” fell to the wayside, as the project was completely ungrowing otherwise. Onsang didn't spawn until October 2013, and it was nowhere near being functionally representative of actual software you interact with until July 2014 (and even then, you couldn't do anything). What a disaster!

I'm sad to say I thought I could still get something out of it this month. I did, in a testament to my willpower (yay, I can do things in spite of a terrific adversary!), but it's not good enough. I can finally interact with the data through Onsang, and I could do a little more work and have effectively the same degree of functionality I have in my old log keeper (which I still use), but after two years, I simply can't ignore the fact that this has been a monumental disaster. Why should I willingly continue to suffer under these projects? They slow me down and keep me down. The efficiency isn't even the problem, it's the poisonous psychological detriment that results from them. Working with or on Hord, or even thinking about doing so, literally made me depressed. And I don't mean depressed as in “boo-hoo, they stole my cookie”, I mean depressed in that I was unable to do anything productive or be happy doing anything else.

I don't think I ever fully articulated or understood why, but now I know: it's simply because Hord is the worst thing I have created ever since I started programming. I mean that sincerely. Even with all of the issues I could find in my ancient projects and their poor (finger quotes) quality, almost nothing comes to mind that has inflicted more damage than Hord to my psyche and my growth as a programmer, as a creative person, and as a human.

So I'm done. I'm throwing them away. Hord and Onsang will forever remind me that mistakes are costly and that my foresight is (or hopefully was) unreasonable. In three days, as I decided earlier this month, I will go back to game development. I will make a short game using my voice for sound effects and continue development on togo. I'd like the game to be done by the end of March at the latest, and I'm going to shift my education slot from French to drawing, which has been delayed for far too long.

The original plan was to sideline Onsang and only work on it as needed, with game projects being primary. I'm almost happy with where it came and even the thesis that it presents in functionality, but not with the crap I had to put up with to get it there, and the crap I still have to put up with to expand it. When I inevitably re-approach Onsang, it will be to tear it down and start anew. The gravest justification for the mistake of formalism was that it mattered to others, or that it had more value as a result, when in reality I was (and am) the only person who would ever use the software. I designed Hord for outsiders who never existed, and latched onto it as a way of teaching myself new avenues, when it only crippled the software. It should be directed by functionality, not formalism, and it should be designed for itself, not for some ulterior purpose. This is what I take away from the biggest mistake I have made & remade & remade & remade since 2013.

This means I have to give up the dream of Onsang, at least for the foreseeable future. Is that so bad? In reality, no, because I can live healthily just as all other humans do by using their brains, and I don't need the data & statistics it would provide, even if I really want it. There is absolutely no way it would have reached the point of functionality I desire without another year of hard work — or worse. Hard work that would have been wasted, and surely by that time I would be utterly disgusted and declare its banishment from the universe, as I am pre-emptively doing now in the interest of myself. That or I would have redesigned it from the inside-out, which I will be doing in the future anyways! Good riddance.

Comments