On tracking

by Coranna Howard in Self

Time, biometrics, nutrition, finances, media, what my body is up to.. I realize I'm hyper-obsessed with tracking things.

I didn't suddenly wake up one day and decide to do all of this, though. It grew over time, as with all things that “define” a person.

I started in 2011 with tracking time for work. This included a bit of context (i.e., what I was working on), and became more detailed & segmented through the year. At some point early on, I decided to fill in the empty space between work segments.

Why not? That thought repeated itself as I expanded what I was tracking: what about all this other information that I'm leaving out?

  • Why not track the items & measurements in a meal?

  • Why not include addresses in transit entries?

  • Why not describe specifically what I am doing on work & projects, instead of just the context?

I am an explorer and a perfectionist at heart, the only apparent limit to which is my level of knowledge in what I am exploring. So, naturally, it feeds itself.

As complexity rose, so too did another question: how do I structure this information? Up until 12 August of this year, I was using a time tracker called Hamster. It had a simple design: activity & category (as a hierarchy) types an entry, tags collate similarities, and a free-text blob describes an entry.

The free-text blob (“description”) is sufficient to include anything I want (textual, at least), but cumbersome and not very helpful for statistical analysis, as Hamster doesn't understand the data within. It also bogged down Hamster as the amount of information and fragmentation grew.

I wanted a better solution, but didn't know where to start. I had already been using Notery, a glorified spreadsheet application I created with Mono (C♯) & Gtk♯, for logging media & tea, among other things. I thought maybe I would do something like it again, but Gtk♯ was out-dated, and I wasn't very fond of C♯. I also wanted to make a pared-down version for my phone, so code reuse was a big factor. I briefly considered using HTML & JavaScript, but quickly threw that idea out — I am intensely more loathsome of web tech than I am of C♯.

In 2013, I started building the scaffolding for this system in C++ as Hord, postponing the user interface until I had decided what system to use. I got distracted from my main goal and didn't get very far. I prematurely optimized and designed. Instead of exploring the UI and the structure of tracking data, I generalized and obsessed over minute details that weren't important.

I abstracted the interfaces and systems to a degree that made it hard to work with them. I was fed up with the design of every UI stack, and somehow decided to start building the frontend, Onsang, with a terminal-based UI in late 2013. Abominable, yes, but I still consider that a better route than, say, the haven of bloat that is Qt (depending on the application).

Onsang was poisoned by the design thought I was mired in at the time and thus, by extension, Hord. Onsang was barely functional by 2015. I was initially focusing on the data model and replacing my spreadsheet application, but it was barely capable of replacing it by that point. The idea was that I could build tracking systems on top of the flexible data model, which was effectively no different from a relational database. I considered using SQLite for storage, but I was unhappy with its data model. I wanted exact, optimal storage of records, which SQLite didn't exactly provide. This was the biggest premature optimization I made, and one that was wholly unnecessary.

At the end of my sprint on Onsang's functionality in January 2015, I finally concluded Hord & Onsang were a toxic duo that weren't going anywhere good. I threw them away.

Meanwhile, the data I was storing in Hamster continued to grow and crystallize into a recognizable, parse-able format. Kinks were ironed out as I unified entry data with a notation that was created for nutrition. I was again eager to replace the sluggish & insufficient Hamster. I started exploring the syntax & structure of a text-based tracker, which spawned Quanta.

The core of Quanta is a notation honed for description. It has a single data type, the “object”, which stores value (null, boolean, number, time, money, string, identifier, expression), value uncertainty, source/variant (e.g., manufacturer), quantity (e.g., mass measurement), tags, and children. With this, I have the freedom to design and explore systems through data.

It works well for this purpose. I quickly designed the core tracking system and further unified the structure of actions. I didn't write a line of code for parsing the notation, yet I had a concrete system for tracking time. grep is sufficient for my casual historical & statistical analysis at present, though it will be a while before I have developed the code to do real statistics and integrity validation.

The first thing I built after pinning down the notation and tracking system was an assistant plugin for Sublime Text 2. The assistant has shortcuts to create time entries and auto-complete for actions. It is much more efficient to use than Hamster. I also created scripts to manage tracker data within a “vessel”, a container for any Quanta data. My main vessel is a tiny USB thumb-drive. I haven't yet envisioned a need for more than one.

On 12 August, I switched from Hamster to Quanta. Immediately, I realized I could track even more information, and at a reduced overhead compared to Hamster. I expanded the action set, and the entry count per day rose by about 1.4-2.0 times that of Hamster.

Accuracy also rose. Hamster tracks to the second, but doesn't show them, and elides them on user input. With Quanta, I track entries to the second (when known). Informational density within actions also rose, as the data is more expressive and comprehensive.

I am now able to retire Notery. Once I started to track (macro-level) nutrition, I stored entity data within Notery. This was very limiting. Though I wasn't yet tracking mass (only basic quantity at that point), it wouldn't have been sufficient. It would have been very cumbersome to include nutrition data within Notery — possible, but atrocious.

As Quanta is simply a notation, I can describe entities in full. I eagerly define a “universe” and populate it with entities. Whereas before the ephemeral tracking system only knew (I say this loosely, as the system was in my head) about entities as “food” and “drug”, it can now describe and understand anything. I can also use an entity category as a context to search within when writing actions, instead of specifying the whole identifier of an entity. Many actions have implicit contexts; for example, the action for eating has an implicit context of “food”, the root category for foodstuffs.

This might just as well be thought of as part of the coded system that understands the eating action (and therefore that its context is the “food” category), but I am developing a more flexible system that can understand classes of actions and entity instances & compositions, without necessitating the creation of code whenever a simple concept is introduced. Obviously, that by is the facility of a program, one that is necessary, but whose intervention I am trying to minimize.

I might eventually have systems that create & modify primary data, but I expect this will only be for special situations. Mobile is a good example: I'd like to have a simple application for my phone that can create time entries to be fixed up later on a PC. There's no reason it shouldn't be Quanta data. It would likely write directly to Quanta notation, and I'd copy the data over and fill in missing details when I am back at a PC. The process of insertion into a tracker file could be automated, but it would have to be able to retain the layout of the notation — the Quanta writer would have to be more intelligent before I would do that. Compactness matters, as I am directly seeing & manipulating the notation.

My current goal is to calculate nutrition data. It's not a difficult problem, but it's slow going as I have outstanding issues with depression and motivation. In fact, the system is the easy part. It'll be more difficult to collate the nutrition data about foodstuffs to attach to the entities.

To automatically troll through a tracker file and pick out the nutrition-related entries, I'll have to develop the rudiments of the overarching analyzer, which will dole out entries to sub-analyzers.

It looks like the majority of Quanta will be written in Lua, with C++ backing it for efficiency. So far the object interface and hobbled-together parser are in C++, and not much else. This is a promising approach. I think the model of the analysis tool will be common; I have other domains of data that I also track, which I could pull into Quanta and expand the toolset further.

I am excited to see what I can do with it. The yearly retrospective posts I've been doing (2013, 2014) are interesting looks at how I've spent my mortal time on this planet — to me, at least. They haven't been granular enough to satisfy my whims, but with Quanta I'll be able to go beyond what I hoped to do. I probably won't be doing statistics for my tracking this year, but there's plenty to say otherwise.