I’ve been back a week from a work trip, and — as I often do after a busy period — I’ve been taking stock of what I need to do next and refining my systems a bit. Cynics might argue that this is either procrastination or yak shaving (and they wouldn’t be entirely wrong), but when I have been at full stretch at work I do find it helpful to have a period of sorting out the mess that I had to let accumulate, to tie up loose ends and to plan the next bit of work.

I am spending more and more of my time in Emacs for all things, so I wanted to refine both my agenda and journalling setup.

I had tried using org-super-agenda a while back, but had somehow failed to get it set up correctly. I couldn’t seem to get it to output what I wanted, and eventually went back to setting up a few custom groups using the built-in agenda features. This was OK, but didn’t allow me to do quite what I wanted. Then I noticed that Jack Baty had posted his org-super-agenda configuration, and I decided to give it another go. I copied Jack’s configuration (substituting def-package! for use-package because I am using Doom Emacs), and found that it worked — success! After some thinking about what was important to me to see on my agenda, I ended up with this configuration:

(def-package! org-super-agenda
  :after org-agenda
  :init
  (setq org-agenda-skip-scheduled-if-done t
      org-agenda-skip-deadline-if-done t
      org-agenda-include-deadlines t
      org-agenda-block-separator nil
      org-agenda-compact-blocks t
      org-agenda-start-day nil ;; i.e. today
      org-agenda-span 1
      org-agenda-start-on-weekday nil)
  (setq org-agenda-custom-commands
        '(("c" "Super view"
           ((agenda "" ((org-agenda-overriding-header "")
                        (org-super-agenda-groups
                         '((:name "Today"
                                  :time-grid t
                                  :date today
                                  :order 1)))))
            (alltodo "" ((org-agenda-overriding-header "")
                         (org-super-agenda-groups
                          '((:log t)
                            (:name "To refile"
                                   :file-path "refile\\.org")
                            (:name "Next to do"
                                   :todo "NEXT"
                                   :order 1)
                            (:name "Important"
                                   :priority "A"
                                   :order 6)
                            (:name "Today's tasks"
                                   :file-path "journal/")
                            (:name "Due Today"
                                   :deadline today
                                   :order 2)
                            (:name "Scheduled Soon"
                                   :scheduled future
                                   :order 8)
                            (:name "Overdue"
                                   :deadline past
                                   :order 7)
                            (:name "Meetings"
                                   :and (:todo "MEET" :scheduled future)
                                   :order 10)
                            (:discard (:not (:todo "TODO")))))))))))
  :config
  (org-super-agenda-mode))

It looks really complicated, but the nice thing about org-super-agenda is that items aren’t duplicated between the headings, and also that if there is nothing for particular heading (for example, there are no tasks marked priority ‘A’), then that section will not be shown. What I end up with is a concise list of the key stuff I need on my radar right now. I should say that I don’t keep all my tasks in org-mode: I use OmniFocus for that, but I copy tasks over each day into my journal entry for the day, which is the source for the “Today’s tasks” heading. I also sometimes enter tasks or meeting entries (“MEET” keyword) for particular projects into the Org file for that project so that I can see it in the context of my notes. The todos get duplicated in OmniFocus and meetings are in my calendar. It might seem like needless duplication, but the way I look at it is that OmniFocus and my calendar are my trusted repositories of information which I am scrupulous about keeping updated, but copying some information over to my Org files helps me to focus on actually doing the tasks, and also creating a record of what was done when and why, by adding contextual notes around each item. Plus, I get to mark something as done twice, which is double the satisfaction!

The other component of this is org-journal. This is another package which I had tried out before and stopped using. When I last investigated it (which was probably at least a couple of years ago), I think it only offered the option to create one journal file per day, which felt too fragmented to me. I like having a bit of context so that I can see the current day’s entry together with at least the rest of the week. At the time, I settled for using an org-capture template with a datetree, so that Emacs would create year, month and day headings for each of the entries in one file. That worked OK, and searching was easy because it was all in one file, but navigating around entries wasn’t as smooth as it could be.

When I went back to look at org-journal I found that there is now a setting to create either daily, weekly or monthly files. I settled on generating monthly files and it is working really well for me. The file stays a more manageable size while still providing me with the context I need, and the navigation features of the package make it easy to move back and forth between entries. I also like the calendar integration, which marks dates on the calendar which have journal entries. The calendar mode map also has keybindings which enable you to move to the next/previous day with entries, view the entry on a particular day, or search for entries in the selected day, week or month.

However, I did have to tweak the keybindings a little. The bindings built in to org-journal conflict with the evil mappings used in Doom Emacs. I therefore added some custom bindings to the SPC leader mapping and also to the calendar-mode-map to fix this.

;; in ~/.doom.d/+bindings.el
(map! :leader
      (:prefix ("j" . "journal") ;; org-journal bindings
        :desc "Create new journal entry" "j" #'org-journal-new-entry
        :desc "Open previous entry" "p" #'org-journal-open-previous-entry
        :desc "Open next entry" "n" #'org-journal-open-next-entry
        :desc "Search journal" "s" #'org-journal-search-forever))

;; The built-in calendar mode mappings for org-journal
;; conflict with evil bindings
(map!
 (:map calendar-mode-map
   :n "o" #'org-journal-display-entry
   :n "p" #'org-journal-previous-entry
   :n "n" #'org-journal-next-entry
   :n "O" #'org-journal-new-date-entry))

;; Local leader (<SPC m>) bindings for org-journal in calendar-mode
;; I was running out of bindings, and these are used less frequently
;; so it is convenient to have them under the local leader prefix
(map!
 :map (calendar-mode-map)
 :localleader
 "w" #'org-journal-search-calendar-week
 "m" #'org-journal-search-calendar-month
 "y" #'org-journal-search-calendar-year)

So now I can hit SPC j and then see a list of single key bindings to create, search or navigate among journal entries. In calendar mode, I only need to hit a single key (e.g. ‘o’ to open the entry for the selected day), unless I want to use the searches, which are under the local leader binding (SPC m).

My final tweak was to alter they way that the clocktable is presented. As I mentioned, at the start of each day, I copy across the tasks for the day from OmniFocus to my journal entry for the day. I am trying to get more accurate at estimating how long something will take to accomplish, so I have started adding ‘Effort’ properties to these tasks and using org-clock to time how long I work on them. Incidentally, clocking in to tasks has other useful side effects, because I can use various functions in org-mode that allow you to jump to the clocked-in task to add notes (either directly or via org-capture), which saves me hunting around for it.

At the start of the month’s file, I add a clocktable report, which lists all the timed entries and their estimates, with subtotals of clocked time per day. After a lot of searches to figure out how to do it, I’ve also managed to tweak the table so that it adds another column which subtracts the actual time from the estimated time to show me how much I over- or under-estimated the time needed, and also renames the columns and moves the estimate columns to the right of the table.

;; Format org-mode clocktables the way we want to include Effort
;; In the clocktable header:
;; :formatter my/clocktable-write
(defun my/clocktable-write (&rest args)
  "Custom clocktable writer.
Uses the default writer but shifts the first column right 3 columns,
and names the estimation error column."
  (apply #'org-clocktable-write-default args)
  (save-excursion
    (forward-char) ;; move into the first table field
    (org-table-move-column-right)
    (org-table-move-column-right)
    (org-table-move-column-right)
    (org-table-next-field)
    (insert "Est. error")
    (org-table-previous-field)))

This is used in the header of the table, and automatically generates the #+TBLFM: line below the table, like so (actual entries removed for reasons of privacy!):

#+BEGIN: clocktable :scope file :maxlevel 3 :link t :properties ("Effort") :formula "$5='(- $1 $4);U::@1$1=string(\"Estimate\")::@1$3=string(\"Total\")::@1$4=string(\"Task time\")" :formatter my/clocktable-write
#+CAPTION: Clock summary at [2019-08-10 Sat 19:02]
| Headline                                    |  Total | Task time | Estimate | Est. error |
|---------------------------------------------+--------+-----------+----------+------------|
| *Total time*                                | *6:53* |           |          |      00:00 |
|---------------------------------------------+--------+-----------+----------+------------|
#+TBLFM: $5='(- $4 $3);U::@1$4=string("Estimate")::@1$2=string("Total")::@1$3=string("Task time")
#+END:

I didn’t know you could programmatically manipulate tables like this — it’s really useful. However, I did come up against a frustrating oddity. Even though the ‘Est. error’ column is clearly column 5, (that’s how I create it above with $5), there doesn’t seem to be a way to target that column later to change the column heading (which is why I ended up doing that in my function) or to alter the total time line to sum up the estimation errors. If I try that I get error messages that the column doesn’t exist, and if I try to refer to the last column using the $> shortcut, it modifies column 4. If I add the code manually to the #+TBLFM: line it works fine, but gets over-written when I update the table with C-c C-c. Anyway, it’s not really a problem. The table is useful as it is, and I can finalise its appearance manually at the end of the month by modifying the #+TBLFM: line.

Emacs may provide the gnarliest of rabbit holes in which to get lost, but I continue to be amazed about how adaptable it is, even if — like me — you don’t really know much Lisp. Even if you are a beginner, being able to use the built-in help system to easily find the source code for particular built-in features (like that for org-clocktable-write-default above) and use that as a model for the changes you want to make, or to figure out which package is setting a variable or a keybinding is incredibly powerful. Essentially you are modifying and building upon your copy of Emacs itself, not just sticking on a plug-in which is limited in power and scope.