Mundy Reimer

Math Notation as UI: Does it need an update?

29 June 2019

Created: 2019-06-29
Updated: 2021-04-17
Topics: Mathematics, Philosophy, Cognitive Science
Confidence: Speculative
Status: Still in progress

TL;DR - Our written scripts and mathematical languages are tools. Can we mindfully design our these tools with the end user experience in mind?


For the past number of years I’ve been on a quest to improve my intuition and understanding of the concepts behind the dense and often archaic-looking magical symbols in the world of mathematics. I call it cultivating a feeling for Pattern. It might catch some people by surprise, but I’ve pretty much mostly sucked at math a big chunk of my life (and honestly I currently still do!) I mean sure, like many others throughout their educational experience I could memorize a few rituals here and there (taking the typical engineering math curriculum for example), like oh yeah so this is where I use that one logarithm trick the teacher mentioned, or ok so I use integration-by-parts here, this is a Taylor series expansion there, or this is a non-homogenous differential equation so let’s use the method of undetermined coefficients, or hey this is an optimization problem so I’ll need my Hessian matrix and now okay let’s see how those eigenvalues are characterized yadda yadda, etc etc. However, all of this was just basic regurgitation…follow the particular ritualistic movements of the professor, memorize the various templates of problems, and match the corresponding pre-created method/key to the pre-formatted problem/lock. It’s what turned me away from math at the beginning because I felt guilty and fearful about never really getting it or understanding anything. I was just being a machine going through the motions.

So these last couple of years got me thinking when was the last time I actually felt math, and created it like how one would create art or music. When did I last play with it. And for me, that was Geometry (and much much later Graph Theory). It might be because evolution and natural selection have biased us in favor of being predominately visual creatures, but I felt the closest affinity to mathematics when I could actually visualize and see things happening. Languages, grammar, and symbols are flippin’ hard to learn, but pictures, heck, those are actually easy and fun.

Now I totally understand the common criticism of a need for rigor and not getting led astray by one’s visual intuition, but I don’t think that’s necessarily mutually exclusive with using pictures and diagrams as a language itself. Let me repeat that - Using pictures does NOT mean you have to give up rigor! The Greeks achieved great feats using just their lines and curves, Leonardo da Vinci himself relied upon visual perspective + proportion and did not know much ‘symbolic math’ at all, electrical engineers reason using pictures all the time with circuit diagrams, chemists with stereochemical+molecular diagrams, physicists with feynman diagrams, and sheesh, if we really want to generalize, then algebraic manipulation of letters and numbers is still a graphically-based reasoning process we do on paper and chalkboards (why else would teachers keep emphasizing that you physically go through the motions of writing and solving a problem on paper). Many of us don’t realize that this symbolic manipulation game that characterizes the vast majority of mathematics education today is really a byproduct of the historical notation introduced by Francois Viete (I highly recommend his Wiki page!) who claimed that “all problems could be solved with it”, and whose ‘Algebra’ only actually started taking off and being used from 1638 onwards. Does that mean that all math done before that was never really considered math? I don’t think so. As Judea Pearl puts it, “To proclaim algebra the UNIVERSAL language of science, would sound today like proclaiming Esperanto the language of economics. Why would Nature agree to speak Algebra? Of all languages? …[However], you can’t argue with [its] success.”

Regardless of success, what we can argue is that the cognitive overhead required by Algebra and the language of symbols that we partake in today is still quite demanding in that we have to keep much of the actual semantics and meaning of what we are doing in our heads (in addition to storing the vast majority of ‘what-this-all-means’ in a collectively agreed-upon mathematics pedagogical culture). As a result of this, we suffer by blindly becoming machines manipulating symbols who consequently experience a loss as to how to relate this all back to reality. Think about all the students who often complain about when they will ever use this. And any parent helping their child with homework knows that without practice this symbol manipulation ability that they were drilled in when younger inevitability gets eroded by the ravages of time and memory loss. Historically the symbols may have freed some of our short term memory when compared to before, but they left most of us with our motivation and insight left beaten and blindfolded in the dark. Many of us never partake in recreational math for fun and many of us have trouble picking the symbolic language up again. And yet many of us will still play graphically-based logic games like the various incarnations of ‘Bejeweled’ on our phones everyday, so it’s not for lack of our inner desire for puzzle-solving, but instead maybe the problem lies with the symbolic script-centric education itself.

While it could be argued that things may indeed have improved intuition-wise with the marriage of algebraic symbols (from the Arabic ‘al-jabr’), Indian numerals, and the Greek geometry united by Descartes and his Cartesian coordinate system which ultimately led to the birth of what we now popularly call Trigonometry, I would argue that while innovative, this was more a temporary band-aid than an actual cure for our loss of visual insight and feeling for meaning.

So instead of this loss of meaning from using ‘dry’ symbols, maybe we could alternatively be using symbols or glyphs that are a lot more graphically intuitive and ease the learning process for the first time, as well as making the process easier to ‘pick-up-again’ after extended periods of non-use. For instance, we can adopt mathematical symbols that actually visually embody or ‘look like’ pictures that are closer to their underlying conceptual mechanisms, analogous to how for instance the Korean script, Hangul, was creatively designed such that their basic consonants visually mimic the shape of the speaker’s throats when pronouncing each (gosh do I envy their script!) This ability to visualize processes is important. I don’t think it’s just a coincidence that those ‘human-calculators’ we see in competitions were originally trained on a physical abacus and still mentally visualize one + move their fingers/hands shuffling imaginary beads around when performing their seemingly superhuman feats of numerical performance (it is also interesting to note the historical conflicts between the abacists and the algorists back in Fibonacci’s time; only finally getting resolved sometime around the 16th century in favor of the algorists). Anybody who has studied the brain or knows how competitive memory champions undergo massive amounts of memorization would know how powerful our internal visual system is if used properly.

In addition to adopting intuitively looking glyphs, this collective grouping of images could also be designed such that the computation is embedded in the process of making/reading the picture itself, similar to the causal logic embedded in electrical circuit diagrams. On a relevant note, it’s interesting how Pearl’s probabilistic graphical modeling that is revolutionizing the study of causality and statistics was also inspired by electronic circuitry pictures. In fact, Pearl champions the view that pictures give one the sense of how each part or component fits into the whole and how its logical relationship with every other part is made much more explicit, allowing us to intuit at-a-glance any possible scenarios where the logic might break (think of a plumbing layout for a house and how you could intuitively figure out the consequences of hypothetical scenarios where one plumbing line might break), compared to if we used a language like algebra which requires us to serially read line-by-line, further demanding us to mentally build up and imagine the conceptual architecture in our heads while maintaining that mental construct in our short term memory and further still being able to manipulate it (in addition to Pearl’s argument that these equations don’t fully capture the causal network of reality, but that’s beside the point).

For example, I remember this one time where my younger sister asked me for help understanding this equation and I was like ‘oh, there’s some energy or cost minimization occurring there and also some barrier causing it to rise steeply here’, and she was like ‘huh, what???’, to which I further explained ‘that’s a hyperbolic looking curve shifted over this much with these oblique / slanted asymptotes here’, and she responded with ‘I don’t remember how to do my graph shifting by reading these equations and our teacher only taught us how to find vertical and horizontal asymptotes not diagonally looking ones, how did you know that?’, to which I yacked off something about the degrees of the polynomials in the numerator and denominator and how she can also figure out stuff by the symmetries inherent in the even-ness or odd-ness of the equation. Anyway, my point is that if you didn’t memorize these ritualistic procedures and esoteric facts or yet alone given the opportunity to learn these concepts from a gifted teacher, all of these symbols are basically gobbledygook to you (and I can definitely say are still considered chicken scratch most of the time for me too unless I constantly review and practice my fundamentals, like a musician practicing scales or a martial artist doing kata. Furthermore, I was not one of those lucky ones gifted with awesome math teachers, but instead most of my math knowledge was hard-earned through the sweat of self-study and reading books on my own free time, often after I had already encountered trouble with it in the formal school setting). Why can’t mathematical machinery like this be more visually intuited from a more mindfully designed set of graphical symbols rather than an often scary looking equation, so that even with years of non-practice one can still comfortably read and visualize the underlying concepts that are written?

So rather than viewing our picture-oriented intuition as an innate bug embedded in our native visual wetware of a brain, maybe we can explicitly design our linguistic tools such that this visual intuition is seen as a feature instead. Shift our view from our visual intuition being an error-prone biological burden, to it being a champion GPU-like stallion. Where we see flaws, Mother Nature saw finely-tuned instruments and sharpened tools. Let’s work with Her, shall we?

And this is why I applaud brilliantly creative efforts like this one person’s series of blog posts on creating a graphical version of linear algebra similar to the circuit diagrams found in electronics. To see the concepts behind these symbols, distill out their essence, and then recast it into an intuitive picture-based language. Linear algebra is a beautifully geometric subject (and arguably the most useful), and I’m glad this person is trying to free it from its currently dry symbolic representation and definition-centric pedagogy (from which they argue is partly the result of the Bourbakian move towards abstractionism in mathematics). And through this newer visual representation, maybe even new insights could be gleaned that were not cognitively readily available before…

So my fundamental message is thus: Our written scripts and languages are tools. We don’t make tools ignoring how the end user will be using them. So why do these linguistic tools which are used not to carve wood or hammer nails, but arguably more importantly used to expand our minds and parse reality in ways that transcend our physical bodily limits, mostly ignored for improvement? We can mindfully design these tools with the end user experience in mind. Let’s take the art of mathematics more seriously and start playing with pictures!


The Essence of Graphical Linear Algebra

Enlightening Symbols: A Short History of Mathematical Notation and its Hidden Powers: Did you know that “as late as the 19th century, some writers suggested that negative numbers should be written as positive numbers flipped horizontally”? Now obviously there are downsides to this using our Hindu-Arabic derived characters for our numbers (for example, 0 and 8 are symmetrical when flipped), but it’s this kind of thinking that I’d like to propose in re-designing our math symbolic manipulation system to become more aligned with our visual intuitions.

Notation And Thinking: And it seems Kenneth Iverson is my guy! How did I not hear about this history of APL before. “The importance of nomenclature, notation, and language as tools of thought has long been recognized. In chemistry and in botany, for example, the establishment of systems of nomenclature by Lavoisier and Linnaeus did much to stimulate and to channel later investigation. Concerning language, George Boole in his Laws of Thought asserted “That language is an instrument of human reason, and not merely a medium for the expression of thought, is a truth generally admitted.” Mathematical notation provides perhaps the best-known and best-developed example of language used consciously as a tool of thought. Recognition of the important role of notation in mathematics is clear from the quotations from mathematicians given in Cajori’s A History of Mathematical Notations. They are well worth reading in full, but the following excerpts suggest the tone: By relieving the brain of all unnecessary work, a good notation sets it free to concentrate on more advanced problems, and in effect increases the mental power of the race. —A.N. Whitehead The quantity of meaning compressed into small space by algebraic signs, is another circumstance that facilitates the reasonings we are accustomed to carry on by their aid. —Charles Babbage”

Notation as a Tool of Thought by Iverson. “Kenneth Iverson was a mathematician who is most famous for designing APL. This was the name of his programming language, and it cleverly stood for “A Programming Language.” The language is unique—unlike almost any other language—and contains many powerful and interesting ideas. He won the 1979 Turing Award for this and related work. Today I want to talk about notation in mathematics and theory, and how notation can play a role in our thinking. When I was a junior faculty member at Yale University, in the early 1970’s, APL was the language we used in our beginning programming class. The reason we used this language was simple: Alan Perlis, the leader of the department, loved the language. I was never completely sure why Alan loved it, but he did. And so we used it to teach our beginning students how to program. Iverson had created his language first as a notation for describing complex digital systems. The notation was so powerful that even a complex object like a processor could be written in his language in relatively few lines of code: the lines might be close to unreadable, but they were few. Later the language was implemented and had a small but strong set of believers. Clearly, Alan was one of them who once said: A language that doesn’t affect the way you think about programming, is not worth knowing.”

“A foolish consistency is the hobgoblin of little minds, adored by little statesmen and philosophers and divines.” - Ralph Waldo Emerson

How can we develop transformative tools for thought?: Inventions which can “change the thought patterns of an entire civilization” (Alan Kay) 🧙‍♂️❤️…brilliant article by Andy Matuschak and Michael Nielsen.

“Let it have been drawn”: the role of diagrams in geometry: “Diagrams. What are their role in geometry? Some people like to think that the logic of a geometrical proof doesn’t need the diagram. Mathematics is supposed to be pure and absolute. Diagrams seem connected to the visual, the intuitive, that makes it kind of psychological, and perhaps therefore even subjective. Certain people don’t like that association one bit, so they try to minimize the role of diagrams. Maybe diagrams are just crutches to help those with weaker minds, whereas a perfect logical reader could follow the proof from the text alone. Some people like to think so. It’s a dogma that fits modern tastes. But, historically, that interpretation is a pretty poor fit. In some ways, classical geometry appears to have embraced visuality rather than tried to replace it with abstract logic. There are signs of this attitude in the very language of Greek geometry. The word for proving is the same as the word for constructing: grafein, to draw. To prove something is literally to make it graphic. And a theorem, in ancient Greek, is a diagramma, a diagram. Instead of the Pythagorean Theorem the Greeks would say the Pythagorean Diagram.”