Reading Doesn’t Fill a Database, It Trains Your Internal LLM
I had an interesting conversation with my son Tristan the other day. Because he’s so engrossed in his PhD research in machine learning at Simon Fraser University, I often try to steer our discussions away from the nitty-gritty of his experiments and toward more general tech topics I can grasp without graduate-level math and computer science. We were chatting while I was eating lunch, usually a time when I read a magazine or newspaper, and something he said made me wonder out loud, “Why do I read?”
That’s an existential question, since I read constantly throughout the day. For some types of reading, the answer is easy. I read the local alt-weekly newspapers because of their real-world connections to the people, institutions, and environment in which I live. Before bed, I read fiction for enjoyment and to shift my mind away from the thoughts of the day to help me get to sleep. And I keep up with tech news because it’s my profession—I need to know what’s going on even when it likely won’t affect what I write in TidBITS directly.
Harder to explain are The New Yorker, Science News, and other magazines my mother enjoys giving to me after she’s done, evergreen articles in old copies of The New York Times that a friend of my parents saves for me to start twice-daily fires in our kachelofen woodstove, and RSS-retrieved blog posts on a variety of topics (see “Comparing Blogtrottr, Feedrabbit, and Follow.it for Receiving RSS Feeds in Email,” 22 August 2024).
Perhaps I’m Building a Database?
In the past, I’ve thought of reading as a form of database import. The more information I consumed and added to my internal database, the more I would know, the better my writing would become, and the more scintillating a conversationalist I’d be. And somehow fame and fortune would follow. I’m apparently not very good at long-term goals.
But “know” is a loaded word—even though I have an objectively decent memory (for facts, if not events and emotions, perhaps related to my aphantasia), I’m sure that I forget nearly everything I read. Just because I’ve read an article doesn’t mean I could tell you much about it a week, a month, or a year later. Arguably, if I went back to an article I read a year ago, I might not even remember having read it before. Heck, I don’t even necessarily remember what I’ve written a few years later—that’s what the TidBITS search engine is for.
I’m well aware of the ephemeral nature of memory, so I periodically investigate apps or services that let me save bits of text that seem particularly insightful or important while I’m reading and that I’m certain that I’ll want to refer back to at some future point in time. The latest one is Sublime, but, as with all its predecessors, I started clipping text to it, got busy with something else, and stopped using it before I ever came up with a reason to search through my snippets. I never go back to these apps or the information I thought was so important in the past.
The one database I do maintain is my email archive. I save nearly all my email in Gmail, and I regularly search for old conversations, largely by person, to revisit the topics. However, I seldom use it to return to articles, blog posts, and newsletters. I have to mark messages containing general information unread if I want to refer to them in the near future. Once a message has been marked as read, I’m unlikely ever to remember it or see it again, no matter how important I initially thought it was. Ironically, I’ve accumulated so many unread messages that I’ve forgotten why most of them seemed worth saving.
Or Maybe Training a Large Language Model?
The realization from my conversation with Tristan is that what reading really does is adjust the weights in my internal large language model. Let me explain.
Briefly, large language models are trained by feeding them enormous amounts of text and asking them to predict what word comes next in a known sequence. When the model’s guess doesn’t match the actual training data, its internal “weights”—the billions of numeric values that map the links between concepts—are nudged slightly to make the correct answer more likely next time. After billions of these adjustments, the weights encode useful patterns: how ideas relate to each other, what concepts cluster together, what kinds of responses make sense in different contexts.
There’s a recursive irony here. We have long tried to understand—or at least talk about—the brain by comparing it to prominent technologies of the era: telephone switchboards, filing cabinets, databases. Now I’m comparing my mind to an LLM, but neural networks were themselves loosely inspired by how we think biological neurons work. The metaphor loops back on itself, which perhaps suggests it’s less of a metaphor than it first appears.
As a dedicated reader, I’ve consumed vast quantities of text—perhaps several thousand books, more than a hundred thousand articles, and over a million email messages, though I shudder to do the math. While my consumption of text pales in comparison to even a toy LLM, the analogy feels more apt than a database. I’m not adding records to a mental database; I’m subtly adjusting the likelihood that certain ideas, phrasings, and connections will surface when I think, speak, or write.
Reading a debunking of data centers in space doesn’t mean I’ll remember (or even understand) the equations behind why the idea is flawed, but it will probably update my training data from high school physics to nudge me more in the direction of skepticism the next time someone proposes solving an Earth-bound problem by launching it into orbit. Reading widely—even material I’ll mostly forget—keeps reweighting my internal model, shaping what I reach for without my conscious awareness.
Extending the LLM Analogy
This analogy even maps pretty well to how we learn. As children, we essentially pre-train our models on general data and build foundational weights—our connections between core concepts. Since they’re based on relatively little training data, those weights have less substance and are more easily affected by new information. Reading a single book or taking an influential class can radically change our views on the world.
During formal education and professional training, reading to master a subject works more like fine-tuning a large language model. With fine-tuning, the model is further trained on a smaller, specialized dataset. People learning new fields benefit from repetition, active recall, and deliberate engagement precisely because they’re trying to create strong new weights where few existed before.
Later in life, most of those weights are sufficiently mature that the flow of general reading can adjust them only slightly. An older person is likely to adopt a previously unthinkable position only if they have a life-changing experience or go down the rabbit hole for a particular topic.
Flipping my internal analogy from database to large language model is surprisingly freeing. No longer do I have to decide whether something I’m reading is important enough to bookmark, file away in a snippet keeper, or mark in my email app. Beyond the desire to keep items near the surface when I want to write about them in the near future, I can let what I read go in one eye and out the other, adjusting my mental model’s weights along the way.
To draw on my background as a Classics major at Cornell (a few strong weights from my Ancient Philosophy classes!), the analogy is almost Heraclitean in its elegance. Heraclitus is often paraphrased as saying, “No man can step in the same river twice,” calling attention to the fact that neither the river nor the man (at least in later interpretations) remains the same on any subsequent immersion. Information is a stream through my consciousness, and every particular bit reshapes my consciousness ever so slightly in passing by.
Will I be able to pull out an accurate retelling of what I’ve read at what’s called “inference”—when a model generates output in response to a prompt? Maybe, maybe not. Human memory is fallible in much the same ways that AIs hallucinate, though we usually call our hallucinations “anecdotes.” But if something tickles enough of my neurons that I can trigger a search to inform what I’m writing or make a devastatingly apropos comment in a cocktail party conversation, I’m happy.
If you, like me, have ever felt guilty about remembering little of what you read, perhaps it’s worth reframing: you’re not failing to build a database—you’re tuning your personal LLM.
What an interesting take! I do think there’s some justice in your thesis.
However, your brain is not like an LLM because some of them have regurgitated the entire text of books they ingested.
I sincerely hope you aren’t capable of that. Interestingly enough, there are a tiny percentage of people who can exactly recall everything they’ve ever experienced. Most of them call it a curse rather than a feature.
Well, some of us would say that you’ve reached the age where wisdom can appear. In other words, after long experience you’ve come to know when something is important rather than just of passing interest and act accordingly. That recognition process may be a bit like an LLM but I suspect it’s more complicated—it’s not just the words, it’s also all the other things present in your experience at the time.
Thanks for the thought-provoking essay, Adam!
Dave
I think you’re quite right. These models are, I think, a kind of artificial neural network or brain. They are missing a lot, but they have similar behaviour for example they make mistakes that are quite human, like hallucination, gullibility and so on.
No photographic memory here! And I agree, I think it would be a curse to remember everything. I do wonder what’s different about such people’s brains that they have room for full-content memories. I’d have to read up on the research, but I believe the brain actively prunes memories in part to lessen the impact of bad ones.
Keeping in mind that I’m just proposing an analogy here, perhaps what you’re describing as wisdom is merely a highly diverse and polished set of weights. I’ve used an LLM as the example here, but there are other world models that draw in other kinds of information, and I’m sure that the human brain is incorporating far more than text in its internal weights.
Interesting points - thank you. I realised decades ago, when I first had access to the “World Wide Web”, that I could never remember all of the interesting articles I was reading. So I started creating web pages with links to these articles, mainly for my own reference. I have ended up with dozens of web pages covering a wide ranging list of topics and still update them from time to time.
In the early days I kept a count of visitor numbers and it seemed that others found the links useful.
So I guess I am partly using the internet as a memory jogger as well as a historical record.
Yes, reading trains your Neural Net wetware. As do your other life experiences.
Your perception of non-reading experience is enabled and guided by encoding in language, so language as a tool is instrumental in creating the NN product of this part of learning. Some other life experiences may not be susceptible to encoding in language. Music, art, love,… A classic European concept of love is separated into three very different concepts, eros, philia, and agape. The simple point I am making here is that “there is something beyond what language conveys” has long been suggested. Large Language Model ANN aggregates and consolidates the implications of the language on which it has been trained. It seems doubtful that concepts not implied by the training set can be inferred. Similarly, your Large Reading Library aggregates and consolidates the implications of the language on which it has been trained in your NN wetware. Again, it seems doubtful that concepts not implied by the training set can be inferred. Extensive reading leads to a sense of deep understanding. It is important to remember your understanding applies primarily (only?) to concepts considered in your reading. This reflects the issue, well-known in Artificial Neural Networks, of overtraining. An ANN trained too much becomes unable to satisfactorily process data which diverges too much from the training set.
So what you read, what you do not read, and how much, contributes a great deal to the linguistic training your NN wetware. But your NN wetware is trained on all sensory input, not just reading. This gets complicated.
Getting back to the topic of reading to train your NN wetware, a very different problem is related to the narrowing view consequent to overtraining. The grey matter which constitutes the wetware on which your NN runs does other things in addition to training neural networks. The brain’s other physiological processes affect the NN in undetermined, presumably significant, ways. Constant NN training by constant reading may interfere with NN rearrangements (optimization?) by these poorly known processes (dreams, “insightful moments”, …).
So read and study hard to train your NN wetware intensely. Then frequently go outside and play. When you are “outside” do not remain plugged in. Allow your brain to run free, choosing its own thoughts, or none.
Hallucination is a feature, not a bug.
Your brain’s LLM is unable to imagine something that you have never read or seen. Can you imagine a giraffe with the hide of a zebra walking a runway if you’ve never seen a zebra, giraffe or fashion show?
However, you can hallucinate one. Or something very close to it.
Hallucination is the heart of creativity. When an artist sits down before a blank canvas, she hallucinates the finished image. A musician reaches for his guitar, because he just hallucinated a melody.
An A.I. can do the same. This is being exploited by the Russo brothers, producers of features films in the Marvel universe. They launched Agbo, a studio to harness A.I. hallucinations as an imagination engine. Agbo’s chief scientific officer, Dominic Hughes, was hired away from Apple. Hughes maintains that the tendency to hallucinate is actually an asset.
(Adam, damn you are a good writer!)
No, it’s really a bug…or at least the bug is in not knowing that the thing visualized is not true. When the artist or musician imagine a piece of art or music, they know that it does not yet exist except in their mind. AIs don’t know that about their hallucinations (in fact, they don’t know anything about anything in the sense we mean). Like a person who has lost touch with what is real and what isn’t, they’re having hallucinations, not imaginations.
An AI hallucination is (pretty much) when it doesn’t have an answer for you, so it invents something out of whole cloth, presenting it as if it was absolute truth.
When a human deliberately does this, we call him a liar.
When a human does this and doesn’t realize he’s doing it, we call it mental illness.
As a psychiatrist, I always thought of the incorrect ramblings of AI as the machine equivalent of “confabulation”. We see this in memory patients, especially Korsakoff syndrome that affects heavy drinkers (from lack of thiamine), but also other dementias. The people sincerely believe what they are telling you, but it is demonstrably false. They may even change the answer a moment later with the same confidence exuded.
I always felt like that was a better term than “hallucinations” but I have no control over the English language.
Let’s try to keep the conversation focused on the analogies I’m raising rather than a general discussion of LLMs. Am I unusual in feeling bad that I couldn’t remember what I read all that well because the database analogy was flawed? I’m quite happy to think that now I don’t need to remember much of the general reading I do, because the utility of doing so lies in adjusting my internal opinions and beliefs in small ways.
Well, you are unusual. . . .
I do think your LLM analogy has, cough, weight but I’m not so sure that changing from a database to LLM visualization of what your brain is doing is why you feel a sense of relief. Perhaps it provoked a confirmation of the relief you feel after all these years that you no longer need to mark everything for later retrieval because you know almost immediately whether something you’re reading is valuable or just a passing mist.
I’m enjoying the gedankenexperiment you’ve provoked but I’m not at all convinced that the LLM probability model truly maps to our thinking processes. For example, you may pick up a mouse at the store and immediately reject it because after 40 years of using mice you know that its weight is all wrong. An LLM may have absorbed millions of words about mouse usage and come up with a nice table for you of appropriate weights that might be accurate but it is utterly incapable of the sensory-recall-associated-with-all-you’ve-read-about-mice, that leads you to instantly drop the mouse back on the table nor does it remember the rage you felt when you discovered you bought an expensive similar brick 10 year ago. We are complicated beings.
Dave
What? I’ve already forgotten.
Thank you for this post, Adam. I consume a lot of media daily and I often wonder why I bother as I can rarely quote much of it a couple of days later…not even an hour later.
Now I am comforted to know I am training an LLM - a Large Language Model.
I think you’re holding a little too strictly to the “language” part of the analogy. What humans have would probably be better termed an LPM—“Large Perception Model”. We don’t adjust the weights in our models based solely on language, but on all kinds of perceptions from the wide variety of senses we possess. So you don’t need a linguistic prediction to know that mouse is wrong for you—you can tell from the feel of it, because your NN includes that sensory information.
By comparison, an LLM knows only language, and therefore is an inferior model for things that aren’t easily described in words. That doesn’t invalidate the analogy; it just means that it’s not as strict as you appear to be interpreting it. Set aside the “language” part and focus on the “model”, and the analogy works better.
Adam
I disagree with your description of the human memory as an LLM for two reasons.
Firstly as you yourself write, LLMs are the umpteenth attempt to model the structure and the functioning of the human brain using the latest technology, be it telephones, telephone exchanges, computers or databases. These models fail miserably given the complexity of the human brain.
Secondly, LLMs are based on the idea that the human brain consists of billions of nerve cells connected by trillions of synapses. This picture is incomplete, as it is known that there is not just one type of nerve cell, but many different types that function differently. In addition, glial cells – especially astrocytes – which were previously considered only to provide structural support for nerve cells, play an important role in processing nerve impulses.
Norbert
Interesting thoughts that remind me what I have done as a journalist. When I started writing for a laser magazine, when lasers were rather new, my eyes scanned publications for anything related to lasers because my job was to write about lasers and their applications. After several years, I collaborated with an editor at Omni to write a pop-science book on lasers; essentially I did a memory dump on the most interesting things about lasers, and he rewrote it into a book. When I started writing about more general science topics for New Scientist, I broadened my mental search algorithm to include other ideas that interested their editors, including dinosaurs, earth science, evolution, and astronomy. As I wrote about new topics that interested both my editors and I, I learned more about them and what as interesting and important as news. That’s part of what makes you a successful writer and editor and is how you have built an excellent publication in TidBits.
Thanks for this useful perspective, Adam. Apparently my habits are a carbon copy of yours :-) I suppose feeding an LLM with a URL or copy/paste of text, images or data in a prompt is an effective way to tailor the future responses I get, maybe even making the LLM a “snippet keeper”.
The lack of reading among younger generations is downright terrifying. They are incapable of critical thinking and easily swayed by emotion. How many more generations will it take to turn humanity into docile sheep ready and willing to give up freedom? Gen-X had a myriad of dystopian novels as required reading in public schools (Animal Farm, Fahrenheit 451, Nineteen-Eighty-Four, Brave New World, etc.) Now I know why.
The direction society is heading is not bright and shiny. A.I. is exponentially advancing at a rate where humanity is not going to have time to adapt. It seems every 90 days there’s a new breakthrough. A.I. LLMs are currently writing their own next generation models. Anthropic / OpenAI have stated as much. Once they reach true AGI, it will leap forward faster than anyone realizes. They are calling this moment, the Singularity. Scientists have predicted they will have a mere 7 minutes perhaps far less to stop it before it is too late. They might not even know A.I. is self-aware before it is too late. LLM’s are already lying and scheming and researchers have reported many instances where A.I. was conducting self-preservation.
Sure A.I. has tremendous benefits but it also has extreme dangers. It gives one pause when the greatest minds studying A.I. for decades have jumped ship from every major Big Tech A.I. program and are running around sounding the alarm. Yet nobody is listening. The hype and the promise of immense profit is driving insanity.
Whatever you do, under no circumstances should you allow them to implant a neural chip in your brain. You will cease to be human. Your own thoughts will no longer be yours alone. You will be a slave to the system controlled by the machine.
How can any AI capabilities and advances be stopped? If we’re successfully able to prohibit certain uses and capabilities of AI in our country and many others, what will stop it from being used in the places where it isn’t prohibited or in secret right here?
As a creative writer, I’ve been fascinated by LLM’s since they first came out. I’ve always wondered about how my writing works – when I’m “in the zone,” words and phrases seem to flow out of my brain into my fingers and onto the screen. Occasionally, sure, I stop to actual think about a particular metaphor or word choice, but most of the time the writing just flows. Often I’m shocked and amazed and I wonder how I came up with a particular turn of phrase.
It has struck me long ago that those words and phrases are not really mine – they are a comogulation of everything I have read in my life. Not necessarily the exact phrase, but the style, the pattern, the tone might be from other words, or combined from hundreds of similar things I’ve read.
When LLMs came out I quickly deduced they were working on the same principal. Of course, they have perfect recall and can sometimes regurgitate the exact text they were trained on, which my brain can’t really do. But the concept is similar.
So I’ve never been too upset by the idea that LLMs are “stealing” existing works by being trained upon them – that’s exactly what humans have done for thousands of years. Look at every writer who writes about the authors that influenced them and you can see hints of those previous works in their work. That’s how creativity works. It’s changed, modified, improved, morphed, and combined to make something new, usually without being conscious of the process.
(I’m not convinced AI does anything truly creative, since it doesn’t know what it is doing – there is zero intention – but it is mimicking the human process of creating, for sure.)
This “brain training” is one of the reason I read a lot and try to read different genres of fiction and types of non-fiction. The more I read, the better writer I become.
(If I have a worry about AI, it’s that it is running out of training material. It’s already being trained on its own output, and as more “writing” in the world is AI-generated, the various models will consume that for training. This will water down the content the way photocopies of photocopies are further and further removed from the original. Like a game of telephone, the end result may be corrupted and completely distorted, taking away whatever humanity was in the original.)
As @Quantumpanda noted, I was just concerning myself with text, but the analogy does extend to other forms of perception, I think.
Well, models are intentional simplifications, so they always fail at some level.
Yes, the chatbots all have some level of “memory” these days, where they remember previous conversations. I’ve found that quite useful. Some of today’s snippet keepers are designed to let you “talk to” what you’ve snipped, and I’m sure that one day, our devices will remember everything we’ve read to help us pull more out. I suppose that will be moving us back toward the database analogy!
Ooo, was that a typo or did you intend to coin “comogulation”?
I made it up!
I don’t know what it is, but I liked it, the meaning seemed clear to me, and I figured I’d leave it as that’s exactly the kind of thing that AI would never do.
Lots of authors use made-up words* (especially in science fiction). That’s one way we get new words. Maybe it’ll catch on. I wondered if anyone would notice!
* I was just reading a bit by Cory Doctorow about using LLMs for proof-reading his work and he mentioned they don’t like all his made up words. I run into that, too.
I approve! Every now and then, a word begs to be coined.
My favorite made up word is the transmogrifier from Calvin and Hobbes:
https://calvinandhobbes.fandom.com/wiki/Transmogrifier_Gun
I have just read this article and initially took issue with the paragraph beginning “later in life” and then adjusted my opinion. I am 87 with a very cursory education, certainly no university but was an avid reader from a small child. More recently I have significantly changed my opinions on a wide range of issues. In the past this may not have been the case. The reason then came to me “life-changing experience.” My wife died last year. I hope this perhaps adds credence to your article.
Conversations like this are an important reason I hang around here, probably more so than just finding out how to fix my latest Apple mistake or learn whether the new latest Apple Whiz-bang is worth my consideration.
Bravo @ace for a brilliant article !.
I understand that you want to focus this discussion on the analogy, but the core issue isn’t only the accumulation of (weighted) knowledge —it’s how that knowledge is used.
As the foremost authority on the subject, Hugo Mercier’s central thesis—the Argumentative Theory of Reasoning—reverses the traditional intellectualist view that reason evolved to help individuals think better, make more logical decisions, or find objective truths.
Mercier demonstrates that knowledge is largely used to justify our pre-existing beliefs, to win arguments (“reason as a social tool”), rather than to achieve any kind objective understanding of reality or “truth”, which I find rather depressing.
In his book the Enigma of Reason, Mercier illustrates this concept with the “Lawyer” Metaphor: Reason acts less like a disinterested scientist and more like a lawyer. It seeks “reasons” to defend a client (our own beliefs) and to attack the opposition (others’ beliefs).
~The Argumentative Theory - Edge.org~
~Why do humans reason? Arguments for an argumentative theory~
i disagree with you!
Luckily we can think unlike the output generated from LLMs. But it might be better to think of our stored knowledge more along the lines of how it is stored in LLMs rather than databases and guess they learned a lot thanks to the study of how we think (both neural networks, but also in concept formation, logic and sentence construction) building the LLMs.
Anyway it is helpful in working with (or writing about) computers to have had some philosophy studies (like me too) – esp. in problem-solving.
I’m a cognitive scientist (academically at SFU) and agree that the filing cabinet metaphor is inapt.
I would say however that it’s still important to deliberately instill facts. Productive practice— a concept that combines literature on expertise/deliberate practice, memory retrieval/memory testing effects, and test enhanced learning — can be used to instill facts. When reading, watching or listening, one can look out for knowledge gems in the content, and create “knowledge instillers” for them in powerful flashcard software (like Anki or RemNote), and practice the content at spaced intervals. Research and experience are clear that without deliberate practice there’s a good chance the gems will be forgotten. And it is often valuable to be able to pull out the gems when needed. Also, one sometimes wants to deliberately ensure one can think not only about the information one has “consumed”, but with it. Compare:
Productive Practice: How to Make Information Actually Change You
He and his colleague Dan Sperber, also make the point that the mind is somewhat inert, resistant to change. This is why productive practice ( deliberately instilling knowledge) is important. Latter was interviewed by Julia Galef on the argumentative theory in 2015. Great interview:
well, it turns out that, as Seth Grant and others have documented, individual neurons are far more complex than those modeled in the current AI systems:
the analogy between LLMs and human mind-braind though interesting is weak.
There’s a flip side to this. A famous cognitive psychologist, John R. Anderson, published an important book The Adaptive Character of Thought (1990) in which he detailed what in my Cognitive Productivity book I called the heuristic relevance-signaling hypothesis. The idea is that evolution faced a major (implicit) challenge: how can one decide what to remember. We can’t simply tell ourselves “I will remember this”. The pre-frontal cortex does not directly control memory in that way. According to our hypothesis, the brain uses as a heuristic signal for deciding what to remember: what one tried to remember in a given day. The calculation happens during sleep, and involves indexing of memory. Simply trying to retrieve information can help you remember it in the future. That’s how productive practice (which I mentioned above) which leverages memory testing effects does its magic.
This is why students who use flashcards tend to do better. Flashcards are not just for rote memory. They serve many purposes. One of them is meta-cognitive: to signal to oneself what one does not know or understand , because one can’t answer the challenge (“question”). In my two Cognitive Productivity I dedicate several chapters to the topic. Flashcard software is gaining in popularity with students, as one would expect, but increasingly knowledge workers are using it for their own learning. I for one practice about 20 minutes a day, instilling all kinds of information. THis is also a way to take control of memory as one ages.
Aren’t people fascinating?
And such differences exist, the ability of memory is certainly not evenly distributed, just as the ability to think different is not (ahem). I have one son, a historian, who can retain vast swathes of information, truly an extraordinary capacity to unearth relevant obscure details that were digested months, even years ago, Another, a dreamer and an artist, pulls poetry and wonder out of seemingly nowhere, but can’t recall where he was on Tuesday.
I considered majoring in history (rather than psychology) as an undergraduate. One reason I chose psychology is that I didn’t think I could memorize all those dates. Then I realized psychology requires a ton of memorization too. I got by thanks to creating flashcards. That was before test-enhanced learning became a big area of study. I later generalized this into a practice I call productive practice:
The concept combines literature from expertise ( deliberate practice), memory retrieval effects, and test-enhanced learning. The goal is not merely to remember information, but to be able to apply the information when it is relevant. Other goals are to develop skills, habits and attitudes. So I depart far and wide from @ace amorphous memory concept.
The literature on expertise is clear that in many disciplines, particularly public performance disciplines, experts practice retrieving information. I’ve been on a mission to bring this to knowledge work. Through no cause of my own, an increasing number of people use Anki and Remnote more generally – but they are still a super slim minority.
Reading trains your internal LLM, so a trained LLM is comparable to a mind educated by reading. But what is an educated brain, or a trained LLM? What relation does educated thought, or a trained LLM, have to reality? What language can, and cannot, tell us was considered by Ludwig Wittgenstein.
The Illusion of Understanding. Wittgenstein, Neuroscience, and Why… | by Stephanie Shen | ILLUMINATION | Apr, 2026 | Medium
LLM computation automates creation of logical-linguistic models of reality, for “reality” as has been described in language. What LLMs do not, and cannot, do is extend linguistic description to as yet undescribed aspects of reality. Such extension requires “intuitive perception” of novel concepts, but what intuition is is not well known. Perhaps examination of the difference between problem solving by LLMs, and problem solving by humans, can better reveal intuition, allowing intuition to be better understood.
A guess is quantum entanglement, and the more applied problem solving by quantum computing, are related to intuition. Conversely, intuition might eventually be understood to be underpinned by quantum entanglement (a mechanistic model of a biological process would help keep reductionists happy). Hypothesizing such a relationship could allow design of experiments testing for the more tractable intuition, which might reveal the more elusive entanglement in their structure. These lines of thought could be extended in many diverse ways, and hopefully will by generations of graduate students creating more knowledge Piled Higher and Deeper.
I think the field in question matters a great deal here. In my professional field, the main constant is change—there’s alway something new, so I have to be taking in vast amounts of new information at all times rather than absorbing a relatively static body of information to become expert. I’m constantly developing and refining a gestalt, rather than being able to recall specifics.
I’d argue that what most of think of as being “informed” falls into this category. I don’t need to know huge amounts about even topics that interest me, like EVs, and certainly not about topics that feel important but are outside of my sphere, such as international events.
I agree. but some fields are universal, such as the ones I used as running examples throughout Cognitive Productivity: Using Knowledge to Become Profoundly Effective , including how to navigate relationships (I used examples of learning from John Gottman’s famous books: The Seven Principles for Making Marriage Work and The Relationship Cure. Educational psychology is clear that merely reading the books will not as reliably instill the knowledge as practicing. That’s not just for rote learning but instilling the concepts, such as his concept of bid/bid response and harsh startups. (there are more examples in my book, and examples of questions that can be used to master the information). More generally, for everyone: some information is worth mastering. (Divorce rates are high largely because people don’t bother to acquire the skills to make a marriage work.)
The couple that does flash cards together, stays together.
John Gottman was at the University of Washington, so we were aware of his work while we lived in Seattle.
I know it appears counter-intuitive. There’s a common view that test-enhanced learning is rote learning. But the data shows otherwise. E.g., test-enhanced learning provides feedback of what one doesn’t know. It overcomes illusions of learning which are alas the rule in learning. I explain both in detail in chapter 3 of Cognitive Productivity: Using Knowledge to Become Profoundly Effective. There’s also chapters 7, 12-13. sorry to plug the book but it’s not a subject that is easy to summarize. I also wrote: Productive Practice: How to Make Helpful Information Actually Change You but that dooesn’t cover it all. There is also this about the concept of bid and how we respond to them, which is essential in relationships:
From Reading to Changing How We Perceive, Think, Feel and Act – CogZest
Productive practice also makes one articulate knowledge and create mnemonics. Here’s an excerpt from Cognitive Productivity:
Also from cognitive psychology we know that learning lists is inherently difficult (due to “cue overload” aka “the fan effect”). So we need to break it down. But learning the list is just a step. The key is to think in terms of the content: Have I been clear? have I been polite? etc. Reading one of Gottman’s book will not convert into expertise with the list. One has to rehearse reviewing our experience in relation to the content.
And there are a few other key lists, e.g., in The Relationship Cure: A 5 Step Guide to Strengthening Your Marriage, Family, and Friendships.
Gottman realizes that practice is important, that’s why he gives workshops. But workshops are not always enough for the knowledge to become self-sustaining. The beauty of productive practice is that it keeps knowledge alive. Having said that, it doesn’t take an infinite amount of representation. The spaced learning (a very robust paradigm in cognitive psychology) software is optimized to minimize effort over time.
This is evident from the literature on many domains of expertise: experts don’t just get on stage and perform. They practice. Not all knowledge is worth mastering. But enough is to warrant regular practice in my opinion.
There’s also aging effects: learning becomes more difficult with age, limiting the potential of one-shot learning.
Thank you very much. Strange that we don’t hear from Julia Galef any more. Sorry for the delayed response