AI–Human Perception Mismatch

“I wonder if it'll be, but then at that point I'm sort of working against what everyone thinks the efficiencies of the LLM are. Now I'm sort of doing more machine learning in some way, and I'm still relying upon the LLM's ability to interpret that into an interface that it cannot see. This is the thing that most people don't realize, is that LLMs, when you say, "Hey, go look at this website," it doesn't read an interface the same way a human being does. This comes across oftentimes when you say, "Hey, how well is this page designed according to this criteria?" and it will be like, "Check, check, check, check, check." Then you go and you look at the interface and you're like, you understand that the LLM doesn't digest or read an interface with the same variability or weaknesses as a human. So the issues of prioritization and visibility and obscurity, like, "Well, it exists on the page." You're like, "Well, but does it exist in the right spot on the page in a way that someone's going to see it, or in the way that human beings associate proximity with relative relationships," all these different factors that go into how humans actually do it. Now, if you're asking an LLM to create an interface that another machine can read, yeah, it's going to kick ass doing that. That's why I think everyone is very excited about agentic, where agents are going to be talking to agents, and humans are going to be taken out of that, except as a completely confirmation type of role. But in terms of interfaces that people use, it's important to understand that human beings digest information very differently, and LLMs do a very good job of obscuring that fact. When you ask them, you and I, I've been going back and forth with Claude about this and I've been like, "All the ways in which you perceive an interface are very different from the ways that I perceive an interface, and there are, there seem to be, hidden obstacles in that process." It will tell you, "Yes, you're absolutely right. I'm looking at in terms of a markdown. I'm looking at in terms of, what if there's a schema behind it." All these things that are completely invisible to the user, and I'm making value judgments based on that. Now, from a research standpoint, does that hurt it when it comes to a design standpoint? Probably not so much. But again, what it's going to do is, it's going to look at examples out there that it thinks, "Hey, everyone's doing it this way. This is the right answer.”