Augmentation Not Replacement

“I haven't completely offloaded any tasks for it. I pretty much just use it to augment what I do.”

“Use it as a learning tool and not a do it for me tool. If I can tell that you've clearly just thrown it in there and said, "Do this for me," then you kind of lose that personal credibility in my eyes.”

“The way I describe it is that for a research activity that would take a researcher alone five days to complete, if you look at it with AI alone, it might take a day, but in order to do a good job of it, the necessary human AI interaction, you might get closer to three days.”

“I've been looking out for, I guess, the effect that people discuss about how it kind of detracts from your own thinking or your own creativity to rely on AI to produce outcomes. And I don't think that's happened to me. And I think if it indeed has not happened, I think it may have something to do with how I see the AI interactions.”

“It's not sort of the replacement, but as an assistant, kind of a sounding board, if you will.”

“In a way, we've created an information environment where we need it. We need AI. And I really think that that's the biggest promise of it, is to stop using it as a potential replacement for humans and use it as a way for us to manage this infosphere that we've built ourselves.”

“I really don't trust a synthesis or analysis and synthesis done entirely by AI. But even with the overview, I think it misses a lot of important stuff that's between the lines. And by the way, it's not just that. I think it's part of the process for us as researchers to immerse ourselves in the data. If we skip that, we don't understand the data afterwards, and we are not retaining that important knowledge that is kind of layering in the back of your mind.”

“The problem is that that kind of analysis gives you tunnel vision. So you don't get the context in which that is said. You don't get if they said it before or after something else. Because the moment you code in a sequence, you follow the conversation, you follow the flow. There is some logic behind it.”

“So I don't think it impacts the way I'm thinking. It's just helping me work with my thinking, or articulate what I'm thinking, or challenging what I'm thinking. So in that sense, it's not the thinking, it's probably the how I work that changes.”

“it's like working with a partner because it gives me ideas that I couldn't really figure on my own, different insights, but of course I have to double check everything. So that's actually a good example of guardrails. Before, I used the same tool to create personas for users of a data cataloging tool that we're looking to buy. So I gave all the interviews to the AI and said, create these personas, and it created five great personas, but it wasn't based on the data, it was based on general knowledge.”

“I'm going to run it through AI first and see if it comes up with some starter ideas instead of me doing a whole exploration. And it came up with something that I thought was pretty neat, and I just rebuilt it in Illustrator and gave it more depth and just more human touch, if you will.”

“that's not something that I can go in and type to Copilot, "Find me a molecule to replace." You have to use it as a tool that augments what you do.”

“Everything we're doing is building the plane as it flies. And I have work in Figma which hasn't been fully translated into our product. And so that work is still there to actually do those refinements, and even to truly implement the designs from Figma into code while we're still building out new features and whatnot. And so sometimes I'm using it to do the work that a front-end engineer might do to clean up our implementation.”

“So we are using it a little bit for that too. But it hasn't gone back into Figma yet. I feel like that's still a work in progress. And then there are times like when I was taking that LLM kind of experience, the chat-based interface project, and I spent just a couple days just working on that and I was really designing as I was building it because I had my engineer's work to start from so I was refining their work, I was cleaning up what they had done. But there would be times when I'd give maybe a general prompt and the output, maybe 50% of the output worked and 50% didn't. So, I say, "Oh, that's a good idea. We'll keep that, but then change these five things." And it's just kind of like an iterative building process. There have been other times where I'm like, "Okay, I'm going to try and use Figma Make because I haven't used it very much" and I'll give it an idea that I'm working on and the output just took a while and it's not helpful at all.”

“So it would need access to GCP and Azure. It would need access to our Bamboo and the other tools that are in the infrastructure that are needed for this production so that it could do the full cycle. So right now I'm the monkey in the middle. I tell it what to do in the tangible code. It does it. Now I have to run the test, take the logs from it, and feed it back and say check the logs, did it work or did it not? So I'm really in the way of the velocity.”

“I think it can expand on ideas that are already generated, but it still needs people to feed it. We still need people to come up with new things, right, to feed the tools.”

“Yeah, I think that's my biggest concern is, when is this going to make us irrelevant? I don't think so, because there's still something to say about best practices, and understanding the problem fully, and being able to ask questions on the fly. Could someone use a tool to do that? Maybe. But it's like the memes that you see out there, like the rules. It's like, you still have to understand all the complex problems that a business customer needs has, right? Or understand the technical architecture. I could see part of someone's job putting in all of the design criteria, but then I think it's going to fall flat because people aren't going to actually do that. They're just going to say, "Well, I'm just going to design it the way that I want to.”

“I used to be really resistant to it. I feel like, used in the right way, it's a benefit. I think it can expand on ideas that are already generated, but it still needs people to feed it. And I mean, I still don't know all of the ins and outs of it.”

“Like, I'm not going to pretend to know, but it can expand on ideas that are already generated. But we still need people to come up with new things, right, to feed the tools. What really scares me though is the ability to create images with AI and not knowing if they're real.”

“That's one of the other things, AI is a very good copycat. It's very good at giving you something that you think, based on what everyone else is doing. But if the thing that you're doing is not something that really exists elsewhere or is sufficiently different, then, it can still be a very good tactical execution tool. You can give it a task saying, "Okay, I need you to come up with some ideas about that." But it's not like you can just say, "Here's a marketing website," and what it's going to do is generate that. So I don't know if we've had any real failures with it yet. I think we're still trying to figure out exactly the best ways to use it in a way that still leverages what we are good at as an agency, and promotes that difference to our clients to make ourselves still relevant and useful. I do think research is one of those, honestly. Research is one of those things that goes in and out of favor a lot.”

“Yes, and the ability to, I mean, one of the nice things about it is that we've always had this qual, right? With this, potentially there's the idea that you can scale qual a little bit more, and still make it so, instead of doing 20 sessions, you can do 50 sessions. The key is not just to have a transcript and dump it in there and be like, "Tell me what it says." The key is being able to ask very, very cogent questions in that, and then be able to do pattern analysis across it. But it's going to come down to the quality of the questions, because I've been doing a lot of comparison of, "Well, what would be the things I recommend about changing on a website versus what AI is telling me?" There is definitely some overlap for sure, but the areas of overlap is, AI is good at what is best practices it's found someplace else.”

“Yeah. And I think, the augmentation, the one thing about AI, it's very good at looking like it has really good specific and general knowledge, right? But it doesn't. Like you said, it's a probabilistic engine that is looking for what is most likely the next right word to put together. It's very good at seeming like intelligence, but really at the end of the day it's not. All it's giving you is, "This is what basically everyone else is saying." If you have tasks which are sort of wisdoms-of-the-crowds-oriented, then it works really well. But if you have tasks that are not based on that, that are more individualistic or unique, then it's not very good at that. The question is, a lot of people don't understand the difference between those two things.”

“Theoretically as a good design strategist, that opens up doors for me to actually do execution that I wouldn't be able to do normally, Like being able to connect a prototype to Supabase, which three years ago we could have, but it would have been a lot of diving deep into the... A lot of back and forth, and involving a lot more people. Whereas now I can take the concept much further into that process. It's interesting because we've been having conversations like, we're back to the whole sort of unicorn thing, right? Every time a new technology comes out, everyone's like, "Oh well, it's just going to take one guy, and my company's looking for the person who can do it from stakeholder interviews, through research, through design, all the way to coding, and then just hand it off to like a DevOps person who's just going to make it work." That can work, theoretically, but I think the other side of the coin, that people tend to not really appreciate, is that a large part of the process that we do is not just executing a design but socializing designs within organizations for the purpose of decision-making and collaboration. That process, as much as you automate it, or make it AI assisted, it still requires a certain skill set that is more human-based than anything else. A big part of my job is, working with the client, "Okay, so what are we trying to achieve out of this?" If we do some research, being able to say what are the most important or germane parts of that research, and how do we flow that into design? And then, "Okay, here's some overall concepts.”

“But when I was doing it, as someone who was very excited that I could actually put together a live prototype, it wasn't so much that I was replacing our information designer or our visual designer. It was the fact that the things that I needed this interface to be able to do were very hard to communicate in terms of requirements or in terms of business logic, or anything like that. You were able to visualize that very easily, and to make this artifact something that incorporated that information in a way that was much easier to understand. So you don't have the risk of a designer not reading a note about how something should work, because it actually works that way. One of the things that we talked about was, push, as we're doing all these different things, that becomes the artifact that we push through the pipeline. We can do a very high-level diagram of what it looks like, and then we can visualize that in a functional prototype in one way or another. But then the information designer could go in and actually change all the screens, and even the functionality there, but still understand what it's supposed to do. And then we pass it off to visual designer. For me, this prototype as being a warehouse of project knowledge was excellent. We toyed with the idea of annotating it, just having it so, like, as a rule, it's like throwing up the business logic so that anyone can see it as it's happening, and to bring that all back to the idea is that, I never viewed this as an opportunity to supplant the other people in my team or doing that process, but it allowed me to get them an artifact that was much richer and much more effective, and potentially to share that with our clients. Because clients notoriously have a very hard time, you know, you can show them 10 different screens and walk them through how they're all going to work together, but they're not professionals. They don't see it that way. So there's always a risk there, that their assumptions are coming into play, or their lack of ability to understand what you're actually telling them, and what they're agreeing to. So doing that in that way, I did view it as a way of saying, it's a way for me to extend, not my skill set per se, but to visualize what I do in a way that is more effective. I don't know whether everyone else is going to see it that way, whether, or even whether a client will see it that way, but I view it that way. I am not a visual designer. I'm a pretty good information designer. But my value to our projects is not that. My value is being able to look at the whole thing and say, "Is it really doing what we needed to do? Here's an area, or here's a way that we might approach that type of problem," and letting other people solve those problems.”

“Oh god. There's a lot of things that I say I'm going to continue doing the hard way and then I don't, because I'm short on time. I really try to continue designing, because I just redid our research integrity websites and I really try to continue brainstorming design on pen and paper. Like, starting my initial outline on pen and paper, and then maybe having a discussion with ChatGPT about how the information should go in. And even then, I still feel a little weird, because I don't want it to weaken that creativity or those skill sets. I'm sure there's other things too, but time management really is the key, and that forces me to continue using it.”