Vibe Code Governance

“Yeah, that is also happening and I haven't talked about that yet. I mean this is almost like very detailed product documentation but it's also that we have sort of a process of them going through AI to come up with that and then come up with the technical details to start implementing it. Also they are vibe coding some of those interfaces and sharing them with me and the team, and that's been its own interesting challenge. How so? Tell me more about that. So I think there are a few aspects of it. In [our customers' industry] I think the bar in terms of end UI design is not always terribly high and so when someone vibe codes a design, puts it out there like "oh we're done, look, so and so did it, it's there," and I start looking at it and there are some things on the surface that are fine and they're working and maybe there are a few good ideas that I haven't thought of too.”

“But then you start to peel away the surface and there's so much that doesn't make sense in terms of what we're doing and the layout in addition to just like maybe the design system we're using. It doesn't map to the design system we've already established. And so there's all those aspects of it, but then there's even translating the domain and the intent into the interface that I never, like sometimes I'll just, in the past maybe I'll get handed one of these live coded interfaces without that context and I'll have to go back, either I'll have to do my best to extract that intent out of the interface or I'll have to go back and ask 20 questions just to figure out what was going on. And so this is something I sort of have unsuccessfully proposed which is that we do a better job of documenting our intent if anybody's going to be vibe coding interfaces and put some structure to that so that we can say, okay, so and so made an example of this application, what were you thinking, what were you hoping to accomplish, and with the idea that maybe if that was documented we could assess it together and see whether it was working.”

“It can be great as a conversation tool or great from a design tool of iterating quickly, but I still think we need to be cautious because the same way that we would want to only show grayscale designs at the beginning of an ideation session, you don't want to show a full clickable prototype either.”

“But from design it's like they're all about using AI, but it's almost scary because I have some teams that work with business that are like, "well, here, I just made this in Figma Make. Can you just make this?" And there has to be some boundaries with certain product teams because that's not necessarily something that we would want to make. But yeah, I mean, from a measuring perspective, I'm sure there are ways that they're measuring it. I don't know what they are, unfortunately.”

“Yeah. So, luckily it hasn't been with my direct team. It's been with a co-worker. Their product owners have been using either Figma Make or, I don't know if they've been able to use Miro.”

“But those are the two programs that we use most, and they will come up with a design and essentially it's like, "well, why does it take you so much longer to do what I could do here in five minutes?" So I think it's a balance of using it as a communication, like gently saying "stay in your lane" but also using it as a communication tool of, "okay, well, tell me more about why you think this is the best solution and maybe there's gaps in our communication." Because we also have to. I think there are integrations that they can use, like our own design system. Because just because Figma Make comes up with a design doesn't mean that it is in compliance with our design guidelines. So keeping that in perspective: there's this big project that is being done at the very high leadership level, that someone put together an idea in Figma Make and the development team and leadership thought, like, "okay this is ready to go and ready to be built." And it was really just a tool to explain what could be done.”

“So that's kind of the other aspect of design, is using it and sharing it with business and they're like, "okay, well, let's go build it." And it's like, well, this is just a tool, this is a starting point, let's build on it, like, understand where are the data points and what. This is not actually built in a true design fashion. We haven't figured out dependencies or anything like that. So I see it on both ends. It can be great as a conversation tool or great from a design tool of iterating quickly, but I still think we need to be cautious because the same way that we would want to only show grayscale designs at the beginning of an ideation session, you don't want to show a full clickable prototype either.”

“With the project that we did, one of the things that we found doing this sort of vibe coding that was going on, is, actually, it was still super useful to do high-level flows and functionality, because once you, if you just try to jump in and start coding it, the complexity gets to you really quickly. You can recover from it, sure. It's not like you're going to go, "Oh my God, I can't do it." But it makes a lot more sense to think through the entire experience of what you're trying to do for it, and then to use AI to execute either on a screen level or on a functionality level.”

“Yes. The LLMs suffer from the same problem on a coding level, is that it starts to iterate out code based on assumptions that were created earlier in the process, before you actually were able to describe the overall experience. So now it's working backwards and trying to figure it and fix it... Yes, you can go back to Claude Code and like, "Okay, so now this thing is done. I want you to go through and I want you to optimize all the code, I want you to optimize the database, I want you to look at it for security holes." It can do that, but the question is, is your prototype full enough now? It's gone down these, you know, it's connected 17 different rabbit holes. Can it still step back and say, "Okay, this is how we should think about this holistically, not just from a stylesheet level or anything like that, but from a sort of overall functional level?”