Modern Web Development Is Fundamentally Broken

No one cares about building quality products anymore

I’m glad I got laid off from my job recently. I’ve been a web developer for over twenty years and I’ve watched this industry change in ways that simply make no sense and I can’t get behind.

This is a mostly un-edited free-write. I may come back to turn these into proper articles.

This is going to sound like the ravings of a lunatic conspiracy theorist, but I believe that a huge percentage of decisions made in corporate web development have nothing to do with building a quality product or providing a better user experience. Year after year, I see developers caring less about the real-world user experience. Instead, the industry now focuses on unit test coverage and the holy-grail of a “pure” coding architecture.

Users are suffering from this because we are delivering buggy products that are increasingly disconnected from the user’s real needs. Developers are suffering because the art is being stripped from the process and we are left feeling like button-pushers.

Needless Complexity:

I think the real reason we have automated testing and complex state management systems like NgRx is it makes us feel like we’re part of a club of smart people. And for the job security.

We are lying to ourselves. I use myself as an example: I spent years going along with this, pretending this all made sense to me. I lied for years about who I was and what I believed as a software developer, because I figured no one would want an employee who fundamentally disagrees with practically every decision you make. So for the last decade I settled for being a highly paid but mindless code monkey, doing things I fundamentally disagree with because the guy who signs the paychecks told me to, never telling them I think they’re driving their company into the ground.

I was once asked to refactor some component code into NgRx state management. It took a couple days even with the help of AI, and in the end, we had the exact same functionality, except now the thirty lines in one component file was spread across four or five files totalling three hundred lines of code.

I mentioned this to another developer and he said, “Oh yeah, but the component is easier to reason about now.”

And I just let it drop, but I wanted to ask: “Were you struggling to reason about the component? I found those thirty lines pretty straightforward.”

And he would have said something about the importance of getting business logic out of the component… because… you know… some smart software guy working in a completely different situation once said we must have separation of concerns.

Then you have the design side. Same deal. Massive amounts of unnecessary process and complexity. Companies are trying to roll their own proprietary design systems, using dozens of six-figure developers, before the company is profitable, when there are countless pre-built systems ready to go.

Accessibility is definitely important, but that’s even more reason to go with pre-built systems that already solve those problems. In the beginning it’s actually a good thing if your interface looks like a lot of other interfaces.

Take Craigslist. They have roughly $20 million annual revenue per employee, and they haven’t changed their interface in decades and put virtually nothing into design.

And no one cares. No one goes to a website for its design. They go to accomplish a task.

Automated Testing:

Around 2010, when unit testing was the shiny new toy, I immediately got on board and suggested the idea to my boss, getting him on board.

My partner in the project, however, fought back. “It’s a bunch of extra work. It won’t provide any real value.”

We argued for a while and good points were made on both sides, but in the end, I won. We started unit testing our new video player.

“Fine,” he said. “But do me one favor: Keep a list of every bug these tests find. I want data to prove these things provide real value.”

“Okay,” I said. “Real world data is always the way to go.”

Then a couple months later, after putting in several hours a week into writing and maintaining unit tests, he asked, “So how’s that spreadsheet coming? How many bugs have we found?”

And I had to admit I still hadn’t made my first entry.

Months went by of consistent unit test work. He would occasionally ask for an update on my spreadsheet. “If I took all the hours I’ve spent working on unit tests and used those to manually test the site, how many bugs do you think I’d find?” he asked.

Then one day he came to me complaining that a unit test was failing and he couldn’t figure out why. He’d already spent an entire day on it. I looked at it and couldn’t figure it out. Finally, we both sat down at his computer to pair-program. We spent four hours working together on this one unit test, and finally pinned it down to a coding inconsistency we had overlooked.

“Hey look at that, it actually found something. You finally get your first entry.”

But I replied, “I don’t know if that counts. Was that a real bug? We just spent two full days to fix a minor inconsistency that never would have actually affected any user.”

“Yeah, I was gonna say that, but I didn’t want to pour salt on the wound,” he said. “You should go ahead and count it.”

And I’ve kept on counting, through all my jobs, and countless hours fighting with unit tests and other automated testing, and that list is still only one item long. I’ve literally never once seen a unit test illuminate a bug we wouldn’t have otherwise found before going to production.

But they make us feel good.

Look boss, 100% coverage! That means product is perfect. All the tests passed. That means I didn’t break anything… but if I did break something, well, that’s someone else’s fault for not writing good tests.

Yes, I’m sure there are exceptions, some cases, perhaps where user’s lives are on the line, then there is a case for automated testing, but in my experience as a common web application developer, automated testing is security theatre that causes far more harm than good.

I’ve heard the phrase from multiple developers at multiple companies: “Humans should not be testing software. Software should be testing software.”

This statement is so mind-boggling to me. At some point we’re gonna be so separated from the end-user that we don’t have any idea what product we’re even working on.

Imagine buying a car and the dealer says, “No test drives for you. We already had a robot drive it and everything checked out.”

The only true test of a product’s quality is the end-user experience. At the end of the day, that is all that matters. That is all we should care about, at least as developers.

Once we developers accomplish that, a small marketing team should have no problem turning the product into profit. Only the marketing team should be thinking about profit and engagement. Everyone else should be thinking only about the quality and security of the final product.

Manual Testing

Manual testing is still the best way to ensure quality and the best way to understand the user experience. We are lying to ourselves when we claim automated testing is better than manual. We can clearly look at the amount of hours spent on each and count up the total number of bugs found.

But manual testing doesn’t need to take up a highly paid developer’s time.

My first job had a quality assurance team consisting of two college girls who had never used the internet. They were paid nine dollars an hour, and they are to this day, the best quality assurance team I have ever worked with in my entire career. A lot of that is because management let us run things how we wanted instead of forcing us through layer upon layer of process, so we could interact with QA the way we felt was right instead of what the process dictated.

But beyond that, those two girls accurately matched our end users better than any other QA team I’ve ever worked with. They were people who might have actually used the product, and were coming at it from a perspective of ignorance, which is exactly where our users are coming from. Every other QA team I’ve ever worked with has been filled with six-figure professionals who have a deep understanding of how the web works behind the scenes.

The more technical understanding they have, the less they understand the user experience.

Most of what QA actually does could be done better with hourly, uneducated workers. Some of what developers do could be offloaded to them as well.

Code Comments

There’s a weird trend in software to avoid commenting our code. There’s a stigma about it, and some actually say “If you need to comment your code, you didn’t design it right.”

This is another situation I believe our emotions are clouding our judgement about what’s best for the product. Comments mean being critical of our code. It means openly admitting when we’re using a hack. It means putting your ideas out there on the table for everyone to question. That’s not comfortable. That’s the real reason we avoid commenting.

I’ve done this too. My last job I rarely commented my code, despite it being a core aspect of how I look at programming. The social pressure of being in a team of 100 highly skilled developers, none of whom comment, well, I didn’t have the self-confidence to stick out.

But when I have control over coding standards, I require nearly every aspect of the code to be represented in comments. So unless the usage and reason for a function is painfully obvious to a beginner, it needs to be commented. Every time you realize there’s a more “proper” way to do something, you comment it, admitting you are not following standards, and explaining why.

The “why” of nearly everything should be clearly commented. Any concerns that come up in discussions, disagreements, alternative options etc. All that should be in code comments. If the text becomes too bulky, create a .md file and include it in the most appropriate directory. Just remember to create clear standards for those .md files.

The “how” of things is not as important but still valid. Proper naming conventions and code structure needs to come through. Your code still needs to be totally understandable as if there were no comments. Long variable names are fine but you need to follow strong, well thought out naming conventions. And document them in comments.

But even with the “how” of things, there is frequently times when even that should be documented. If the next developer reads a thirty second comment, even if that function is very well formed, that comment still might save the next developer a couple minutes. It also helps verify.

But one of the most important aspects of commenting is your own understanding. I can’t even count the number of times I’ve forced myself to explain both the how and the why of a function, finding the act of writing illuminates oversites and edge cases I wouldn’t have thought of when looking at code alone.

The experience that sealed this belief for me happened at a huge bank where I was leading a project to build a set of data visualizations for the executives. We only had a few dozen internal users, but they were using my visualizations to make billion dollar decisions.

So I had implemented a standard of extensive commenting and inline documentation across our app. It had been this way for a couple years when they brought in an intern fresh out of college to add to my tiny team. 

They called me into a meeting for introductions.

“You’re going to need to coach this guy,” my boss said. “I know you have a lot on your plate but for the next couple weeks I need you to sit with him and get him up to speed.”

My heart sank. “Have you looked at the code?” I asked. I had been deep in a coding rabbit-hole and did not want to break my momentum. 

“I opened up the repository,” he said, fear in his voice. “This is like, ten times bigger than anything they ever showed us in college. I have to warn you, I don’t even know where to begin. I’m gonna need a lot of help.”

So I told him exactly which file to start with and which ticket was next in the queue. “Go into that file and start reading the comments. See how much you can figure out, then if anything takes more than five minutes, then come get me and I’ll walk you through it.”

Then I went back to my desk and promptly forgot about the kid and my boss’s expectation that I focus on coaching him.

Three days later he casually strolled past my desk and I suddenly remembered. I flagged him down. “I’m so sorry I forgot about you,” I said. “Do you still need help?”

His face lit up, “No!” he said with clear joy. “I don’t. I started looking at the code like you said and I read the comments, and it’s like, every time I have a question about anything I just read the comments and it either answers my question or leads me to some other function that does. I already finished the first ticket. I’m learning things faster than I ever did in college. They never bothered to tell us commenting makes everything easier. So no. I got no problems. I can handle this. I’m moving to another team in a few weeks and I’m sure I’ll have no problems there either just so long as they’re commenting the way this team is. I didn’t realize this was the standard out here in the real world.”

Then I broke his heart when I told him it was only on my team.

Needlesss Process

Every individual piece of every feature is burdened with layer upon layer of processes, ticketing, pointing, and reviews. It’s like if you were a contractor assigned to remodel a kitchen but you need an individual set of reviews, mockups, time estimates and a full description of the work for pulling out the old sink, one for putting in the new sink, one for sealing around the sink, to the point where you’re spending more time dealing with paperwork than you are actually remodeling. Every time you finish even the tiniest task, you then need to stop, manage a pull request, put together a description.

Managers keep saying “I don’t want to micromanage you”, but never acknowledge that micromanagement is fundamentally baked into the system.

Developers Just Wanna Have Fun

Developers love technology and code. It’s like a giant artistic puzzle you build yourself. Coding is actually a deeply emotional experience. I think people don’t realize that. We can be brought to ecstasy when we finally figure out a solution or see those metrics soar after our latest release.

But a bad situation can lead us to equal lows. Making a mistake makes us feel like shit, even if it didn’t impact a user.

Those emotions drive our product design more than anyone realizes.

You get the shiny new toy mentality, where developers see something new and immediately want to apply it to their current project without worrying about the cost of changing everything or if this new technology actually adds any benefit for the user. Managers don’t know any better, or they just go along with the flow because in a large engineering team, numbers equal safety. If you have a lot of developers all agreeing it’s a good idea, no one can blame you if it goes south. And if development grinds to a crawl, well, if the company has the money, that just means longer employment.

So developers are not going to choose the process that is best for the real goals of the company and the user. They’re going to go with the technology that makes them feel good, whether that emotion is about safety, looking smart, job security, being part of a club, or the novelty of trying something new.

Modern web development has virtually no safeguards to ensure these emotions do not interfere with the decision making process. No one is incentivized to call out irrational decisions based on emotion.

Like everything in life, our emotions dictate our decisions, then we invent logical reasons to justify them after the fact. Identifying and addressing this is critical.

But developers also create better products when they enjoy what they’re doing.

Modern web development is soul crushing, even when everyone is kind and wonderful. We need to bring back the art and passion.

Company Lost its purpose

On the leadership level, they talk about user engagement, not about adding value. More engagement equals more customers. They don’t look deeper than that, so they’re always just looking at data, trying to find patterns and correlations, like they’re a glorified matching algorithm. They have all the data about what people are doing on the website, but they don’t dig into what is really going on, the deeper problems their app is solving or could be solving, and the real emotional experience of the user.

In developer discussions, user experience is an afterthought. We pretend we care, but ultimately the code is our baby: that beautiful, standard-compliant, “fully-tested” code. That’s what we truly care about.

Related posts

Leave a Comment