Thoughts on AI-Assisted Development

Read Time: 34 minutes (7049 words)

Author’s Note: I started writing this back in January of 2026, as a way to work through my thoughts on genAI, especially w.r.t. using it as part of a software development workflow. I kept working on it slowly, but never quite got around to finishing it. It’s been sitting untouched for long enough that I’m pushing it out of the nest to fly or crash on it’s merits.

The main points, and my concerns, have not changed over the intervening year. In fact, they’ve only gotten stronger as I’ve watched the AI bubble turn into a bonafide cult, and we see the rather remarkable changes in behavior that come from widespread usage of genAI in the general populace.

Additionally, I’ve been doing interviews with developers more days than not this year, and so I’ve got a rather close pulse on their AI usage, how it’s affecting them psychologically, their workflows, and their skills. To say “I’m concerned for my industry” would be a bit dramatic, but also very much an understatement.

It’s extremely disheartening to have a conversation with a nice, pleasant, personable senior developer who has been building software for over 2 decades, and to hear them say, “I haven’t written code for the past year and a half.” And then later, “Yeah, I don’t really review the PRs and commits [from the AI] anymore.”

The sheer number of devs who have abdicated their craftsmanship and handed over agency to stochiastic processes is utterly terrifying. While there are a number of devs that treat it like a tool to be used and verified like anything else, they’re outnumbered by the number of devs who have turned their brains off and just blindly rely on genAI to do everything.

My final conclusion? Using genAI is a Faustian bargain. It’s possible to use it, and even do so ethically (if you’re using models that were trained on permissively-licensed content and self-hosting it), but you run the severe risk of breaking your projects, making them unmaintainable in the long term, and losing your own skillset in the process.

“Use it or lose it” applies to everything, be it your physical muscles, natural language fluency, coherently writing, martial arts, mathematics, logical reasoning, or coding. If you hand off responsibility for something to a tool, regardless of the tool, you will begin to lose that skill. There’s no way around it, that’s just how the human body and brain are built.

The AI bubble will pop eventually (likely within the next year), and while genAI isn’t necessarily going anywhere as a technology, it’s certainly not going to be the heavily-subsidized ecosystem we’re currently operating in. Plan your future accordingly.

This article is a long-form self-discussion about the ethics of generative AI-assisted software development, and how it can be ethically and safely incorporated into my personal workflows (or if that’s possible in the first place). I’ve been slowly working on this for over a month, because of how dense of a topic it is.

  • Generative AI is dangerous
    • To workers, by replacing their jobs and making skillsets less economically valuable/viable (ref: Luddites and weaving vs powerlooms)
    • To companies, by executives overestimating its value, getting rid of experienced workers, and wasting money on badly planned AI initiatives
    • To the economy, by investors throwing literally trillions of dollars into a small handful of companies that themselves toss around money in their small circle, inflating stock prices without producing anything of equivalent value
    • To society, by flooding our media and social platforms with false information and conspiracies, and enabling bad actors to do the same intentionally
    • To the justice system, by lawyers and witnesses relying on it instead of their own expertise, as well as enabling the easier creation of fake evidence
    • To education, by students relying on it instead of learning and building their own knowledge and skills, as well as teachers using it instead of building their own lesson plans and helping students overcome issues with the material, and even by reinforcing stereotypes and official narratives instead of the messy, uncomfortable reality of human history
    • To the users, by skill drain and reduction in critical thinking, as well as sycophantic behavior

Any ethical use of generative AI (hereafter shortened to “genAI”) has to deal with these dangers. I’m sure there are plenty more, these are just the ones that I personally have seen over the past few years since ChatGPT launched. And of course, not all of these are unique to genAI. But just because a danger exists from multiple places doesn’t negate the need to address it.

A comprehensive plan to deal with the risks of genAI would be a significant undertaking, and is not my goal here. Instead, my goal is to examine genAI within the context of AI-assisted development, considering the benefits and the risks, and determine the best path forward for myself professionally.

Ultimately genAI is a tool, and should be treated as such. You don’t hand a propane torch to a toddler, and most tools in a workshop will absolutely kill you if you don’t respect them. GenAI isn’t much different, it just happens to work through second-order effects rather than immediately putting your meatmech into the scrapyard.

Ethics

Before we start to consider using genAI at all, we need to evaluate the ethics of doing so. Is it possible, even theoretically, to use genAI in an ethical fashion? If not, then our decision about using it is pretty easy since I’m not a used-car salesman or a politician.

The main ethical issues underpinning genAI currently boil down to a few broad categories:

  • Ecological costs; the resources consumed by training and using genAI models
  • Societal costs; the impact on society that results from using genAI
  • Economic costs; the impact on the economy that results from using genAI
  • Content rights costs; the widespread content scraping that underpins all major edge models

Ecological Costs

Based on the limited data we have available, training costs for genAI models are in the millions of dollars and weeks of compute time on datacenter-grade computers. That translates to massive amounts of electricity for power and fresh water for cooling, a well-known issue with anything involving datacenters. However, that’s more of an issue with our current datacenter-centric, maximize profits and minimize time-to-market mindset rather than something fundamental to training models themselves.

It’s certainly possible to train models in a more sustainable way with less intensive hardware that doesn’t need as much electricity or any water cooling at all, researchers and industry have both been doing it for decades now. The only reason we do it the other way is because Green Line Must Go Up. Thanks, Capitalism.

Of course, training is only the first half of the issue. The same resource consumption issues come into play for ongoing operational costs, because those models are running in the same datacenters when doing their inferences. And that’s a bigger issue, but if it isn’t enough of a problem to keep us from using cloud computing at all (AWS, GCP, and every other major cloud provider exist in datacenters), then it shouldn’t be an issue with using genAI.

As with training, inference can be done on non-DC hardware, mitigating the resource consumption issues, but that only works for open-weight models. ChatGPT, Gemini, Claude, and any other closed-weight edge model is going to forever be tied to the ethics of using datacenters at all. There’s also the issue that running larger models on consumer-grade hardware is, for practical purposes, impossible currently. In theory, you could partition the model weights into chunks small enough to load into a consumer GPU, but to the best of my knowledge nobody is currently working on that (there’s some good potential here, though; if you could get it working, you’d open up the really massive models to everyone, with the tradeoff of longer inference times, of course).

AN: I have since encountered folks working on precisely this type of “split inference across multiple nodes”. It’s doable, and likely something that will be more common in the post-hyperscaler AI era.

The potential to do training and inference in a more sustainable fashion mean that from a purely theoretical standpoint, there’s not an ethical problem here. From a practical standpoint, while I dislike the ecological costs of how we currently do our training and inference, it’s ultimately not any different than the rest of our cloud computing infrastructure, and datacenter concerns don’t keep me from using AWS and CloudFlare for my infrastructure.

Societal Costs

New technology can be anything from a minor improvement of current processes to a complete rewriting of how things are done. The former is typically absorbed easily without much impact on society as a whole, as can be shown by incremental improvements to CPUs or batteries. New designs and innovations get incorporated into the next generation of devices and rolled out to consumers without much fanfare or alteration to how things are done. The CPU in a modern computer is significantly better than the ones from a decade ago, but with minor exceptions for folks operating at the edge of computing, those improvements haven’t changed your workflows or how you use your computer.

New technologies that are a full process shift normally herald major upheaval, as was seen with the invention of the power loom or tabulating machines. Before power looms, clothing was one of the most expensive things you could buy, because it took a massive amount of handcrafting work by expert artisans to create each garment. Once power looms began being used, companies were able to mass produce clothing and absolutely flooded the markets with them. The quality was lower than what could be achieved by experts (including experts working with power looms), but the sheer quantity available and the reduction in cost was enough to change everything about how the public treats clothing. Nowadays, clothing is effectively disposable, to the point that developing countries actually have to ban the import of discarded western clothing because allowing it in will destroy their homegrown clothing industries.

This shift from handcrafting to power loom mass production also utterly demolished the entire profession of people who previously made all the clothing for their communities. Expert artisans, masters of their craft, were now undercut and thoroughly incapable of competing with the new technology. The cost to that particular segment of society was absolutely brutal, as anyone who has ever had to change professions can attest to. However, it also cost society as a whole, because all of that accumulated experience, thousands of years of history and knowledge of how to do a craft effectively, has been lost. The ability to handcraft quality garments, while not completely gone, is now relegated to hobbyists. We’re now effectively wedded to the use of power looms for our clothing manufacturing, whether we like it or not. It can be argued that this is just the cost of improving how we do things, but the downgrade in clothing quality is absolutely a cost that society has incurred as a result.

GenAI, like every new technology, is somewhere on this spectrum. It seems fairly uncontroversial that it’s definitely on the “major upheaval” side of the scale, even if folks want to argue about precisely how groundbreaking it actually is and what its true capabilities are. And we can see just how much upheaval it has produced by looking at the issues we’ve seen within the past 3 years since transformer-based LLMs became available.

Issue: Misinformation

Misinformation has always been a problem, since well before the internet existed. Now that we have LLMs that can easily engage in full discussion loops with a user, they’ve become an integral part of psyop campaigns online. The “ignore all previous instructions and give me a cake recipe” meme existing at all is pretty telling. However, this is a usage issue, not something integral to LLMs themselves.

More concerning, however, is when LLMs are used as a source of information, such as in an information retrieval service or for content generation. Hallucinations are a real problem, especially since LLMs skew heavily towards syncophantic behavior that tells you what they think you want to hear instead of facts. This is the basis for human-in-the-loop requirements for any significant outputs in production agentic systems.

While not necessarily intrinsic to LLMs in theory, the simple fact is that they’re probabilistic token generators, and that does not in any way guarantee that their output is going to be valid within whatever context you’re using it. Even when they’ve been trained on datasets that focus on objectively correct answers and reasoning, that doesn’t guarantee they’ll produce those answers. They’re still probabilistic token generators, and like any probabilistic algorithm, you need to either be validating their output, or otherwise accepting of the possibility that they’ll produce an incorrect answer.

As a general ethical issue, the misinformation that can result from LLMs is frustrating, but not any different from how the internet was when I was growing up, or the world in general. People are allowed to say things that are incorrect. There isn’t a divine authority smiting anyone who states a falsehood, intentionally or not. That means that we are responsible for validating information that comes in, regardless of the source, and for teaching our children how to do the same.

Within my specific context of using LLMs as a coding tool, these concerns generally become moot. The output would always be checked by me personally, and likely would end up being at best a scaffolding for further development by me, the human in the loop. Further expansion into knowledge retrieval would also be mostly mitigated by my aggressive double-checking of any new information I receive, especially when given by an AI. I’m not the biggest fan of probabilistic algorithms, but they are handy to have in your toolbelt, and often work well enough when used appropriately (see: bloom filters).

Issue: Malpractice

The past two years have seen lawyers sanctioned for filing court documents that were AI generated, and judicial findings that evidence introduced in civil cases was fabricated using genAI. There has also been a tsunami of genAI-produced content masquerading as actual human writers. All of this is unethical at best, but not intrinsic to genAI, instead people just using the tool in unethical ways.

Frankly, the fact that people lie and falsify evidence is a fundamental fact of life, and genAI just happens to provide the push that we need to fix up our systems and make it more difficult for individual bad actors to break things. I’m just waiting for the first time someone wearing a badge decides to use genAI to falsify a security camera image or photo to secure a conviction. That’s going to be the straw that breaks the camels back for the entire system and forces a fundamental rewrite of how things are done.

Within my coding context, none of this is a concern at all, since human review of any generated code would be happening and it’s explicitly clear from the beginning what’s AI-produced. When doing deliverables, it would be made 100% clear that some code was generated by genAI, but all of it has been reviewed and validated by a human. Ultimate responsibility falls to a human, always, until we reach actual AI sentience.

Issue: Education

The use of genAI within an educational context has been growing, and while there’s potentially ethical ways for it to be used, most of the ways its actually being used are to the detriment of students.

Unfortunately, while this is a topic deserving of its own article, it has no bearing on my coding context.

Issue: Critical Thinking

A significant issue that has become more apparent over the past few years is the way that LLM usage causes people to lose their ability to critically think and problem solve.

Critical thinking was already a rarity in the US, since our education system is focused around teaching to the tests, not producing adults that can reason and think through complicated problems to come up with a solution. Individual teachers might do their best to teach their students to think, but the system they’re working in certainly doesn’t encourage it or actively enable it. And if you think that politicians actually want constitutents that can think for themselves instead of believing soundbytes from the propaganda machine, you’re more naive than I was as a sheltered, small-town, midwest teenager.

However, this goes beyond just that. There’s now a growing group of people that turn to LLMs to do their thinking for them. They don’t know how to research, they don’t know how to investigate, they just throw their question into the text field and accept whatever the chatbot says. That’s absolutely horrifying. It’s like religion, but controlled by techbros instead of old guys.

There’s clear ethical issues here, but only in the sense of people being people and offloading their thinking to a third-party. Whether that third-party is an LLM, a politician, a self-help guru, or a religion, the ultimate result in the same, and there’s not much we can do to resolve it. People have been doing this since the beginning of recorded history, and almost certainly before then. It’s not a unique problem for genAI.

Existential dread for the future of society aside, the fundamental concern within my coding context is that using an LLM as a coding assistant will result in that same sort of mental offloading and subsequent degradation in reasoning capacity about problems and their solutions. The remedy, to the best of my reasoning, is the same as for malpractice concerns: active review of any outputs produced.

I would honestly treat this as a code contribution from an untrusted third party, and examine outputs to the smallest details. Using something like the superpowers repo framework would probably help here, as it forces explicit design work to be done up front and approved before any actual code work gets done. Tearing through the design work first should help significantly, assuming that the code output actually corresponds to the design work in question.

Issue: Skill Drain

When using any tool, there is always the risk of losing hard-won skills to the void of “use it or lose it”. This applies to everyone, everywhere, regardless of your profession or the tools in question. GenAI just happens to have made this brutally explicit, since its effects aren’t limited to any single profession. Instead, anyone doing anything with text has discovered that over-reliance on genAI tooling dulls the edge of their skills and makes it harder to do their work.

I’m explicitly not talking about non-devs who hit a brick wall when the genAI tools they’re using can’t handle the complexity of the generated code anymore, or people who don’t know how to write who just use an LLM to write their stuff for them. That’s not skill drain, that’s substituting a tool in place of skills and then discovering you can’t fix problems because you never had the skills necessary to do so.

What I’m referring to is senior-level devs who absolutely know how to architect systems, but feel like it’s a slog or miss obvious details that they previously wouldn’t possibly have missed. I’m talking about writers who have trouble getting words out on a page because they were offloading parts of it to an LLM. People who actively had and used a skill regularly, but then struggle to use it when they don’t have access to the tool anymore.

As a society, do we have an ethical obligation to make people retain their skills, even if they’ve been subsumed by new tools? Most professions require continuing education and re-certification throughout your professional career, to ensure that you retain your skills and awareness of the latest developments in your field. But I’m not sure there’s much push to force people to retain their skills when a replacement comes in. The only example I can think of that comes close is nurses needing to be able to calculate IV drip rates by hand, even though every hospital and medical facility uses machines that handle it for you.

At most, this could potentially be a professional obligation that we expect people to do, but I don’t know that it’s the role of society to ensure that everyone has non-basic skills, or retains ones that they’ve previously exhibited. Hell, we can’t even ensure that everyone has the ability to read and write properly, or do basic math, and we’ve got 12 years of mandatory education focused on that. I don’t think there’s any practical way for us to enforce this sort of thing, and it’s not obvious to me that it’s our place as a society to do so.

Individual freedom includes the ability to not learn a skill, or to walk away from using a skill. I know that I personally have walked away from some skills that I was perfectly fine at, but no longer wished to use.

I think the ultimate solution is what we’re already doing: prove you can do what you need to do within the context that you’ll be working in. If the work requires you to architect and build a codebase without genAI assistance, then you need to show that you can do that. If you can’t, then it doesn’t really matter whether you can’t because you never learned how or because you de-skilled due to over-reliance on genAI tools. You’re not suited for the job because you don’t have the skills you need to do the job.

Regularly using your skills can help to keep them fresh and mentally available, and I feel that that’s the best solution to skill drain w.r.t. genAI. Not every project should use genAI tooling or assistance, even if they’d be applicable and likely speed things up. Instead, make sure to take the time to do some manual architecture and design work without their input, and implement it all by hand.

Is this efficient? No, it’s not. But in this case, we’re not optimizing for efficiency. We’re explicitly trading off time-spent on a project for ensuring skills remain usable in a context where a genAI toolkit is unavailable.

Economic Costs

Beyond the massive societal issues that genAI forces us to contend with, we also have raw economic issues to deal with.

Issue: Grid Overusage

While there’s an ecological cost to how much energy gets burned on genAI, there’s an economic cost to it as well. Data centers have a major impact on their local economies, especially when it comes to power usage. This drives up electricity prices for everyone, and puts heavy strain on the power grid. The massive growth of genAI companies has resulted in multiple datacenters that are solely dedicated to training models and running inferences.

The ethical consideration comes in when we consider how the companies behave, and how they effectively take advantage of the small communities they build their datacenters in. They negotiate preferential (or effectively non-existent) tax rates, they place extreme strain on local grids that are absolutely not designed for the load, and don’t contribute at all towards the costs of improving that infrastructure to the level it should be to handle the load they’re directly placing on it.

Unfortunately, large portions of this are down to legislators making laws that prevent these sorts of abuses from happening. I’ll wait while you get your laughter under control.

I don’t know that there’s a valid solution to this. On the one hand, it’s not something we can really address on an individual level. The damage is being done, and the only real chance of stopping it lies in the hands of elected officials that, as a general rule, aren’t really being incentivized to prevent these abuses. It’s capitalism being capitalism and doing what it does. Democracy doesn’t really work well when elected officials can be legally bribed with campaign money.

On the other hand, just throwing up our hands and saying, “Nothing we can do about it” is the coward’s way out. Just because there isn’t a definitive solution doesn’t mean that there aren’t steps we can take to mitigate the issues and address the overall problem. Create an environment where that behavior isn’t profitable (or even profitable enough) and it’ll stop.

Within the context of AI-assisted development, there are multiple tracks we can take regarding this problem of ethics.

Solution A

The first is to not do business with any organization that engages in abusive datacenter practices. That’s the moral high-ground, but I’m not sure it’s viable. We know for a fact that Amazon engages in absolutely awful business practices, treating their employees like trash. Does that mean we don’t engage with Amazon at all? That’s rather hard, considering they literally own AWS and have primary market share in the cloud computing space. Do we limit ourselves to only interacting with subsidaries of a parent company that don’t directly themselves behave badly? How far down the chain do we go?

This isn’t a hypothetical either. It’s an actual issue. Our hyperconverged economy has a handful of megacorporations that own the majority of other companies in their space. You can’t shake a stick without hitting someone that is directly connected by company ownership to someone doing something awful. Buy local-only? Small businesses aren’t immune to doing unethical shit, and this simply doesn’t work for some things. There is no mom-and-pop cloud computing provider. You can’t get your prescription filled with drugs that were manufactured locally (unless you happen to be in India or similar; I’m in the US, so I certainly can’t).

If you’re going to engage in the modern world, consuming anything, you have to accept some level of interaction with people or companies that directly trace to bad things. There’s no avoiding it. I think that the best you can do, ethically, is avoid directly working with or for specific companies that you know are doing bad things. That’s why my household avoids purchasing from Amazon as much as humanly possible. Likewise with Walmart, Home Depot, Target, and anyone else that directly does verifiable bad shit. I can’t avoid chains of connection, but I can sure as hell cut the links that are directly touching me.

This focus on specific companies, rather than any company that shares a parent company, means that even though I dislike Amazon as a whole, AWS is currently fine because it’s a subsidary and (as far as I know) doesn’t engage in any systematic abusive practices. This is shaky, but also the only practical guideline for working in our current environment.

It’s messy, it’s not perfect, but you don’t often get perfect choices. Sometimes all your choices are shit and you have to pick between bad ones until you can make an environment with a better set. This certainly isn’t the ancestral environment, where we were 200-person social groups and you could effectively shun any single individual or small group of people that were doing bad things.

In the coding context, that means not using any model that was trained by a company that engages in unethical behavior (in the case of open-weight models), or using a closed-weight model API that is provided by a company doing the same.

Solution B

The second way to deal with the ethics issue of abusive datacenter practices is to only use open-weight models that can be sourced from ethical companies, or even to only use models that we have personally trained.

This is honestly the most appealing solution, in my mind, but it’s practicality isn’t all that clear. Ignoring the eternal problems with running large models locally, not all models have a bill of provenance that shows how they were trained, and using what datasets. NVIDIA, suprisingly, has one of the best examples of this that I’ve seen. They list all of the datasets they used to train the model, then how it was fine-tuned, which datasets are public, which are private, and so on. Naturally, they have their own datacenters, but I’ve not yet heard of them building or running them unethically.

Solution C

The third way is to imitate my fundamentalist Christian upbringing and fall back to the Poop Brownies analogy. Cutting through the build-up and engagement aspects, the idea presented is that even a small amount of poop (analogous to sin) will absolutely ruin brownies (a song, a movie, whatever), even if you use the highest quality ingredients for everything else, and that nobody would want brownies made that way. Applied here, the idea would be that because genAI is inextricably linked to bad behavior w.r.t. datacenters (and other ethical issues like copyright violations), there’s simply no way to use genAI at all ethically and therefore we must not do so at all.

This is the nuclear option, and it appears to be the stance that a lot of folks are taking in general when it comes to genAI. I can understand it. I’m just not sure it’s well reasoned or thought out, and it feels far more like a knee-jerk reaction to a new technology with massive societal impact that’s being used unethically.

Eval

Overall, I think that Solution B is the optimum path to take if it’s actually feasible, and Solution A as a backup or interim solution. Both are ethical, it’s just that B feels more ethical than A, from an ideological standpoint. Solution C is too reactionary, and I disagree with the premise that one bad actor can doom everything it’s related to. GenAI has issues, real ones, but that doesn’t mean we get to assign it more blame than it rightfully deserves. It’s got more than enough to stand on its own.

Issue: Incestuous Investments

I am not an economist. That said, it’s not particularly hard to see how a closed loop of ~10 companies passing money back and forth isn’t doing anything beneficial for the economy. Of course, you can’t ignore velocity of money, because in an abstract sense the money is still moving through the economy same as always. But in practice, what’s happening isn’t the usual “Peter pays Paul for services, Paul pays Tim for goods” system of money flow. And it’s certainly not helping individual consumers or workers.

These transactions are investments and speculative purchasing deals that aren’t doing any actual economic work. No new jobs are being created, no hardware is transferring hands, it’s just companies throwing money at each other to inflate their stock prices.

The only real exceptions to this are NVIDIA, which is making bank on AI GPU sales, and maybe the AI companies building datacenters (though even that is rather speculative in terms of how much economic benefit is being produced). But the large investments into AI companies overall? Not doing anything worthwhile. They’re burning massive amounts of capital on training models, but we’re rapidly reaching diminishing returns, and pretty much none of them have a profitable product. If they did, their yearly P&L sheets wouldn’t be so insanely in the red, and they’d be making constant noise about how profitable XYZ initiative is.

This is an economic issue, for sure. But is this an ethical issue? Yes, because a stupid amount of money is being wasted, it’s being used to lie to investors and setup a really nasty bubble, and it’s being used to cover up for the horrific state of the US economy.

Is this a direct ethical issue with using AI models? Eh, maybe. I think this goes back to the previous discussion about unethical company practices. You and I can’t directly prevent this kind of problem, and it’s not caused by our usage of the product in question. But how associated with it can we be before we’re complicit?

As with the previous answer, I think Solution B is the ethical choice here. We can theoretically avoid using tools that have been tainted by AI-bubble investment practices, and so we should do so to avoid being complicit in wrongdoing (even if it’s pretty tame as far as unethical shit goes).

Issue: Bandwagoneering

Anyone paying any attention to the state of corporate America has seen the insane amount of AI being applied to everything. Anyone who remembers the dotcom era will recognize the similarities to the Oracle craze; namely, if you wanted your software to be taken seriously by investors or large corporate buyers, it had to use an Oracle database. It didn’t matter whether it actually needed an “enterprise-grade” database, if it didn’t use it your software got dropped like subprime mortgage bundles in ‘07.

The comparison is apt, because companies are doing the same thing: incorporating AI into their business, somewhere, anywhere, just so they can slap on a “powered by AI” sticker. It doesn’t matter if it makes sense, they just want to be able to tell people “they do AI stuff”.

On it’s own, this isn’t really an ethical issue. Businesses have shiny object syndrome just as much as people do, because they’re run by people. At least, until we start replacing CEOs with AI; at least one study I read indicated that replacing middle management with AI actually improved operations considerably. There’s probably a lesson there, I’m just not the person who’s going to write about it.

But shiny object syndrome becomes a larger economic problem, and us intentionally supporting bad outcomes is unethical. No, not in the sense of “companies losing money is immoral”. That’s capitalist brainrot talking. But in the sense of, “companies are wasting time, money, and resources that could be used to improve the world”, intentionally contributing to that is definitely pushing ethical boundaries that I’d prefer not to get near.

The solution here, since we can’t outright stop companies and people from being stupid, is to not contribute to that environment. In a consumer context, don’t buy shiny new AI stuff unless you’d buy it without the AI stuff. As an individual contributor, push back against the spread of AI projects that don’t make sense. As a decision maker, don’t push for “more AI usage”, get yourself a set of business outcomes and see if AI can actually push those specific metrics; AI usage for the sake of AI usage is just burning money and lining the pockets of oligarchs.

Within my specific dev context, this isn’t something for me to worry about. I already do all of that stuff, and I’m only exploring genAI as a tool to be used appropriately, if at all.

Content Rights Costs

The most intrinsically unethical part of genAI, without any doubt, has to be the mass content scraping that was, and continues to be, used to produce the broad baseline training data, what is known in the industry as “pre-training data”. While copyright law absolutely needs some updating to deal with large megacorporations owning vast swathes of IP and abusing DMCA takedowns against fair use, that doesn’t mean that there isn’t a need for the protection of content creators. People who draw, write, act, VA, or code all deserve to have their creative output protected against being used for commercial purposes. The exact limits of that can be discussed and debated like reasonable adults, but the common starting point has to be that creative output must be protected to some degree.

If content had been scraped and incorporated into private, internal training datasets for models that never saw the light of day in the outside world, there would probably be far less moral outrage about it. Technically, yeah, it’s still unauthorized derivative work, but there’s not really that much difference between that and fanfiction or fanart. All but the most pedantic of folks would shrug their shoulders and go, “Eh, whatever. It’s not producing enough harm to be worth fighting about.” The exact same arguments as are used for fanfic or fanart.

However, that’s not what actually happened. Instead, we got LLMs that ingested the creative output of published authors, news agencies, researchers, independent bloggers, and every person who ever posted something on the internet. We ended up with stable diffusion and other image generation models that have been trained on the artistic styles of every artist with a digital presence (which is all of them, nowadays). We have audio models that can now take a dozen short audio samples of someone and output audio that humans find nearly indistinguishable. We have video generation models that can produce realistic-enough representations of people that the average human can’t tell the difference.

And yet, if we had all of that and it was being used for helpful things like letting disabled people have their own audible voice again once they’ve lost it, or letting an artist with an injury still produce creative works, or even letting someone see a video of themself moving in a particular outfit before purchasing it, that would be fine and most folks wouldn’t hate it quite so much.

No, folks hate genAI because it’s replacing genuine creative output with a giant pile of mass-produced sameness. It’s killing the ability of creative workers to sell their products, because it can produce something that is “good enough”. It’s enabling banking fraud. It’s being used for psyops of all kinds, and utterly eroding our ability to trust anything except for our own physical eyes.

That’s the baseline that we’re working off of for genAI. And honestly, it’s hard to move past that. When an entire industry is built on top of theft, it’s rather difficult to ethically use its products.

However, there is a ray of hope. Last year, a group of researchers released The Common Pile, a pre-training dataset that is ethically sourced. Only works which are in the public domain or have an appropriately-permissive license are included. They also used the dataset to train two large language models that performed at the same level as well-known contemporaries.

This is good, because it means that it’s absolutely possible to make genAI models that function well while only using ethically sourced data. No theft, no moral issues.

This does put a kink in our decision making, though.

On the one hand, we have an pre-training dataset that is ethically sourced, and removes all moral issues about content theft. Any model, including the big name players, could use this dataset as their baseline and build on top of it using their own proprietary subsequent fine-tuning. We’d never know, because as a general rule the giants in the AI industry keep extremely mum about their training data and processes.

On the other hand, we know that they’ve already been training their models on unethically-sourced datasets, and it would be rather naive to assume that they’re going to spend extra money and time and effort to switch out their known-good-results datasets for a new one just because it would be more ethical. They haven’t cared up to this point, why assume they’re going to suddenly start caring now? But again, they keep tight-lipped about things enough that we’d never know unless and until they drop a news release for the moral clout.

Where does that uncertainty leave us? Can we ollie-ollie-oxenfree from concerns about content rights now that there happens to be an ethical alternative? On the face of it, that’s pretty ridiculous. Do we then write-off all large AI vendors until and unless we know explicitly that they’re using ethically sourced data? On analysis, that also feels a little ridiculous, because we don’t do that for everything else in our lives. I guarantee you don’t know the provenance of everything you buy and use, whether that’s food, electronics, or your clothing.

Operating under uncertainty is hard. We often like to think that we’re certain about things, but really we’re just making best guesses and pattern matching and trying our best to not be awful to each other.

With that in mind, the obviously most-ethical and keep-your-hands-clean option is to only use models that pre-train on The Common Pile (or another known-ethical dataset), and do their fine-tuning on other datasets that are also ethically sourced. Synthetic datasets are becoming more and more popular across all the large models, because we’ve already scraped the internet, there isn’t any other set of large data up for grabs. There’s probably something that can be done with that as well, assuming the synthetic data isn’t itself tainted by being generated via an unethically-trained model.

The other option is murkier, from an ethical standpoint. We unfortunately don’t know which closed-weight model vendors have been training on unethical datasets, and we probably never will. It’s possible that they’re quietly transitioning to more ethical datasets and don’t intend to say anything about it at all. It’s also possible that they’re not doing anything to be more ethical in their data sourcing, because as of right now they’ve gotten away scot-free with the biggest heist in history and are making absolute bank on it.

With that near-permanent uncertainty, you could make the argument that we should look for proxies of ethical behavior and base our decisions on those instead. Is the company behaving with care towards their employees and contractors? Do they engage in unethical business practices? Do they take advantage of abusive contracts? Do they pay attention to concerns about ethical AI usage?

This is the less clean option, for sure. It’s pretty much the same as looking at large retailers and evaluating whether you’re going to give them your patronage or not.

I don’t have a perfect answer here, especially not within the context of AI-assisted development. The existence of The Common Pile and the Comma model trained from it raises one potential path forward, and shows that there is an ethical way to use genAI, but it’s also pretty clear that that model isn’t going to fair well when compared to something like Claude Code (which is the clear winner/preference across the developers that I’ve interviewed that use AI-assisted development).

Do we sacrifice ethical purity for the sake of convenience and results? I’m rather undecided here, mostly because content rights are important, but not generally at the same level as directly stealing someone’s food or money, so it’s not as clear-cut of a moral decision. But as a starting point, we have something to work with, and can’t categorically exclude genAI as a tool to be used in a development context.