Sarah and I have been foster parents for over two decades. We adopted four of our children after many years of temporary care. I mention it up front because everything below comes out of that, and because you should know what kind of witness you are reading: we are participants in this system, and we hold a view about it.

What that time taught us was mainly about the adults.

The Real Constraint

We have sat across from a lot of social workers. They arrive at your kitchen table carrying a caseload assigned them, larger than any one person could honestly serve, holding a folder for your family and a phone full of other families, expected to make careful judgments about the most consequential thing in a person’s life. All while a system measures them on whether the paperwork closed on time. The good social workers are extraordinary, and many of them are good. Almost all of them are tired in a specific way that comes from caring about more people than any person can actually hold.

That real constraint in social work is arithmetic rather than a funding line or a software gap. An individual can only truly care about so many folks at once. Every worker in that profession runs into that ceiling, and then keeps going anyway. The going-anyway is what wears them out.

Two Problems That Look Alike From a Distance

So when people ask what artificial intelligence should do for social work, we think the honest answer starts by naming which problem it is being pointed at. There are two, and they look similar from a distance.

The first is judgment: deciding what is happening in a family and what should be done about it. The second is overhead: the documentation, the duplicated intake, the records scattered across systems that don’t speak to each other, the court report assembled by hand at 9pm. Both consume the worker’s hours, but only one of them is the work.

The industry has spent a lot of energy on the first problem, and we think that was a category error rather than a failure of ambition. A system built to help decide about a family produces a number about a family. A number is a summary of what was previously written down about people who look like them, which in this domain means a summary of who has been watched before.

That is the field’s own record rather than our hypothesis.

What the Data Actually Learned

Look at what these systems were trained to predict. Allegheny County’s published methodology is candid that substantiated maltreatment was unusable as an outcome, so the first version of its screening tool was trained on re-referral and out-of-home placement instead. Those are records of what the system did, not records of what happened to a child. And there’s a structural problem baked into that choice: a family’s odds of removal rise the moment they’re investigated, because removal is something only the system itself can carry out. Train a model to predict removal, and you’ve trained it to predict its own agency’s future behavior, then handed that prediction back as if it were independent evidence. That is the loop. The model learns from what the agency did, then advises the agency, which then does more of it.

The track record is genuinely mixed. Illinois discontinued its predictive program in December 2017, with the department’s own assessment that it “didn’t seem to be predicting much”; it had flagged over four thousand children at more than ninety percent probability of serious harm while missing children who later died. Oregon withdrew its screening tool in mid-2022, explicitly to reduce disparities. New Zealand never launched its trial at all, after an ethics proposal to observe sixty thousand newborns drew the response that these were “children, not lab rats.” Allegheny’s tool, by contrast, is still in use, through an Associated Press investigation and an inquiry by the Justice Department’s Civil Rights Division. Carnegie Mellon researchers found it would have flagged 32.5 percent of Black children for mandatory investigation against 20.8 percent of white children, a methodology the county disputes.

The pattern underneath all of it should worry anyone building here. Eighty-five percent of families investigated for neglect in the United States live under twice the poverty line, and research finds that increasing the earned income tax credit by ten percent is associated with roughly nine percent fewer neglect reports. A model trained on that history will learn to find poverty and report it as risk, and it will be accurate about its training data while being wrong about the family.

Evaluations, Not Scores

Long before we built any of this, we had a rule at home: evaluations, not scores. An evaluation holds meaning, context, uncertainty, even disagreement. It tells you what’s actually going on with a family. A score throws all of that away so a list can be sorted from top to bottom. That’s a useful feature when you’re prioritizing a sales pipeline. It’s a catastrophe when the row being sorted is a household.

Ask anyone who’s watched a caseworker sit across from a parent and explain why a decision was made. That parent doesn’t want a score, they want to know what was believed about their family, who believed it, and on what basis. If the system behind that decision can’t answer those questions, it hasn’t helped the caseworker. It’s handed them something they now have to defend and cannot even explain themselves.

Where AI Should Actually Help

The second problem, the overhead, is the one worth attacking, and it is unglamorous in exactly the way real help usually is. Take the documentation. Take the retyping. Take the seven places the same fact has to be entered. Take the assembly of a report from records that already exist, and hand the worker back the draft and the hours. Leave the decision alone. Leave the family unranked. Give the professional more room to do the thing only a person can do, which is sit at the table and pay attention.

That distinction shapes how we build. It is why our systems are designed so a person stays in the loop with authority and tempo rather than as a rubber stamp at the end: when the folks in the loop set the pace, that is oversight, and when the machine sets the pace it is theater. It is why every consequential output carries a receipt saying what was used and what was decided, so nobody has to defend a conclusion they cannot reconstruct. And it is why we care so much about being able to run locally, on hardware an organization already owns, because case records are among the most sensitive documents in civic life. A child welfare agency should not have to accept “trust our cloud” as an answer.

The Fifth Voice

In barbershop singing, when four voices lock precisely enough, a fifth voice appears that none of them is singing. It is real acoustics rather than a metaphor, and it only appears above a threshold of precision. Below that threshold there are four voices and nothing more.

A foster home works the same way. Two people showing up with love produce something neither of them could have made alone, and it is called a family.

That is the thing worth protecting in social work, and it is the thing a template destroys fastest. When compassion becomes a template, it stops being compassion. Good software can carry the paperwork so a person has the hours to show up with real feeling. Showing up with real feeling stays a person’s job, and anyone selling you otherwise is selling a worse version of the paperwork.

Tell Us What You See

If you work in this field: what fraction of your week goes to the work, and what fraction goes to proving you did it? And if you’re building or funding tools for social work, let’s talk about what it looks like to take the paperwork off a caseworker’s plate without ever putting a number on a family.