Capitol Fax.com - Your Illinois News Radar

DCFS says it’s scrapping predictive analytics program

Posted in:

Illinois

* From a May 9 Tribune story on then-DCFS Director George Sheldon…

In Florida, Sheldon worked closely with the Clearwater-based nonprofit called Eckerd Kids, which last year took in $169 million in government contracts to run child welfare and other programs in that state and others.

In Illinois, DCFS under Sheldon gave Eckerd a $375,000 contract to help develop a web-based program to pinpoint abuse and neglect investigations with the highest probability of serious injury or death to children.

In contract submissions filed as part of its Illinois DCFS contract, Eckerd touted the “remarkable” accomplishments of its predictive analytics method in Florida’s Hillsborough County, where it won a $65 million annual state contract to oversee child welfare services there in 2012.

But the firm has been embroiled in a series of controversies there, according to published reports. In October, a court-appointed advocate filed a lawsuit alleging that Eckerd and a subcontractor negligently placing a minor brother and sister in the home of an accused sexual predator. Eckerd separately acknowledged last year that 43 children were forced to sleep in offices and other unlicensed locations because Eckerd had run out of foster beds — after initially telling Tallahassee media that 17 youth were sleeping in the offices.

* Like much of what Sheldon did here, that contract has not turned out well and DCFS has announced its demise…

Two Florida firms — the nonprofit Eckerd Connects and its for-profit partner, Mindshare Technology — mined electronic DCFS files and assigned a score of 1 to 100 to children who were the subject of an abuse allegation to the agency hotline. The algorithms rated the children’s risk of being killed or severely injured during the next two years, according to DCFS public statements.

But caseworkers were alarmed and overwhelmed by alerts as thousands of children were rated as needing urgent protection. More than 4,100 Illinois children were assigned a 90 percent or greater probability of death or injury, according to internal DCFS child-tracking data released to the Tribune under state public records laws.

And 369 youngsters, all under age 9, got a 100 percent chance of death or serious injury in the next two years, the Tribune found.

At the same time, high-profile child deaths kept cropping up with little warning from the predictive analytics software, DCFS officials told the Tribune.

One child who did not get a high-risk score was 17-month-old Semaj Crosby, who was found dead under a couch in her Joliet Township home in April following at least 10 DCFS abuse investigations and an ongoing “intact family” care plan.

posted by Rich Miller
Wednesday, Dec 6, 17 @ 9:27 am

Comments

What, they harpooned the program because the data is wrong?

Or because the data is right?

Comment by Macbeth Wednesday, Dec 6, 17 @ 9:32 am
The privatization of social services does not work. Profit takes precedence over care.

Comment by Lamont Wednesday, Dec 6, 17 @ 9:32 am
you have to be competitive before you can be compassionate, turnaround agenda

Comment by Rabid Wednesday, Dec 6, 17 @ 9:36 am
Thanks Bruce.

Comment by PublicServant Wednesday, Dec 6, 17 @ 9:42 am
Another no-bid contract to haunt the Rauner administration. Good for Walker for cancelling this “grant” and doing a deep dive into the usefulness of this program.

From the article:

“A May 2017 Tribune investigation found the arrangement with Eckerd was among a series of no-bid deals Sheldon gave to a circle of associates from his previous work in Florida as a child welfare official, lawyer and lobbyist. Sheldon left Illinois under a cloud a month later, and a July joint report by the Office of Executive Inspector General and the DCFS inspector general concluded that Sheldon and DCFS committed mismanagement by classifying the Eckerd/Mindshare arrangement as a grant, instead of as a no-bid contract.

By doing so, the joint report said, DCFS avoided state bidding transparency requirements, making it impossible to determine if Illinois could have obtained the same services from local companies at a lower cost, a requirement of the state’s procurement code.”

Comment by Anon221 Wednesday, Dec 6, 17 @ 9:44 am
Bound to happen given the fact the Gov has no control of state government and no power over contracts. Madigan Madigan Madigan.

Comment by Give Me A Break Wednesday, Dec 6, 17 @ 9:44 am
Haven’t we learned our lesson about those out-of-state private firm contracts?

Comment by Dome Gnome Wednesday, Dec 6, 17 @ 9:45 am
IDK how much they were relying on it, but missing two kids does not mean it has not value. I am skeptical of “predicative analytics” in this case, probably too many variables the system has no access to, but I’m not sure you can say it’s useless. As for the false positives, personally I would rather err on the side of caution, though obviously it would be great to get every case exactly right. Pretty much a “meh” from me, except for the whole no bid contract from a Florida company again, which seems to be Sheldon’s shtick.

Comment by Perrid Wednesday, Dec 6, 17 @ 9:49 am
=== probably too many variables the system has no access to

This. I’m a big believer in predictive analytics, but for it to work you must have quality data. Almost by definition the families in chaotic situations are unlikely to have data collected on them that is sufficient.

Comment by ArchPundit Wednesday, Dec 6, 17 @ 9:53 am
Data are, not data is.

Comment by My New Handle Wednesday, Dec 6, 17 @ 9:53 am
===Data are, not data is.

Sigh. Yes.

Comment by ArchPundit Wednesday, Dec 6, 17 @ 10:00 am
Reminder: the lawyer that approved this failed, no-bid, crony contract at DCFS was then promoted by Bruce Rauner to General Counsel for the Governor’s Office.

Comment by Juvenal Wednesday, Dec 6, 17 @ 10:04 am
I second Macbeth… what if the findings were correct? And if the findings were incorrect then there must be other examples from the data showing how.
Or perhaps there were people busy, busy like cats in the litter box trying to bury unpleasant examples of sloth and ineptitude?

Comment by Puddintaine Wednesday, Dec 6, 17 @ 10:31 am
I work in predictive analytics. First thing to remember is that All Models Fail. All of them. You’ll miss some cases, you’ll get false positives, etc. It’s a model, not real life.

Also, as others alluded to, it looks like a “Garbage in, Garbage Out” situation, where they didn’t have good controls. A data set that only looks at kids in the system isn’t looking at the whole picture.

I’ll bet a million dollars that they overfit the model in order to sell the service.

Comment by ChrisB Wednesday, Dec 6, 17 @ 10:32 am
Shame, shame, shame on whoever is running Illinois and lets kids lives be in jeopardy while Sheldon and his pals are lining their pockets.

Comment by Flynn's mom Wednesday, Dec 6, 17 @ 10:35 am
The Fourth District Appellate ruling in favor of AFSCME’s request to stay the ILRB ruling in favor of Rauner expressed concern over the proposal to demonstrate efficiency and economy prior to privatization of services. Rauner’s obsession with wanting to destroy public sector unions and reward private corporate benefactors even if detrimental to those dependent upon public services is both fiscally and morally bankrupt.

Comment by kitty Wednesday, Dec 6, 17 @ 10:39 am
Meant to say “removal” of the requirement to demonstrate efficiency and economy

Comment by kitty Wednesday, Dec 6, 17 @ 10:41 am
“ChrisB” thanks save me much typing. If the system predicted a certain kid had a problem give the case worker the info and let them figure it out.

Comment by Anonymous Wednesday, Dec 6, 17 @ 10:42 am
Look, I think analytics are inevitable and ultimately a really great tool.
But it’s no substitute for the highly trained DCFS investigator
I have a lot of those folks in my local.
Honestly honestly I would trust them with my life and I would trust them with our precious children
A computer can point you in the right direction
But what we need are more investigators
In the pipeline
The ranks of DCFS have been decimated
By Rauners and Republican
Attacks on state workers
Many retired, many left
But were not replaced.
Rebuild the Department.
Utilize the expertise of veteran investigators
Don’t screw with them till they leave.
dCFS has the most grievances of any agency for our local.
Staff up and treat them right
So that they can do what they do best
Protect our precious kids
Just like the movie I, Robot
“A human would have known that.”
( Will Smith was saved by a robot instead of a young girl because the robot calculated that Spoons had a greater chance of survival. “Save the girl” Spoons screams. Later when sharing the story he says the above line.

Comment by Honeybear Wednesday, Dec 6, 17 @ 11:00 am
Even Superstars screw up :

$2.4 million warehouse lease, bungled

$94 million health insurance portal, bumbled

$12.5 million McKinsey contract, revoked

What’s next? …

Comment by Sugar Corn Wednesday, Dec 6, 17 @ 11:21 am
Honeybear, I love your blank verse styling. Reminds me of Vachel LIndsay…

Comment by Soccermom Wednesday, Dec 6, 17 @ 12:25 pm
Lol soccer mom -
thanks but really my “style”
as several folks have told me they enjoy
Is more a function of trying to type with
my old iPhone screen.
I can’t see full paragraphs and sentences
The width is dictated by the tiny screen size.
It is a “happy little accident” as Bob Ross says
But I do like the rhythm and emphasis
Just the form creates

Comment by Honeybear Wednesday, Dec 6, 17 @ 2:33 pm
HoneyBear is right. Analytics is here to stay given Silicon Valley’s “solutionist” attitude that every perceived inefficiency should be solved with predictive algorithms. Data analyst Cathy O’Neil outlines the problems that can arise with predictive analytics in layperson terms in “Weapons of Math Destruction.

Comment by AlgorithmicSelf Thursday, Dec 7, 17 @ 8:40 am