General issues/Healthcare/Macro Social Work/technology

Weapons of Math Destruction: A Review

Posted on by admin

In the last 8 months or so I have been making a shift from technology enthusiast to a bit of  a tech skeptic.  Technology is certainly becoming more ubiquitous and this comes with benefits but it also comes with costs.  In healthcare and mental health, I have been pondering what pieces of technology will bring about change. Not only that but what kinds of design will people use and is it ethical?

I recently captured a twitter thread about the unintended consequences of technology. Also spending the last month or so reading “Weapons of Math Destruction” by Cathy O’Neil has challenged a lot of my assumptions about data collection.

As somebody who still has my research core from my undergraduate education, I assumed that all this acceleration of data can only yield positive results (especially understanding those in under-served area’s) but the above book has proven me wrong.   There is a lot of data collection that has potential to further divide us and often exclude other. The book goes into vast detail but here are a few examples…

  1.  Judges often use algorithms to determine sentencing of criminals. This determined by “risk of further offense”. This often leads them to people in poverty and minorities; creating a vicious cycle of incarcerating communities and those who associate with them.
  2. Police Departments are using algorithms to predict where crime “may happen” also create a similar feedback loop where they focus on certain neighborhoods. Leading to increased “stop and frisk practices” and increase in arrests in certain pockets.
  3. Insurance Companies (including Life, Health, and Car insurance) have always relied on math of some kind but are increasingly leaving this work up to computers. Life and car insurance companies are frequently relying on credit scores. Not only can these credit scores be inaccurate and prejudicial but some companies use their own opaque “e-score” to determine insurance rates. Leading to people being denied or being priced out without knowing why.

She calls these algorithms that create these feedback loops Weapons of Math Destruction or “WMD’s”.  Going into detail into many other examples she describes  3 main elements of WMD’s.  First is the Opacity or lack of transparency about what is being measured and how. Second is the scale of how many individuals it may impact and lastly what is the damage or potential consequences.

On January 23,2018 the #HTreads (Health Tech reads) twitter chat will gathered to answer some questions about the book. Here were some of the resources/thoughts I found helpful..

What are some early successes with using big data in healthcare?

 

Moving forward Data Scientists have to be critical of the ways we are using big data in healthcare. In a multidisciplinary manner, we have to ensure that we are being more intentional about asking these ethical questions.

, , , , , , ,
admin