U.S. markets close in 4 hours 8 minutes

ICE changed an algorithm to prevent the release of detained immigrants

Ephrat Livni
A child holds a sign during a rally in New York.

Momentarily put aside your positions on immigration policy, if you will, and consider this case of alleged algorithmic rigging. Whatever your feelings on people seeking legal status in the US, you may find cause for concern about humanity’s growing reliance on machines to determine liberty.

Last week, the Bronx Defenders and New York Civil Liberties Union filed a complaint in New York federal district court against local Immigration and Customs Enforcement (ICE) authorities. They allege that the agency adjusted the algorithm it uses to decide when someone should be released on bond. Now, detainees being held on civil immigration offenses overwhelmingly remain in custody even when they pose no flight or public safety risk and regardless of medical conditions. The advocates explain in a statement:

Federal law requires ICE officers to make individualized custody determinations based on whether the person poses a flight risk or threat to public safety. Since 2013, the agency has used a risk assessment tool that considers factors like a person’s family ties, connections to community, time in the country and community, and criminal history. However, the data shows that this tool, which ICE offices use nationwide, was manipulated, most recently in mid-2017, to remove its ability to recommend anyone be released. The tool can now only make one substantive recommendation: detention without bond.

Prior to this adjustment, about 47% of detainees were released on bond while their immigration cases unfolded. Between June 2017 and September 2019, however, only 3% were let out, according to the complaint. The adjusted tool recommends detention almost invariably even as immigration-related arrests are on the rise, which means that more and more New Yorkers are being held in custody for civil offenses without a proper risk assessment and nowhere to turn for help.

Although the algorithm can refer a detainee’s case to the New York Field Office for review, the complaint alleges that local ICE officials effectively treat the algorithmic recommendation as binding and have departed from the machine’s determination in less than 1% of cases. So people are being held in custody for weeks or months before they see an immigration judge and can make their case to a person rather than a rigged machine.

The humans, it turns out, are a little more nuanced in their decision-making. About 40% of those who were held without bond by ICE do end up released by a judge, the complaint says.

This is one indication that the algorithm isn’t doing the job it was designed for. Theoretically, the machine’s results shouldn’t be so extreme or they wouldn’t vary from the magistrates’ assessments.

Realistically, however, the New York ICE office’s algorithm seems to have been adjusted to advance a “no-release policy” that the complainants say violates federal immigration law and the US Constitution’s Due Process Clause. The advocates seek declaratory and injunctive relief on behalf of a class of detainees being held without bond on the basis of these contrived mechanical assessments.

The limits of machine learning?

Their case highlights a growing debate in the US and beyond about reliance on algorithms in justice systems, whether to do the delicate and awkward work of assessing human risk or pretty much anything else important.

On the same day that immigration advocates filed this complaint, the Washington Post ran a piece by Stanford University computational policy researchers on the superiority of machines in making bond determinations in the criminal justice context. Notably, their work shows humans tend to significantly overestimate risk and can’t assimilate new information as quickly as machines with brains.

In other words, a machine assessment could be better than human insofar as it accurately predicts risk…but of course, the tool would have to be used by people operating in good faith and not to advance a deliberately restrictive no-release policy in order for that to be true.

The Stanford researchers concede that algorithms do have limitations though, as they can advance the biases of those who designed the programs. And they don’t suggest people submit all decisions to machines, concluding, “When, whether and how to consider algorithms, however accurate, remain larger issues of policy and ethics—and those decisions are firmly in human hands.”

France, for example, last year banned machine learning for ‘judicial analytics,” barring use of statistics to predict and study judges’ behavior. But it seems highly unlikely that a similar resistance to algorithms will prevail in the US, which is why cases like the one brought against ICE in New York are so important.

As people learn more about how machine learning works—for better and worse—activists and advocates now have to monitor the bots that are making critical decisions about the freedom of human beings.

 

Sign up for the Quartz Daily Brief, our free daily newsletter with the world’s most important and interesting news.

More stories from Quartz:

  • Business
    TheStreet.com

    Did Trump Just Pull an April Fool's Trick With the Oil Market?

    It seems President Trump is as deluded as ever with his grand statements -- perhaps not reckoning the sheer stroke of genius on the part of Saudi Arabia and Russia to actually corner the U.S. via crippling its shale oil industry. Oil prices are down 60% from the highs in February 2020 as price has been hit not only by around 25 million barrels per day of demand lost but also by the surge of around 3 mbpd, as Saudi Arabia finally gave up on being the lone player supporting this market to help U.S. shale and other players grab their lost market share. The oil market has been in perennial oversupply for the past two years, with only Saudi Arabia and Russia supporting the price by taking about 2 mbpd out of the market, helping the U.S. shale companies to keep pumping and raking in more revenue.

  • Investor who called coronavirus collapse: Stocks aren't even close to a bottom
    Business
    Yahoo Finance

    Investor who called coronavirus collapse: Stocks aren't even close to a bottom

    There aren't a lot of investors who can say they warned clients about the coronavirus crisis and eventual 30% stock market collapse, but hedge fund founder Dan Niles is certainly one of them. In a February letter obtained by Yahoo Finance sent to clients of his long-short Satori Fund, Niles warned he was becoming “increasingly worried” that central bank accommodativeness was overshadowing the impact that COVID-19 would have on the global economy. After adjusting his position before the S&P 500 fell by more than 30%, his Satori Fund finished the first quarter in the green.

  • Refinances Come Roaring Back as Mortgage Rates Drop
    Business
    MoneyWise

    Refinances Come Roaring Back as Mortgage Rates Drop

    Like the big cats on the Netflix sensation Tiger King, refinances are roaring — while mortgage rates slink back near their lowest levels in history. As homeowners grab low rates by the tail to cut their monthly house payments, mortgage refinances have rebounded, a trade group says. For every $100,000 borrowed through a 30-year mortgage in 2020, you pay about $60 a month less than you would with a loan that was taken out 2019 when rates were higher, LendingTree chief economist Tendayi Kapfidze said recently.

  • Luckin Coffee shares fall 81% after board suspends COO for alleged 'fabricated transactions'
    Business
    Yahoo Finance

    Luckin Coffee shares fall 81% after board suspends COO for alleged 'fabricated transactions'

    Luckin Coffee (LK) shares collapsed after the company suspended its COO Jian Liu and “several employees reporting to him” for misconduct related to “fabricated transactions.” “The information identified at this preliminary stage of the Internal Investigation indicates that the aggregate sales amount associated with the fabricated transactions from the second quarter of 2019 to the fourth quarter of 2019 amount to around RMB2.2 billion,” the company said in a statement. “Certain costs and expenses were also substantially inflated by fabricated transactions during this period,” the company added.

  • Wells Fargo: 2 Big 11% Dividend Stocks to Buy (And 1 to Avoid)
    Business
    TipRanks

    Wells Fargo: 2 Big 11% Dividend Stocks to Buy (And 1 to Avoid)

    Stock markets fell 4.4% yesterday, marking the third session in a row of losses. There's a feeling of gloom; President Trump has said that the country and economy are in for a hard two weeks in the first half of April as the coronavirus epidemic peaks in the States, and he walked back his previously stated hope to see the country 'get back to work' by mid-month. It's in times like these that investors, when they buy, start looking that much harder at dividend stocks.

  • Business
    Barrons.com

    Dozens of Companies Have Cut Dividends. These 8 Aristocrats Could Weather the Crisis With Payouts Intact.

    As dozens of companies have cut or suspended dividends in recent weeks to preserve cash amid the uncertain impact of the coronavirus pandemic on the economy, the deteriorating payout picture has become a familiar refrain in this column. Barron's started with the S&P 500 Dividend Aristocrats, which have raised their dividends for at least 25 straight years, for ideas. The group has 64 members, and with the help of FactSet data, we whittled it down to eight that appear to have some bulwark to maintain their dividends, as the accompanying table shows.

  • Business
    MarketWatch

    Oil stocks soar after President Trump's tweet of coming Saudi, Russia production cuts

    Energy stocks soared Thursday, after President Donald Trump tweeted that he expects Saudi Arabia and Russia to cut oil production by about 10 million barrels, and "maybe substantially more." The SPDR Energy Select Sector ETF (xle) shot up 11.5% in morning trading, with all 27 components gaining ground, including 23 that were up by a double-digit percentage. Just before the tweet, the ETF was up about 5%.

  • American Airlines Taps Credit Lines For Cash Injection, Analyst Downgrades Stock to Sell
    Business
    SmarterAnalyst

    American Airlines Taps Credit Lines For Cash Injection, Analyst Downgrades Stock to Sell

    A filing has revealed that beleaguered airline American Airlines (AAL) has now accessed a total of $2.73 billion from three revolving credit facilities in order to cope with the coronavirus fallout. At the same time, Stifel Nicolaus analyst Joseph DeNardi downgraded his rating on AAL stock from Hold to Sell. Shares have plunged 63% year-to-date, with a 31% drop in just the last five days, and DeNardi believes the outlook for AAL is particularly worrying.

  • Business
    Investopedia

    Time Running Out for Cruise Line Stocks

    Carnival Corporation & Plc (CCL) announced a $1.25 billion secondary offering when the battered cruise ship line's stock was trading at $13.00 on Tuesday morning. Meanwhile, rival Royal Caribbean Cruises Cruises Ltd. (RCL) has notified lenders that it's drawing down the balance of revolving credit agreements that will now add nearly $3.5 billion in outstanding debt. The company has stopped all sailings until at least mid-May and ran into a stone wall of resistance in Congress about funding a bailout because it is domiciled in Liberia, not the United States.

  • BA may suspend 32,000 staff; Boeing eyes buyouts
    Business
    Reuters Videos

    BA may suspend 32,000 staff; Boeing eyes buyouts

    Reuters sources say the carrier is in talks with unions to suspend around 32,000 staff. The BBC says the number could go as high as 36,000. That after BA cut flying capacity by 75% in April and May.

  • 20 technology stocks with low debt to consider owning in a down market
    Business
    MarketWatch

    20 technology stocks with low debt to consider owning in a down market

    a href="https://www.marketwatch.com/story/20-technology-stocks-with-low-debt-to-consider-owning-in-a-down-market-2020-04-02? siteid=yhoof2" (FTNT) 2.2% -16% -6% 52% 12/31/2019 Computer Communications IPG Photonics Corp.

  • This hard truth about the mortgage markets isn’t being told
    Business
    MarketWatch

    This hard truth about the mortgage markets isn’t being told

    Already, it looks as if the U.S. is moving towards a temporary moratorium on mortgage payments. Fannie Mae and Freddie Mac unveiled an emergency program which provides a two-month deferral of mortgage payments for any homeowner who claims to be facing a hardship because of the virus. Many of you may sense that the virus has undermined what you thought was still a fairly strong housing market around the country.

  • What to expect from Chewy's earnings report
    Business
    Yahoo Finance Video

    What to expect from Chewy's earnings report

    Yahoo Finance's Alexis Christoforous, Brian Sozzi and Heidi Chung break down Chewy's latest earnings report.

  • Business
    Bloomberg

    I Manage Money for a Living. Here’s What I’m Telling My Clients.

    The question that came up most is whether investors should sell their stocks now and buy them back later when they decline further. The spread of coronavirus and the resulting economic damage is expected to worsen. As it does, the thinking goes, stocks will continue to decline.

  • What Is Wells Fargo's (NYSE:WFC) P/E Ratio After Its Share Price Tanked?
    Business
    Simply Wall St.

    What Is Wells Fargo's (NYSE:WFC) P/E Ratio After Its Share Price Tanked?

    Unfortunately for some shareholders, the Wells Fargo (NYSE:WFC) share price has dived 34% in the last thirty days. All else being equal, a share price drop should make a stock more attractive to potential investors. In the long term, share prices tend to follow earnings per share, but in the short term prices bounce around in response to short term factors (which are not always obvious).

  • Business
    Barrons.com

    Boeing Is Laying Off Workers. CEO Says It Could Take Years for the Industry to Recover from the Crisis.

    Commercial aerospace giant (BA) is taking additional action to protect itself from the fallout from the Covid-19 coronavirus outbreak. It will take time for the aerospace industry to recover from the crisis,” CEO Dave Calhoun said in a letter to employees on Thursday. When the world emerges from the pandemic, the size of the commercial market and the types of products and services our customers want and need will likely be different.

  • Need To Know: The Consensus Just Cut Its Arconic Inc. (NYSE:ARNC) Estimates For 2020
    Business
    Simply Wall St.

    Need To Know: The Consensus Just Cut Its Arconic Inc. (NYSE:ARNC) Estimates For 2020

    Market forces rained on the parade of Arconic Inc. (NYSE:ARNC) shareholders today, when the analysts downgraded their forecasts for this year. Revenue estimates were cut sharply as the analysts signalled a weaker outlook - perhaps a sign that investors should temper their expectations as well. Following the latest downgrade, the current consensus, from the six analysts covering Arconic, is for revenues of US$6.0b in 2020, which would reflect a concerning 58% reduction in Arconic's sales over the past 12 months.

  • Shopify Guidance Hurts Stock - Here Are Must-Know Support Levels
    Business
    TheStreet.com

    Shopify Guidance Hurts Stock - Here Are Must-Know Support Levels

    The broad market is rallying in the face of a sharp jump in jobless claims. But Shopify shares aren't participating in the market rise. They're down about 10%.

  • New scrutiny of stock deals by Georgia Sen. Kelly Loeffler and her husband ahead of outbreak
    Business
    MarketWatch

    New scrutiny of stock deals by Georgia Sen. Kelly Loeffler and her husband ahead of outbreak

    The husband of Georgia Sen. Kelly Loeffler recently acquired as much as $415,000 in stock in DuPont de Nemours, a chemical company that manufactures protective equipment in exceedingly high demand because of the coronavirus pandemic. The transaction, detailed in a mandatory disclosure the Republican filed late Tuesday, comes as senators in both parties have faced questions about the stock transactions they made in the weeks before the coronavirus upended the U.S. economy, wiping out jobs and personal wealth. Senate Intelligence Committee Chairman Richard Burr, R-N.C., whose sales of as much as $1.7 million in stocks have come under the most scrutiny, requested an ethics review of his actions in the days before markets dropped in February.

  • Luckin Coffee’s board initiates investigation into $300M potential fraud
    Business
    TechCrunch

    Luckin Coffee’s board initiates investigation into $300M potential fraud

    China-based Luckin Coffee, the fastest growing growing coffee brand in the world, has dazzled VCs, public market investors and frankly us over the years with its dizzyingly high growth, expanding from a handful of stores and delivery kiosks to tens of thousands in a just a few years, quickly outpacing Starbucks' aggressive expansion in the world's second largest economy. In a filing with the SEC this morning, the company's board announced that it has initiated an internal investigation into the activities of its former COO Jian Liu, who may have inflated revenues by the company by an early estimate of more than $300 million (RMB2.2 billion). Legal firm Kirkland & Ellis is the board's counsel in its internal investigation.

  • My Plan for Disney
    Business
    TheStreet.com

    My Plan for Disney

    If there is a large cap business more reliant on discretionary income, I suddenly find it hard to think of one. Simply think of just what it is that the Walt Disney Co. does and does well. Nearly every one of the firm's businesses have been put on the shelf, globally... as governments around the world have had to put a halt to non-essential commerce in response to the spread of the novel coronavirus that causes the infection now known as Covid-19.

  • Business
    Bloomberg

    U.S. Mortgage Rates Fall While Housing Market Grinds to a Halt

    The average rate for a 30-year loan was 3.33%, down from 3.5% last week, Freddie Mac said in a statement Thursday. Rates for 30-year mortgages have been on a roller coaster since tumbling to a record low of 3.29% in early March. While public showings vanish and sales slow, some buyers in contract are trying to renegotiate prices, according to Patrick Boyaggi, chief executive officer of online mortgage marketplace Own Up.

  • This part of the $2 trillion coronavirus stimulus package will save you money in your 401(k) and IRA
    Business
    MarketWatch

    This part of the $2 trillion coronavirus stimulus package will save you money in your 401(k) and IRA

    Retirees don't need to worry about taking the required minimum distribution from their retirement accounts this year. The government waived that rule as part of the $2 trillion stimulus package meant to provide financial relief to Americans. Normally, retirees who have not yet started withdrawing from their retirement plans, including traditional IRAs and 401(k) plans, must take a required minimum distribution from their accounts when they turn 72 years old.

  • Business
    MarketWatch

    Luckin Coffee shares sink 81% after special committee investigates 'misconduct,' suspends chief operating officer

    Luckin Coffee Inc. (lk) shares sank 81% in Thursday trading after it formed a special committee of three independent directors to investigate "misconduct, including fabricating certain transactions" that spanned the second through fourth quarters of 2019 and involves RMB2.2 billion (about $310.1 million). Jian Liu, chief operating officer at Luckin Coffee and a director of the company, has been suspended along with other staff implicated in the misconduct. Luckin Coffee may also take legal action against those involved.

  • Shopify Stock Falls As 2020 Guidance Pulled On Coronavirus Shutdown
    Business
    Investor's Business Daily

    Shopify Stock Falls As 2020 Guidance Pulled On Coronavirus Shutdown

    Shopify stock fell Thursday as the e-commerce platform provider pulled its 2020 guidance amid the coronavirus pandemic. While some of its customers may get a boost from increased online sales, there's also risk for Shopify stock if small businesses fail. Shopify stock fell 5.5%, near 363.40, in premarket trading on the stock market today.