Selecting Dutch people with algorithms for government control is' most likely 'illegal

MARCH 14, 2024·

3 MIN

Selecting Dutch people with algorithms for government control is' most likely 'illegal

JAN-HEIN NOOSE DAVID DAVIDSON

4 CONNECTIONS

13 CONTRIBUTIONS

The fact that the government profiles Dutch people with secret algorithms is “in all likelihood” in violation of the privacy law. This is evident from an opinion from the national lawyer that was sent to the House of Representatives yesterday. “These conclusions have the potential to have far-reaching impact,” report the state secretaries for finance. However, the tax authorities say they are not stopping the way they work.

The tax authorities control Dutch people on a large scale using automated systems. This is how they filter out the risk cases: an incorrect declaration, a surcharge that is too high or fraud. If someone meets certain characteristics, they will receive a manual check. These risk models are not transparent and are not covered by legislation. This means that there is no legal protection — you cannot have the profile that sets up a risk model reviewed by a judge. In addition: you knows not even that you are profiled. This process violates fundamental rights, the Parliamentary Committee of Inquiry on Fraud Policy and Services ruled in its report this month Blind to people and justice. Because algorithms sometimes discriminate, even based on origin. Automated profiling decisions — such as approving a tax return — are prohibited by law. A government can only decide automatically if a law states that it is allowed. So far, the government has argued that profiling with risk models is nevertheless permitted, because after the model has been pre-selected, an official finally decides on manual control. Thanks to this “human intervention”, it would not be an “automated decision”, and therefore no legal basis with guarantees would be necessary.

Groundbreaking ruling

This view is wrong, according to a groundbreaking ruling from the European Court of Justice for the European Union at the end of last year, about Follow the Money earlier this month wrote. The ruling led to major concerns at the Ministry of Finance that consulting asked Pels Rijcken, the state attorney. The conclusions from this have “a potentially far-reaching impact”, wrote the State Secretaries of Finance Marnix van Rij (Tax Administration) and Aukje de Vries (Surcharges and Customs) to the House of Representatives yesterday. The national lawyer concludes that there is very quickly prohibited automatic decision-making. Selecting Dutch people for manual control due to a high risk score is “in all likelihood” sufficient for this, even if an official decides later whether there is an error or fraud. This has to do with the fact that the selection itself can have serious consequences, such as rejecting a tax return.

DOSSIER

DIGITAL BUSINESS

What happens to the data that governments, companies and institutions store about us? What if they are hacked or taken hostage? How secure are our systems and data?

VIEW ARTICLES

An exception to the ban requires a legal basis. But it is therefore missing. For this reason, profiling is most likely illegal in many cases, so concluded experts also consulted by Follow the Money. The administrators are not talking about its “far-reaching impact”. There will be further advice from the national lawyer about “next steps”. In addition, two scientists will be asked what room the government has to “apply automated selection techniques in services and supervision”. After all, risk-based supervision is considered indispensable. This advice is expected in mid-April. The Data Protection Authority (AP) will also advise on the matter, the State Secretaries report. The tax authorities say in a response that they will not stop the risk models, after weighing the importance of privacy and supervision. “Risks are accepted.” The opinion that profiling as it is currently happening is most likely not allowed does not only have consequences for the tax authorities. Other ministries and local governments also use risk models. In a letter to the House of Representatives on 13 February, the state secretaries promised to inventory the use of risk models across the state.

Tax authorities have crossed the line for some time

The tax authorities have long been concerned about the legal sustainability of this approach. But the administrators were silent about that until recently. In early 2021, the Small and Medium Enterprises department was internally advised to stop three models that it used to detect fraud and errors. None of the three of them would comply with the Privacy Act and may even violate fundamental rights. The opinion warned that there was a high risk of discrimination. “Unfortunately, it turned out that none of the models comply with the General Data Protection Regulation.” After a brief stop, the management decided to turn the systems back on. In doing so, the “social and organizational interest” outweighed compliance with the law and avoiding the risk of fundamental rights violations, revealed Follow the Money at the end of last year. Internal documents also stated that the tax authorities are very likely to break the ban on automated decisions, because it also applies to approving decisions. Indeed, assessments are approved automatically, without further control, if the declaration within the model does not receive a high risk score. In another model, which should prevent the tax authorities from wrongly returning money, a payout is automatically (temporarily) stopped when a return is flagged by the algorithm. That delay alone can have significant consequences for citizens, so there is a good chance that it is legally not allowed, lawyers ruled. Nevertheless, the tax authorities are going ahead with it. The service does not want to provide openness about systems because otherwise they would be less effective in detecting fraud.

Share this article, your friends will read it for free