The pandemic has increased interest from cities in the use of crowd monitoring technologies to manage safety, understand economic activity and reshape tourism. However, these systems can raise privacy and surveillance concerns and tap into the wider debate about the ethics and governance of data-gathering technologies in public space.
As part of its goal to be a leading city on the ethical use of technology, Amsterdam developed what it says is the first peer-reviewed privacy-by-design crowd monitoring system.
Public Eye, created with consultancy Life Electronic and smart city design cooperative Tapp, has now been made available as open source for other cities – and in future, companies – to use free of charge.
The project highlights a growing trend of cities taking a stronger stance and trying to set the ground rules for the deployment and direction of emerging technologies.
While Amsterdam doesn't want to take on technology vendors at their own game, it believes being proactive in this way can spur the market to develop new and better products.
The head-counting Public Eye system uses existing city cameras and a computer vision artificial intelligence (AI) algorithm to monitor and predict the size, density, direction and speed of crowds in public space.
It was also adapted to measure social distancing during the pandemic.
"Amsterdam is a very crowded city with lots of tourists, which can lead to some unpleasant but also sometimes unsafe situations," says Boen Groothoff, project manager for smart mobility in the City of Amsterdam's Chief Technology Office. Risks include violence, injury and falling into the canals.
Density map animation. Image credit: Tapp, 2021
The Public Eye system, which is being piloted at the Johan Cruijff Boulevard near the city's football stadium, the Marineterrein historical naval dock and 'living lab' district, and the Plein '40 – '45 square, aims to help combat these risks. For example, if an area becomes too busy, staff can take action to disperse crowds, such as implementing one-way measures, as well as designing longer-term strategies to prevent congestion.
Through predictive modelling, the city can understand where issues are likely to occur and act accordingly.
The data is also used to power traffic light systems that flag when areas or facilities are full, and to support an app and website, as well as digital kiosk messaging to help citizens plan their journey and avoid overcrowded routes.
For example, during the pandemic when gyms closed, Amsterdam used Public Eye at a popular outdoor fitness area in the Marineterrein. If the algorithm detected that people were too close to each other, the LED strips would turn blue and then red if too many people were in the area.
According to Groothoff, enforcement is not the goal. Some are concerned about this as more and more advanced tools emerge. The American Civil Liberties Union (ACLU), for instance, has warned that video analytics technology can turn cameras into "robot guards that actively and constantly watch people."
"We want to avoid that," says Groothoff.
"We don't want people to get [enforcement] tickets or fines; we want to prevent that by making sure areas don't get too crowded in the first place and helping people make informed decisions."
The tool is already delivering results, says Tom van Arman, Founder and Director, Tapp. For instance, the system confirmed that people left the Marineterrein area on time when there was a COVID-related curfew in place. It also pinpointed a rise in harbour swimming during the winter months due to the pandemic.
"The data is helping show facts about what's going on in terms of behaviours and trends and we're using that data to improve our experiences and improve our neighbourhoods," says van Arman.
Future Fitness Garden, Amsterdam. Credit: Tapp, 2021
What's in the box?
Public Eye grew out of the Crowd Monitoring System Amsterdam (CMSA) which uses sensors and cameras to measure footfall density. It was deployed for the SAIL tall ships event in 2015 and is now permanently in areas such as the Red Light (De Wallen) District, Central Station bus platform and near the ferry terminal.
When the city began planning for the 2020 UEFA European football championships, it wanted to monitor crowding around the stadium and nearby train station.
"It turned out there was not really any solution in the market that could actually monitor a whole wide area," says Groothoff.
"Most solutions focus on a small alleyway or similar."
Privacy concerns were also an important factor in developing Public Eye. Groothoff notes that the system decreases the need to monitor camera feeds featuring individuals by turning them into numbers and heatmaps. The camera footage is not saved or recorded, apart from a small number of images for algorithm training, and is processed on a city-owned encrypted network.
Algorithm transparency is also becoming an increasingly important issue for cities.
"It's very hard for the city to determine how privacy is ensured if you don't know how the technology behind it works," Groothoff comments.
"So when you're developing it yourself, you have more control that ethics and privacy is assured."
Private sector solutions are often 'black boxes' with little insight into how the algorithms work or space for them to be scrutinized. Public Eye aims to lift the lid.
The system is open source and Amsterdam enlisted the help of universities and privacy experts to evaluate it.
Public Eye has also been added to the city's AI Register launched last year to help citizens understand how data is being used.
Van Arman says: "This project really explores that paradigm of when cities become owners and operators of their own system, people can then peer review that system to ensure the integrity, which you may or may not be able to with commercial solutions."
Armed with knowledge gathered through developing and piloting Public Eye, Amsterdam is now putting out a tender to vendors for a crowd-monitoring solution.
"We see the [crowd-monitoring] market is also developing, which is very good, and this kind of competition really triggers the market to go the extra mile," says Groothoff.
If there's something as good as or demonstrably better than Public Eye, the city will consider procuring that. Otherwise, it will continue with its own system until there is.
As well as ethics, vendors will need to compete on accuracy. Van Arman says the prediction models have an accuracy rate of 85 to 90 percent.
"We had a fun little competition with a vendor last year and we outperformed them with our open source version," he comments.
Closing the loop
Public engagement is a key part of the Public Eye pilot. Information is displayed underneath each camera to explain what the research is about.
Further, through the Shuttercam project with the Amsterdam Institute for Advanced Metropolitan Solutions, the city is experimenting with cameras with shutters that open and close to show people when sensors are active.
Three prototype cameras are being piloted in the Marineterrein Amsterdam Living Lab – one that has a shutter operating to a set time schedule, another which residents can turn off temporarily or 'opt out' of and a third which has to be manually wound once a week as a physical barrier to make the city more mindful about whether it's necessary for cameras to be on indefinitely.
Shuttercam. Credit: Tapp, 2021
From here, Amsterdam wants to explore expanding the system – its own or a private-sector offering – to count cyclists and cars as well as pedestrians to help improve infrastructure. It could also be used as part of the testing of autonomous vehicles.
On the hardware side, the city is also looking to close the loop with an edge microprocessor so that no data needs to leave the camera.
Tapp and Amsterdam are exploring ideas and the city will also evaluate the market to see whether adequate solutions exist or if it could again develop its own to drive competition.
"This would get us closer to that 100 percent hermetically sealed privacy and ethical technology," says van Arman.
This article first appeared on Cities Today. Header image: Tapp via Cities Today.
