New Gyroscope Design Will Help Autonomous Cars and Robots Map the World - IEEE Spectrum

2022-07-01 23:44:12 By : Mr. Alex Ou

The July 2022 issue of IEEE Spectrum is here!

IEEE websites place cookies on your device to give you the best user experience. By using our websites, you agree to the placement of these cookies. To learn more, read our Privacy Policy.

As useful as GPS is, it’s not a system that we can rely on all the time. There are many situations in which GPS might not be available, such as when you’re inside a building, driving through a tunnel, on a battlefield, in a mine, underwater, or in space.

Systems that need to track location, like your cell phone, an autonomous car, or a submarine, often use what are called Inertial Measurement Units (a combination of accelerometers and gyroscopes) to estimate position through dead reckoning by tracking changes in acceleration and rotation. The accuracy of these location estimates depends on how accurate the sensors in those IMUs are. Unfortunately, gyroscopes that can do a good job of tracking rotation over long periods of time are too big and too expensive for most commercial or consumer-grade systems.

Researchers at the University of Michigan, led by Khalil Najafi and funded by DARPA, have presented a paper at the 7th IEEE International Symposium on Inertial Sensors & Systems on a new type of of gyroscope called a precision shell integrating (PSI) gyroscope, which the authors say is “10,000 times more accurate but only 10 times more expensive than gyroscopes used in your typical cell phone.” It offers the performance of much larger gyroscopes at one thousandth the cost, meaning that high-precision location awareness, indoor navigation, and long-term autonomy may soon be much more achievable for mobile devices and underground robots.

For some context on just how much better PSI gyro is, it’s important to understand the performance you get from the kind of gyroscopes that you almost certainly have in your cell phone.

Your phone has three gyroscopes plus three accelerometers that are combined into an inertial measurement unit (IMU) which can detect both acceleration and rotation in all three axes. This IMU probably costs a few dollars. The gyros in it have a sensitivity* that’s on the order of about 1,000 degrees per hour, meaning that if you spent an entire hour very slowly rotating your phone, the gyro would notice if there were three-ish or more rotations over that time, but if you rotated your phone a bit slower so that it rotated only twice over an hour, the rotation would be too slow for the gyro to detect. This is fine, because your phone only really cares about big, fast rotations (tens of degrees a second), like when you pick your phone up or flip it over.

Along with sensitivity, there’s also accuracy, or how close the rotation that a gyro measures is to the actual amount of rotation that it’s experiencing. Most sensors have some kind of accuracy offset, which is called bias—this is what the sensor sees when it should be seeing nothing, or in the case of a gyro, what it measures when it’s not moving at all. There’s nothing necessarily wrong with bias in a sensor as long as that bias is stable. If the bias is the same all the time, you can calibrate for it, and get accurate results. But if the bias is unstable, and changes unpredictably, that’s going to really mess with the accuracy of your sensor.

For a typical cell phone gyro, the bias instability is anywhere from hundreds to thousands of degrees per hour. Again, this is fine, because your phone is more concerned with measuring tens or hundreds of degrees of rotation on the second scale, which is all it needs for games or to stabilize the camera.

So why do we care about gyros with high sensitivity and stable bias at all? It may not be all that relevant for cell phones, but it’s exceptionally important for submarines, spacecraft, autonomous cars, indoor robots, and anything else where you need to keep track of where you are without relying on GPS. All of these systems use IMUs to track their relative position, taking their last known absolute position (from GPS or a pre-existing map or whatever) and then relying on the IMU to combine continuous measurements of acceleration and rotation to estimate motion relative to that point.

Essentially, it’s just a fancy method of dead reckoning, and like any other method of dead reckoning, small errors build up over time such that the longer you rely on it, the farther off course you’re likely to get. Submarines and spacecraft use fancy (and expensive) gyros full of lasers and stuff to minimize these errors and the resulting drift in positional accuracy, but for ground robots, drones, and (to a lesser extent) autonomous cars, size and mass and cost are significant factors. Whether it’s for a robot making a map in a mine, or an autonomous car driving through a tunnel, you want a gyro that’s sensitive, accurate, affordable, and small, all at the same time.

Photo: Najafi Group/University of Michigan

This is where the University of Michigan’s PSI gyro comes in, because it’s small, inexpensive, and thousands of times more accurate than a conventional cell phone gyro. The secret to this level of accuracy is the heart of the gyro— a resonating structure of ultra-pure metal-coated glass that looks almost exactly like a upside-down Bundt cake pan, or an upside-down bird bath.

To understand why the PSI gyro’s resonator is this shape, and how it works as well as it does, let’s first take a quick look at the kind of gyros that you have in your phone. They’re almost certainly mechanical vibratory gyroscopes, which are very small and easy to make with conventional semiconductor manufacturing techniques.

You can think of the core of these gyros (the mechanical resonator) as being kind of like a tuning fork—one of those metal things with a stem and two arms that vibrates (producing a tone) when you whack it against something. The specific kind of vibration that you get when you whack a tuning fork consists of the tips of the two arms (tines) oscillating in and out from each other (at 440 Hz for standard concert pitch)—that’s where the tone comes from. 

Now, what happens if you give that tuning fork a twist, rotating it around its long axis? Inertia means that the tines want to keep vibrating in the same plane that they started vibrating in (the in and out vibration), but they can’t, since the base of the tuning fork is being rotated, and they’re attached to the base. Instead, the tines begin to vibrate in a second plane in addition to the first, due to the Coriolis effect—the same effect that you have to consider when measuring motion on Earth, because the Earth is rotating.

Anyway, for the rotating tuning fork, the Coriolis effect causes the tip of each tine to start oscillating back and forth (out of the plane of the tines). The more the tuning fork gets rotated, the more the in-and-out vibration gets converted into the back-and-forth vibration by the Coriolis effect, which means that measuring how much the tines of the tuning fork are vibrating back and forth can tell you how much rotation is happening at the base.

There are a bunch of different designs for these vibrating structure gyroscopes (sometimes called Coriolis vibratory gyroscopes), and only some of them have resonators that actually look like tuning forks, but they all work in pretty much the same way—whack a thing so that it vibrates, and then measure how much the in-and-out vibration gets converted into the back-and-forth vibration to calculate rotation.

The PSI gyro looks nothing like a tuning fork—its resonator has that Bundt pan or bird bath shape, more generally referred to as a wine glass shape, which is helpful because (at least for me) it’s somewhat easier to imagine a vibrating wine glass. Just like the tuning fork, you can whack a wine glass and it’ll start vibrating. If you look at the wine glass from above, this vibration is the rim of the glass deforming slightly, going back and forth from a circle to ellipse. And also just like the tuning fork, if you rotate the wine glass, the Coriolis effect induces a second mode of vibration offset from the first that you can measure to figure out rotation.

A wine-glass-shaped resonator is much better at measuring rotation than a tuning-fork-shaped resonator for several reasons. First, the shape is axisymmetric, or symmetrical around a central axis. This symmetry means that the two modes of vibration happen at almost the exact same frequency, so when the Coriolis effect transfers energy from the first mode to the second, it’s very efficient, meaning that you get more vibration per amount of rotation, leading to a more sensitive gyro.

Wine-glass resonators also have a very high “Q factor,” which is a dimensionless measurement of the ratio between the amount of energy that’s stored in a resonator, and how much energy the resonator loses every time it oscillates. When you whack a wine glass, it vibrates, moving and making noise. If it kept doing this forever, it would have infinite Q, but of course it’s giving up energy in all sorts of different ways (acoustic, thermal, mechanical), so eventually it’s going to stop vibrating. A high quality wine glass might have a Q of several thousand. A typical cell phone gyro has a Q of a few hundred and vibrates for much less than a second. The resonator that makes up the core of the PSI gyro has a Q of 5.1 million (!), meaning that once it’s packaged up in a vacuum it’ll vibrate for 300 seconds after being whacked one single time. 

The reason why high Q is important is because you want that primary vibration of your resonator to be as clean as possible, to more effectively transfer vibration from one mode to another. Other characteristics of the resonator that improve accuracy are relatively high mass and stiffness, both of which also make it less prone to noise.

Put all of this together, and the bias instability (accuracy of motion detection) of the PSI gyro is an incredibly low 0.0014 degrees per hour, which is several tens of thousands times better than a typical cell phone gyro. This is easily sensitive enough to detect the motion of the Earth rotating under your feet.

But maybe you don’t need to measure the Earth’s rotation, so let’s take a look at how much of a difference this gyro might make for an autonomous car trying to accurately track itself without GPS.

We’ll assume that this car is using an IMU with flawless accelerometers, but the kind of gyros that you typically find in cars (a bit better than the one in your cell phone). We’ll also assume that the car is driving somewhere flat, meaning that we’re only concerned with the output of one single gyro (yaw).

After just one minute of driving at 5o km/h, the errors that accumulate from a standard automotive gyro will result in a positional error of seven meters. After five minutes, that positional error has increased to more than 850 meters. But for a car equipped with a PSI gyro, one minute’s worth of positional error is just two millimeters, and only 30 centimeters after five minutes, which (for an autonomous car) is potentially the difference between staying in the correct lane and ending up in the next town over. It’s huge.

The new gyroscope with the vacuum package open, revealing the resonator inside.Photo: Najafi Group/University of Michigan

The PSI gyro’s resonator is manufactured from ultra pure fused-silica glass covered with an electrically conducting layer. The resonator is flipped upside-down, and then attached to a glass substrate by the center bit. The rim of the resonator isn’t attached to anything (so that it can resonate), but there are electrodes around the outside of the rim that drive the resonator (they do the whacking) as well as sense the vibration modes of the resonator to detect rotation.

The whole thing is packed up in a vacuum-sealed box, which is less than two centimeters in size. Getting a high Q factor, high mass, and high stiffness in a resonator gets much more difficult as you scale things down, which is what’s so unique about the PSI gyro—it’s not just the performance itself, but rather the combination of performance, size, and cost. 

The researchers say that the performance of the PSI gyro is comparable to those super fancy gyros that the military uses on submarines, except that while those cost thousands or tens of thousands of dollars each, in production volume ( > 1 million units) the PSI gyros would cost as little as $50 each. It’s also at least 10 times smaller than those other high-performance gyros, which really makes you wonder why anyone would keep using them as soon as these things become commercially available.

Fortunately, commercial availability is part of the plan, we’re told—the folks behind the PSI gyro have formed a startup calledEnertia Microsystems to commercialize both the PSI gyro and very similar design called the Birdbath Resonating Gyro (BRG), which is not quite as high performing as the PSI gyro, but is a bit smaller. They’re targeting autonomous vehicles, robotics, and some consumer applications, where there’s enormous potential for improvement through better, more affordable sensors.

*The technical definition of sensitivity is somewhat different than this in the context of gyroscopes, but the definition used here is a simpler way of communicating the same idea.

Our investigation reveals that Europe is turning to remote sensing to detect seafaring migrants so African countries can pull them back

Migrants in a dinghy accompanied by a Frontex vessel at the village of Skala Sikaminias, on the Greek island of Lesbos, after crossing the Aegean sea from Turkey, on 28 February 2020.

It was after midnight in the Maltese search-and-rescue zone of the Mediterranean when a rubber boat originating from Libya carrying dozens of migrants encountered a hulking cargo ship from Madeira and a European military aircraft. The ship’s captain stopped the engines, and the aircraft flashed its lights at the rubber boat. But neither the ship nor the aircraft came to the rescue. Instead, Maltese authorities told the ship’s captain to wait for vessels from Malta to pick up the migrants. By the time those boats arrived, three migrants had drowned trying to swim to the idle ship.

The private, Malta-based vessels picked up the survivors, steamed about 237 kilometers south, and handed over the migrants to authorities in Libya, which was and is in the midst of a civil war, rather than return to Malta, 160 km away. Five more migrants died on the southward journey. By delivering the migrants there, the masters of the Maltese vessels, and perhaps the European rescue authorities involved, may have violated the international law of the sea, which requires ship masters to return people they rescue to a safe port. Instead, migrants returned to Libya over the last decade have reported enslavement, physical abuse, extortion, and murders while they try to cross the Mediterranean.

If it were legal to deliver rescued migrants to Libya, it would be as cheap as sending rescue boats a few extra kilometers south instead of east. But over the last few years, Europe’s maritime military patrols have conducted fewer and fewer sea rescue operations, while adding crewed and uncrewed aerial patrols and investing in remote-sensing technology to create expanded virtual borders to stop migrants before they get near a physical border.

“The main reason is because the E.U. wants to step away from having proactive naval operations,” says international relations researcher Maria Gabrielsen Jumbert of the Peace Research Institute Oslo, in Norway. Physical encounters with migrants involve at least two forms of legal jeopardy that European countries are trying to avoid: an obligation to rescue seafarers and, once they are on land, an obligation to evaluate any seafarers’ claims of asylum.

In the last five years, Europe has bestowed massive new regulatory and spending power on the European Border and Coast Guard Agency, known as Frontex, which has in turn issued contracts worth hundreds of millions of euros to major engineering firms for remote border-control hardware, software, and know-how. Europe’s research initiatives, treaties, and contracts reveal an interest in peering across the Mediterranean into North African countries and dissuading or preventing migration at its point of origin. Meanwhile, legal scholars and civil-society groups are asking whether a hands-off border can really keep Europe’s hands clean.

Francesco Topputo, an aerospace engineering professor at Milan Polytechnic, Italy, who has worked on satellite-based surveillance research, says that the fate of migrants detected by his system isn’t up to him: “I would say that it’s not the decision of the technicians, of the engineers…it’s our job to give the information to the authorities. It is a problem of the entire society.”

Mediterranean migration hit international headlines in 2015 when the Syrian civil war helped drive up numbers to around 1 million people. But that was an unusual year. The U.N.’s International Organization for Migration (IOM) reports 225,455 arrivals in 2014, and by 2019, numbers were below 125,000. Irregular immigrants, whose movement “takes place outside the regulatory norms of the sending, transit, and receiving country,” according to the IOM, represent around 5 percent of the European Union’s total annual immigration of 2.7 million people. In other words, it’s a small fraction of legal migration, which in turn is an even smaller fraction of Europe’s overall population of around 447 million people.

Europe’s 125,000 irregular immigrants in 2019 also number fewer per capita than irregular immigrants to the United States, which has just three quarters the population of the European Union yet reported more than 1 million irregular immigrants at its borders in 2019.

Meanwhile, the European Union spends at least €2 billion (US $2.13 billion) a year internally on managing migration, not counting national-level spending. In 2015, under pressure to address migration from Syria’s civil war, European leaders failed to build a working redistribution of asylum seekers, but they did set in motion a legal framework for a newly empowered European border agency.

At that time Frontex had an annual budget of €142 million (US $156 million) and acted as a kind of liaison network between national border agencies. But the post-2015 rules ballooned its budget. By 2020, when Frontex had gained a more independent legal status as an agency of the European Union, its budget had more than tripled to €450 million and was scheduled to climb another 20.6 percent to €543 million in 2021.

Now, Frontex is refocusing its resources from shipboard patrols to aerial and remote sensing, according to its requests for orientation on the latest technology. The cost of shifting from rescue operations to border enforcement may be harder journeys for migrants and the deaths of some.

In early 2017, in the forest highlands of eastern Guinea, a man I’ll call Jacob began a journey that would take him across five Saharan countries and multiple failed sea crossings. He first set out from home after his father died, to look for work in Mali, he says. Mali was a conflict zone, so he moved on to Algeria, but he lacked a work permit, and employers would underpay him or fail to pay him altogether. The police hassled him and other migrant workers.

The workers created informal networks and shared information about where they could get work and how to avoid the police. Following those tips, Jacob worked his way across the desert, sometimes accepting loans from employers or traffickers that turned him into a modern indentured servant.

Meantime, the E.U. was slowly changing its hodgepodge of barriers to keep out Jacob and hundreds of thousands of other migrants. When the 2015 migration surge to Europe began, Spain employed one of the most technologically advanced border-control systems in Europe: the Integrated External Vigilance System, or SIVE (the Spanish acronym). Migrants in those years faced a multisensor gauntlet, involving radar and infrared cameras on towers, aboard ships, and on ground vehicles, that sought to centralize situational awareness by combining as much of that data as possible in a control center in Algeciras, in Spain.

If a boat following a smuggler’s route reflected a radar ping back to one of these sensors, and an officer of one of Spain’s national police forces, the Guardia Civil, happened to be watching the screen, Spain could send a ship to intercept the boat. The Guardia Civil credits SIVE with nudging a larger share of Mediterranean migration to the central and eastern routes. The popularity of other routes, such as to Italy’s Lampedusa Island, 140 km from Tunis, and to Greece, some of whose islands are within sight of Turkish beaches, grew.

Disparities in border technology and the adaptability of migrants and their traffickers are among the reasons Europe decided to convert Frontex into a full agency and triple its budget and staff: “Frontex seeks to create cross-border collaboration in a situation which might otherwise result in a spending arms race on border control between E.U. Member States,” says geographer Dan Fisher of the University of Glasgow, who has published on SIVE.

Following Spain’s SIVE experience, in 2011 Frontex invited industry partners to demonstrate tethered surveillance balloons and now uses them in at least two locations to detect migrants who manage to get past its aerial and space-borne sensors. The present model of balloons can remain several hundred meters up in the air for up to 40 days, providing a persistent visual, infrared, and radar sensing capability across an area of around 11,310 square kilometers.

But that’s a tool of last resort, capable of monitoring people who are already on or near European land. First, migrants must cross the Mediterranean, which is dangerous. Jacob, like many Mediterranean migrants, made multiple attempts from several African countries to reach different European countries. He had limited information about the best way to Europe, but the traffickers who had captured him adapt all the time to changing border security situations. They, in turn, take advantage to coerce people like Jacob into working for them. “When I got to Libya, I didn’t have money,” Jacob says. “The traffickers who got me said, ‘Here you have to pay for your jail, then you pay for your journey.’ I told them, ‘I don’t have money, I have a mother who doesn’t work, I’m just a farmer, not a worker for a ministry or government.’”

While migrants and traffickers fight over the cost of their crossings, European entities have fought over how to stop migrant crossings. For more than a decade, European courts have documented and declared illegal routine European state border agency actions that include navy vessels pushing migrant boats back into international waters and abandoning them there.

In response to those rulings, European governments shifted tactics but not their goal: They began funding Libya and other North African governments through migration control and security pacts. Then European governments and Frontex began investing in tools to detect migrant boats before they reached European waters and calling the Libyan coast guard to pull back migrant boats.

After the 2015 border crisis and in the wake of court rulings against European countries’ “pushback” policies, Frontex, its sister agency the European Maritime Safety Agency (EMSA), and national border agencies invested in large, long-range drones to monitor slices of the Mediterranean, alongside crewed aircraft.

The E.U., through its main research funding program, also began calling on academia to help it make sense of all the new border data. For example, researchers at the Information Technologies Institute (ITI) in Thessaloniki, Greece, won grants as part of a consortia to fuse video data from uncrewed aerial vehicles, ground robots, and sea drones and use machine-vision software to flag likely items of interest, as seen in this demo video:

The systems send alerts to a dashboard accessible on a computer by border staff. The video system is similar to the security cameras that consumers install on their doorbells or in their homes to detect motion, but with a more sophisticated algorithm designed to detect migration and other illegal activity.

“The previous state of the art was that users had one pilot per drone. Our longer-term goal is to move toward operators managing several UxVs [uncrewed vehicles],” says Athanasios Kapoutsis, an ITI engineering researcher. One of the ways they will do that is by simplifying information from each data feed to prevent overwhelming the user. Rather than display raw footage from a camera or radar, or a dashboard with pages and pages of information, consortia members built an augmented-reality display that might suggest, for example, an 80 percent probability of a particular object being a boat. The group conducted its first real-world tests of the system’s ability to detect ships and humans in 2021.

Border authorities are also experimenting with using migrants’ electromagnetic signatures as tracking tools. It’s feasible because traffickers often hand a satellite phone to migrants and tell them to call for help once they are in international or European waters. Satellite phones emit signals that are detectable from space: In 2019, Frontex issued a contract for satellite-phone-detection services. Hawkeye 360, which also markets its service to the shipping industry and security services in countries such as the United States, was the only bidder and won the contract.

Crew members of the Royal Danish Air Force inspecting computer screens on board a Frontex aircraft, during a press day in December 2021. Sipa USA via AP

In 2021, Hawkeye 360’s satellite network offered downloads of their detection data a few times a day, using two clusters of three satellites. Now Hawkeye 360 is on track to have 30 satellites in orbit by early 2023, the company says. Frontex officials must have considered the technology promising, because in 2020 they issued another public tender for a new satellite-radio-detection contract. If the technology matures as fast as Hawkeye 360 predicts, border agencies might soon be capable of detecting satellite-phone-carrying migrant boats on a near-hourly basis, long before the boats leave the search-and-rescue zones of North African countries, without needing to keep aloft a fleet of aircraft or drones or sifting through hours of video feeds.

Social scientist Özgün Topak of York University, in Toronto, Canada, calls the shift to remote detection of migrants and cooperation with origin countries an adaptation strategy by European authorities that helps them comply with court orders and laws but continues to shift responsibility to other countries. In other words, European agencies are treating their direct contact with migrants as the problem, instead of treating the migrants’ precarious situations at sea or in North Africa as the problem. The tech is there to rescue Europe from the migrants, not the migrants from the sea.

Jacob says he spent somewhere between three and four years trying to migrate from Guinea to Spain. He eventually found work for someone Jacob describes as “connected to traffickers.”

“There are people who work for the traffickers for a year or so,” he says, to pay their way across. One day, Jacob’s contact told him it was his turn to try crossing. But getting in a boat isn’t the same as getting across.

The sea is hard, Jacob says, the boats insecure. The ones he used were made of a plastic that couldn’t endure many hours of exposure to seawater or the gasoline splashed on it by the rickety motor. The weather would change. “People panic from one minute to the next, the boat can have a problem, and then the Moroccan police can catch you at sea and take you back,” he recalls.

On one of those failed journeys, Moroccan police accused Jacob of being a trafficker himself, perhaps because when they asked for someone to help drive the boat back to shore, he agreed. The judge jailed him and couple of the others, then the police dropped them off in a different inland town.

Two men caught by Moroccan police after a failed crossing to Spain. Migrants say the police drove them far inland to delay subsequent attempts.

Migrants rest on the Algerian-Moroccan border.

Migrants often travel great distances overland before they board a boat to cross the Mediterranean Sea, exposing themselves to dangerous human traffickers and extreme heat.

If his boat had managed to reach international waters and called for help, passing merchant vessels might have taken its passengers, as they have hundreds of other migrants, and then called authorities for guidance on where to deliver them. That is the goal of most migrant boats: Their owners do not equip the boats for success in a full crossing. Passengers are lucky if they can carry more than their ID and mobile phones. The boats often have just enough fuel to get away from the North African coast but not enough to reach Europe.

Some people—not Jacob—also have reasonable grounds for requesting asylum in a safe country. International law protects those people, but if North African authorities capture them first, those people cannot ask European countries for asylum.

That may be why the E.U. funds a bevy of projects that aim to stop people from reaching international or European waters. The most cutting-edge involve looking past Europe’s borders. The European Space Agency has pitched using satellite imagery to look for migrant activity before people cross borders, Frontex uses aircraft overflying international portions of the Mediterranean for “early-warning” of potential crossers into European waters, and a since-canceled Frontex tender mentioned a “Pre-warning Mechanism” and forecasting irregular migration using social media. In other words, the border agency wants to see the future on the other side of its borders.

Just as the United States pressured Mexico to crack down on its border with Central American countries, Europe’s real goal may be to predict, detect, and prevent irregular migration from North Africa long before would-be migrants ever leave the shores of Tripoli, Libya’s capital. For example, one E.U.-funded research project published the results of using Google search data to predict international migration. Another set of E.U.-funded projects seek not only to understand perceptions of the E.U. by potential migrants, but also to reach across the border and change those perceptions.

They might not need to. Migrants, including Jacob, give their compatriots back home a clear-eyed perspective: “I tell people who want to cross that it’s very dangerous…it’s not easy like people imagine it in Morocco,” Jacob says.

Civil-society groups have argued that the European practice of calling North African countries to pick up remotely detected migrant boats sidesteps the law of the sea and the right to request asylum ( here and here). The present E.U. strategy may satisfy the narrow orders resulting from pushback-related court rulings in the early 2010s, but a raft of new cases may reshape Europe’s border policies all over again.

In 2019, human rights lawyers took a case against the E.U. to the International Criminal Court in The Hague, in the Netherlands, which most often addresses war crimes. That case is a long shot, says international law scholar David Fernández Rojo of the University of Deusto, in Bilbao, Spain, but it or similar cases could create a more open record of misbehavior and put moral pressure on the E.U. in response to its policies and actions.

For example, in May 2021 the Office of the United Nations High Commissioner for Human Rights reported multiple instances in which Frontex had shared surveillance information with Libyan authorities so that Libyan coast guards could pull migrant boats back to unsafe ports. While the office of the commissioner has limited power to act on its findings, it did recommend that the E.U. stop disembarking migrants in Libya and noted that Europe has an “obligation…to prevent mistreatment by third parties, including private actors or other States operating within their jurisdiction or effective control.”

While the main players in Europe’s border surveillance and enforcement are large companies and government agencies, it is easier than ever for armchair detectives to find and report on bad border behavior. One tool, called Alarm Phone, is an alliance between migrants and a civil-society group that invites embarked migrants to call or send messages requesting rescue during their sea crossings. The group then alerts maritime rescue authorities and the public, to hold the authorities responsible.

Civil-society groups are also turning to public data to police Europe’s border police. In late 2020 Bellingcat, an investigative group, used public ship and air tracking data together with videos from migrants to accuse Frontex of participating in illegal pushbacks led by the Greek coastguard. Another group, called Space-Eye, developed tools to detect migrant boats in publicly available satellite imagery. It claims to have corroborated at least one pushback using satellite data.

“If it wasn’t for civil society using technology you wouldn’t be hearing about this,” says David Hammond, founder and trustee of Human Rights at Sea, a civil-society organization in Havant, United Kingdom.

Disparities in border technology and the adaptability of migrants and their traffickers are among the reasons Europe decided to convert Frontex into a full agency and triple its budget and staff.

Academic researchers had been nibbling at satellite-powered migrant boat detection for years. Growing amounts of data and processing power mean that now even dilettante data scientists can write blog posts ( here and here) about coding your own boat-detection software. But ability isn’t the same as responsibility: At least one of those data scientists was competing in a public contest funded by Airbus, a major border-technology provider.

“When you deal with people and satellites, it’s always a problem because it’s a thin line between helping and surveillance,” says remote sensing researcher Urša Kanjir at the Research Center of the Slovenian Academy of Arts and Sciences in Ljubljana, Slovenia, who published an Acta Astronautica paper on the possibility of using Sentinel-2 satellite data for rescuing migrant boats.

It’s too early to tell whether sousveillance, or watching the authorities from below, will work in favor of migrants’ human rights or against them. But civil-society accusations of pushbacks have provoked scrutiny from several European oversight organizations: Europe’s antifraud office in January 2021 began investigating Frontex for its alleged participation in pushbacks, among other things. In June 2021 the European Ombudsman reported that Frontex had not implemented its recommendations on dealing with human rights complaints and recommended the agency provide more transparency in its interactions with civil-society groups. In July 2021 a European Parliament working group concluded that it lacked enough evidence to accuse Frontex of violating human rights, despite acknowledging the Bellingcat and other reports. It did say, however, that Frontex “did not prevent [European member state] violations, nor reduced the risk of future fundamental violations.” The series of investigations may have contributed to the European Parliament’s decision in October 2021 to withhold 12% of Frontex’s 2022 budget.

The border technology arms race isn’t a race to solve the causes of migration. Like any wall, it can only force potential migrants to think harder about how to get across and raise the stakes of failure. While European authorities have invested tens of millions of euros to make it harder for citizens to see how their border patrols push back against migration, whistle-blowing participants, disgruntled neighbors, errors, or mere contempt for international rule of law will reveal what even the most sophisticated technology might hide for a time.

Even if the destination countries detect them along the way, those migrants have rights enshrined in laws set by the destination countries that entitle them to rescue at sea and to apply for asylum in a safe country.

Outbursts of violence, economic precarity, and perhaps even climate change will motivate migrants despite the technological barriers. “The border policy is just a policy to bother migrants, but it will never stop them,” Jacob says. He says he would have preferred to work in a North African country than migrate all the way to Spain, but those countries’ unwillingness to offer legal residency to West Africans and their police brutality made it too dangerous to stay. One friend died after a fall—or being thrown—from a police station’s upper floor in Algeria, he says.

Now his lack of legal residency in Spain hobbles his earning power and prevents him from visiting home, but it is not as dangerous as it was in North Africa. Still, he is isolated from his family: His mother and brother have poor mobile connectivity, so he only has intermittent contact with them. “I don’t know when I’ll see my mother and brother. That’s one of the hardest things now,” Jacob says.

Having migrated through six countries and survived more attempted sea crossings than he can remember, Jacob says migrating to yet another country probably won’t improve his lot. “I don’t want my whole life to be running. I’ve run what I can run, from my country, crossing the sea. The day I’m tired I’ll return to my country.”

By then, thousands of other young people will have begun the same journey. Even if the destination countries detect them along the way, those migrants have rights enshrined in laws set by the destination countries that entitle them to rescue at sea and to apply for asylum in a safe country. Technology won’t change that, says Hammond, the human rights worker. Instead, he says, finding the right balance between the safety of migrants and the safety of the countries to which they want to travel, “has got to be dealt with at a geopolitical level.”

Editor's note: Article was updated on 14 February 2022 to more precisely describe ITI's research project.