Recent scholarship on technology and security has emphasized the importance of algorithmic systems as enacted through relations between human and nonhuman actors (Aradau and Blanke, 2015; Bellanova et al., 2021; Hoijtink and Leese, 2019; Suchman, 2006). Sociotechnical systems act in "co-production" (Jasanoff, 2004), as "actants" in a network (Latour, 2005), or in "intra-action" (Barad, 2007). In these understandings, technology forms an ontological assemblage, in which human agency is tied in with the sociomaterial arrangements of which it is part. Humans and non-humans, technological objects and infrastructures, all populate complex, sometimes messy networks where the boundaries between entities are enacted in situated practices (Haraway, 1988). This conception of technology "draws attention to the fact that these relations are not a given but that they are constructed — and thereby relates them back to cultural imaginaries of what technology should look like and how it should be positioned vis-à-vis humans and society" (Leese, 2019: 45)

In this context, how can we understand the characteristics and effects of security systems built on the analysis of sensor data through “deep learning,” and the new security politics that they introduce? On the technical level, the novelty of “algorithmic security vision” does not lie in the sensors themselves, but in the new abilities of “artificial intelligence software” (McCosker and Wilken, 2020). The promise of the systems is that the multiplication of the sensors and modalities of knowing, the ability to create information feeds that are under the scrutiny of automated systems means that data collection and data analysis are no longer separated; surveillance can happen in real time, capturing life as it unfolds, so that the operators can act on hotspots, clusters, or the moods and emotions of a crowd (Andrejevic and Burdon, 2015; Isin and Ruppert, 2020).

To make sense of such developments, Suchman’s concept of configuration is a useful methodological “toolkit.” It helps “delineating the composition and bounds of an object of analysis” (Suchman, 2012: 48) and allows us to conceptualize algorithmic security vision as heterogeneous assemblages of human and nonhuman elements whose agency is "an effect of practices that are multiply distributed and contingently enacted" (Suchman, 2006: 267). We are interested here in a framework that underscores "how the entities that come into relation are not given in advance, but rather emerge through the encounter with one another" (van de Ven and Plájás, 2022: 52).

Suchman also draws our attention to the “ways in which technologies materialize cultural imaginaries, just as imaginaries narrate the significance of technical artefacts” (2012: 48). For Suchman, “configuration” is a tool for “studying technologies with particular attention to the imaginaries and materialities that they join together” (2012: 48). The configuration of humans and machines is constructed through discourse and practice, which, drawing on Haraway, she conceptualizes as “figurations.” Sociotechnical systems thus do not exist without their intended uses and users. Such discourses are an important part of individual experience, collective professional practices, and narratives about technology. Technologies bring together elements from various registers into stable material-semiotic arrangements. Those configurations draw attention to the political effects of everyday practices and how they institute bounded entities and their relations. If we take Suchman’s suggestion that algorithmic security vision is complex and multiple, how can we get to "know" it as an object of research, while acknowledging its partiality? When taking the coming together of algorithms, vision and (in)security as configuring imaginaries and practices in heterogeneous and complex networks, how can we explore their politics?

Suchman defines figuration as “action that holds the material and the semiotic together in ways that become naturalized over time, and in turn requires ‘unpacking’ to recover its constituent elements” (2012: 49). The first step in her methodology therefore requires us to “reanimate the figure at the heart of a given configuration, in order to recover the practices through which it comes into being and sustains its effects.”

In her work, Suchman has used a variety of methods of inquiry to “reanimate the figure.” Qualitative interviews and ethnography have been instrumental in producing the raw material for the analysis. In this paper, we expand the methodological toolkit envisaged by Suchman to multimodal methods that go beyond text to capture the materiality of imaginaries and practices. We explore the epistemic possibilities of capturing figurations as both semiotic and material traces.

The result of our theoretical and methodological quest is a tool that allows us to produce “time-based diagramming.” We use this method for both elicitation and multimodal data collection. We presented our participants with a large digital tablet, and asked them to draw a diagram while answering our questions. Ruben van de Ven programmed an interface that could play back the recorded conversation in drawing and audio. The participants could not delete or change their drawings, so their hesitations and corrections remained. The ad hoc figuring out of the participants’ descriptions thus remains part of the recording.¹ In the phase of data analysis, the software allows the diagrams to be annotated, creating short clips. The diagrams thus enable a practice of combination and composition (O’Sullivan, 2016), providing for a material-semiotic support to analyze various imaginaries of algorithmic security vision.

Diagramming is a key method in the field of technology, most notably in the conceptualization and design of computational practices (Mackenzie, 2017; Soon and Cox, 2021: 221). We don’t assume that the materiality of the drawings brings us any closer to the materiality of the actors’ practices, which are of a different order. Our interest is in the possibilities offered by the diagrams: they are composed of elements that are not necessarily similar, but are connected by their mere appearance on the same plane, thus allowing heterogeneous elements to co-exist. Diagrams are composed of parts that can be separated and recombined in different ways, creating new formations and expressions (O’Sullivan, 2016). Using such a multimodal tool seemed a pertinent methodological setup to capture figurations and configurations (see van de Ven and Plájás, 2022).

We interviewed twelve professionals who developed, deployed or contested computer vision technologies in the field of (in)security.² We asked them to describe the coming together of computer, vision and (in)security from their professional vantage points.³ In what follows, we focus on three figurations and two configurations that emerged from the diagrams.

Diagram 2. Sergei Miliaev distinguishes three sources of training data for facial recognition technologies.

In most of our conversations, algorithmic security vision is understood to involve a particular subset of algorithms: deep neural networks.⁴ Such a machine learning-based vision brings to the fore one key dimension of security practices: the question of training, and the ability to “see.” Training has been assumed as part of the discussion around the socialization of security professionals (Amicelle et al., 2015; Bigo, 2002), and algorithmic systems (Fourcade and Johns, 2020) but scant attention has been paid to how training elaborates upon and incorporates specific sets of skills.

Some authors have explored the way in which the “seeing” of security agents is trained at the border, by building on the literatures on “skilled vision” (Maguire et al., 2014) or “vision work” (Olwig et al., 2019). Maguire mobilized work in anthropology that locates human vision as a “embodied, skilled, trained sense” (Grasseni, 2004: 41) that informs standardized practices of local “communities of vision” (see also Goodwin, 1994). Skilled vision is useful in that it draws attention to the sociomaterial circumstances under which vision becomes a trained perception (Grasseni, 2018: 2), and how it becomes uniform in communities through visual apprenticeship. This literature examines the production of “common sense” by taking training, exercise, peer monitoring and other practices of visual apprenticeship as locus of attention. Yet these works fail to capture the specificities of the type of machine learning we encountered in our research. How then is visual apprenticeship reconfigured under algorithmic security vision?

In our conversations, the “training of the algorithms,” figures as a key stake of algorithmic security vision. The participants in our diagram interviews explained how deep learning algorithms are trained on a multiplicity of visual data which provides the patterns a system should discriminate on. In Diagram 2, Sergei Miliaev, head of the facial recognition research team at VisionLabs in Rotterdam illustrated this point.

Miliaev distinguishes three sources for training images: web scraping, “operational” data collected through its partners or clients, and “synthetic” data. The first two options, Miliaev argues, have some limitations. Under European data protection regulation it is very difficult to obtain or be allowed to use data “from the wild” because it is often illegal to collect data of real people in the places where the algorithm will be used. Additionally, partners sometimes resist sharing their operational footage outside of their own digital infrastructures. Finally, when engineering a dataset, one cannot control what kind of footage is encountered in “the wild.” This has led to the emergence of a new phenomenon: training data generated in the lab.

Synthetic training data is often collected by acting in front of a camera. We see this in the case of intelligent video surveillance (Intelligente Videoüberwachung) deployed in Mannheim since 2018. Commenting on this case, chief of police (Polizeidirektor) Dirk Herzbach explains that self-defense trainers imitated 120 body positions to create the annotated data used to train the behavior recognition technology. In another example, Gerwin van der Lugt, developer of software that detects violent behavior, stated that given the insufficiency of data available, they “rely on some data synth techniques,” such as simulating violent acts in front a green screen. Sometimes even the developers, computer scientists or engineers themselves re-enact certain movements or scenes for training their algorithms. In Diagram 3, two developers involved in the project at seaside boulevard in Scheveningen give a striking example of how such enactments of suspicious events require the upfront development of a threat model that contains visual indicators that distinguish threat (a positive detection) from non-threat (a negative detection). The acting of the developers embeds these desirable and undesirable traits into the computer model.

Diagram 3. Two developers involved in a project at the seaside boulevard in Scheveningen describe the use of computer vision to distinguish the legal use balloons from their illegal use for inhaling the nitrous oxide gas.

Sometimes the meaning of “synthetic” or “fake” data is pushed. Sergei Miliaev explains how in the context of highly sensitive facial recognition algorithms, software companies use faces generated entirely through artificial neural networks to train their algorithms. Miliaev mentions Microsoft’s DigiFace-1M (Bae et al., 2023), a training dataset containing one million algorithmically generated faces. Such synthetic training sets complicate the borders between sensor-originated and other types of images. In the use of artificially generated images, one GPU generates bytes that are interpreted by another. Algorithmic vision occurs without direct reference to people or things that live outside electronic circuits.

These technical developments offer a changing figure of the skilled vision of security that calls for new research directions. While these technologies are still in their infancy, our interviewees see it as a token of “better” and “fairer” technology that can circumvent racial bias, as any minority can be generated to form an equal distribution in the training dataset (Stevens and Keyes, 2021). But with an emerging concern for algorithmic hallucination (Ji et al., 2023), glitches or undesirable artifacts in the generated data, one wonders what kind of vision is trained using such collections. Learning from synthetic data thus produces an internalized vision, providing insights by circulating data through a chain of artificial neural networks. While appearing in new technological assemblages, the processing of images to form archetypes is reminiscent of the composite photographs created by Galton (1879). His composites were used to train police officers to identify people as belonging to a particular group, circulating and reinforcing the group boundaries based on appearance (Hopman and M’charek, 2020). Which boundaries does “fake” or synthetized training data perpetuate? Skilled vision shifts attention to the negotiations that happen before algorithmic vision is trained, such as how algorithmic vision depends on access to data and regulations around data protection.

With the use of synthetic data, the question of a "community of lookers" — the embodied social and material practices through which apprenticeship is perpetuated — appears in a new light. Such a community becomes more dispersed as generative models circulate freely online. For instance, a generative model from Microsoft, trained on images shared online, is used for training an authentication system in the Moscow Metro system. Such models are informed by communities of looking from which their training data is sourced, and the norms of that platform. These norms then circulate with the model and become "plugged-in" to other systems. Algorithmic vision, trained on synthetic data, is thus a composable vision, in which different sources of training data mobilize imagery from all kinds of aesthetic apprenticeships. The cascading of generative and discriminative models thus reshapes security practices. Furthermore, to comprehend changes in the politics of vision, attention to the training of vision, as a moment of standardization and operationalization, could be extended to the training of security professionals.

Conversations with practitioners revealed yet another dimension of the figure of vision in flux: its relation to time and movement. Deep learning-based technologies distinguish themselves from earlier algorithmic security systems based on their status as prediction models, which by definition raises questions on the temporal dimensions of their processing (Sudmann, 2021). Yet, how algorithmic security vision reconfigures temporalities has yet to receive scholarly attention in CSS and related disciplines. While literature on border studies has located border security in multiple places and temporalities (e.g. Bigo and Guild, 2005), scholarship on image-based algorithmic security practices have often focused on a photography-centric paradigm: biometric images (Pugliese, 2010) facial, iris and fingerprint recognition (Møhl, 2021), and body scanners (Leese, 2015). These technologies capture immutable features of suspect identities. In the diagrams, however, vision appears less static. Instead, two central dimensions of the figure of vision appear: the ability to capture and make sense of the movement of the bodies in a fixed space, and the movement of bodies across spaces.

On the first point, we notice increasing attention to corporeality, how physical movements render certain individuals suspicious. This process takes place through the production and analysis of motion by composing a sequence of frames. Gerwin van der Lugt, who helped develop a violence detection algorithm at Oddity.ai, stresses how “temporal information integration” is the biggest technical challenge in detecting violence in surveillance footage: a raised hand might be either a punch or a high-five. In Diagram 4, van der Lugt visualizes the differences between the static and dynamic models. A first layer of pose or object detection often analyzes a merely static image. Oddity.ai then uses custom algorithms to integrate individual detections into one that tracks movement. It is then the movement that can be assessed as violent or harmless. From these outputs, Oddity.ai runs “another [...] process that [they] call temporal information integration—it’s quite important—to [...] find patterns that are [even] longer.” This case illustrates how algorithmic security vision temporarily attributes risk to bodies, in accordance with the ways violence is imagined and choreographed in the training data.

Diagram 4. Top: Frame 1 is processed by YOLO, an object detection model, producing Output 1 (O1). Other frames are processed independently. Bottom: Frames 1 to 10 are combined for processing by the customized model (“M”), where it produces outputs (O 1-10, O 11-20). These outputs are then processed in relation to one another by the temporal information integration to find body patterns over longer periods. Drawn by Gerwin van der Lugt.

Our interviewees figured movement in a second way. Bodies are tracked in space, leading to an accumulation of suspicious data over time. Ádám Remport explained how facial recognition technology (FRT) works by drawing a geographical map featuring building blocks and streets. In this map (Diagram 5), Person A could visit a bar, a church, or an NGO.

Diagram 5. Ádám Remport explains how a person’s everyday routes can be inferred when facial recognition technology is deployed in various sites, montaging a local, photographic vision into spatio-temporal, cinematic terms.

If “FRT is fully deployed and constantly functioning,” explains Remport, people can be “followed wherever [they] go.” Remport’s drawing therefore suggests that in this setting it is not important to be able to identify the person under surveillance; what matters is that this person can be tracked over different surveillance camera feeds. The trajectories of bodies and their “signature” marked through the reconstruction of their habitual movements through space are used as a benchmark for the construction of suspicion. Cinematic vision is thus made possible thanks to the broader infrastructure that allows for the collection and analysis of data over longer periods of time, and their summarization through montage.

The emerging centrality of movement thus opens up a new research agenda for security, focused not on who and what features are considered risky, but when, and through which movements specific bodies become suspicious. While earlier studies on biometric technologies have located the operational logic on identification, verification, authentication, thus knowing the individual (Ajana, 2013; Muller, 2010), figuring algorithmic security vision as cinematic locates its operational logic in the mobility of embodied life (see Huysmans, 2022). While many legal and political debates revolve around the storage of images as individual frames, and the privacy issues involved, less is known about the consequences of putting these frames into a sequence on a timeline and the movements that emerge through the integration of frames over time.

Our third emerging figuration concerns the place of the error. A large body of literature examines actual and speculative cases of algorithmic prediction based on self-learning systems (Azar et al., 2021). Central to these analyses is the boundary-drawing performed by such algorithmic devices, enacting (in)security by rendering their subjects as more- or less-risky others (Amicelle et al., 2015: 300; Amoore and De Goede, 2005; Aradau et al., 2008; Aradau and Blanke, 2018) based on a spectrum of individual and environmental features (Calhoun, 2023). In other words, these predictive devices conceptualize risk as something produced by, and thus external to, security technologies.

In this critical literature on algorithmic practices, practitioners working with algorithmic technologies are often critiqued for understanding software as “sublime” (e.g. Wilcox, 2017: 3). However, in our diagrams, algorithmic vision appears as a practice of managing error. The practitioners we interviewed are aware of the error-prone nature of their systems but know it will never be perfect, and see it as a key metric that needs to be acted upon.

The most prominent way in which error figures in the diagrams is in its quantified form of the true positive and false positive rates, TPR and FPR. The significance and definition of these metrics is stressed by CTO Gerwin van der Lugt (Diagram 6). In camera surveillance, the false positive rate could be described as the number of fales positive classifications relative to the number of video frames being analyzed. Upon writing down these definitions, van der Lugt corrected his initial definitions, as these definitions determine the work of his development team, the ways in which his clients — security operators — engage with the technology, and whether they perceive the output of the system as trustworthy.

Diagram 6. Gerwin van der Lugt corrects his initial definitions of the true positive and false positive rates, and stresses the importance of their precise definition.

The figuration of algorithmic security vision as inherently imprecise affects the operationalization of security practices. Van der Lugt’s example concerns whether the violence detection algorithm developed by Oddity.ai should be trained to categorize friendly fighting (stoeien) between friends as “violence” or not. In this context, van der Lugt finds it important to differentiate what counts as false positive in the algorithm’s evaluation metric from an error in the algorithm’s operationalization of a security question.

He gives two reasons to do so. First, he anticipates that the exclusion of stoeien from the category of violence would negatively impact TPR. In the iterative development of self-learning systems, the TPR and FPR, together with the true and false negative rates must perform a balancing act. Van der Lugt outlines that with their technology they aim for fewer than 100 false positives per 100 million frames per week. The FPR becomes indicative of the algorithm’s quality, as too many faulty predictions will desensitize the human operator to system alerts.

This leads to van der Lugt’s second point: He fears that the exclusion of stoeien from the violence category might cause unexpected biases in the system. For example, instead of distinguishing violence from stoeien based on people’s body movements, the algorithm might make the distinction based on their age. For van der Lugt, this would be an undesirable and hard to notice form of discrimination. In developing algorithmic (in)security, error is figured not merely as a mathematical concept but (as shown in Diagram 6) as a notion that invites pre-emption — a mitigation of probable failure — for which the developer is responsible. The algorithmic condition of security vision is figured as the pre-emption of error.

Diagram 7. By drawing errors on a timeline, van Rest calls attention to the pre-emptive nature of error in the development process of computer vision technologies.

According to critical AI scholar Matteo Pasquinelli, “machine learning is technically based on formulas for error correction” (2019: 2). Therefore, any critical engagement with such algorithmic processes needs to go beyond citing errors, “for it is precisely through these variations that the algorithm learns what to do” (Amoore, 2019: 164), pushing us to reconsider any argument based on the inaccuracy of the systems.

The example of stoeien suggests that it is not so much a question if, or how much, these algorithms err, but how these errors are anticipated and negotiated. Thus, taking error as a hallmark of machine learning we can see how practices of (in)security become shaped by the notion of mathematical error well beyond their development stages. Error figures centrally in the development, acquisition and deployment of such devices. As one respondent indicated, predictive devices are inherently erroneous, but the quantification of their error makes them amenable to "risk management.”

While much has been written about security technologies as a device for risk management, little is known about how security technologies are conceptualized as objects of risk management. What happens then in this double relation of risk? The figure of the error enters the diagrams as a mathematical concept, throughout the conversations we see its figure permeate the discourse around algorithmic security vision. By figuring algorithmic security vision through the notion of error, risk is placed at the heart of the security apparatus.

The first configuration is related to the evaluation of the error rate in the training of algorithmic vision systems: it involves datasets, benchmark institutions, and the idea of fairness as equal representation among different social groups. Literature on the ethical and political effects of algorithmic vision has notoriously focused on the distribution of errors, raising questions of ethnic and racial bias (e.g. Buolamwini and Gebru, 2018). Our interviews reflect the concerns of much of this literature as the pre-emption of error figured repeatedly in relation to the uneven distribution of error across minorities or groups. In Diagram 8, Ádám Remport draws how different visual traits have often led to different error rates. While the general error metric of an algorithmic system might seem "acceptable," it actually privileges particular groups, which is invisible when only the whole is considered. Jeroen van Rest distinguishes such errors from the inherent algorithmic imprecision in deep machine learning models, as systemic biases (Diagram 7), as they perpetuate inequalities in the society in which the product is being developed.

To mitigate these concerns and manage their risk, many of our interviewees who develop and implement these technologies, externalize the reference against which the error is measured. They turn to a benchmark run by the American National Institute of Standards and Technology (NIST), which ranks facial recognition technologies by different companies by their error metric across groups. John Riemen, who is responsible for the use of forensic facial recognition technology at the Center for Biometrics of the Dutch police, describes how their choice for software is driven by a public tender that demands a "top-10" score on the NIST benchmark. The mitigation of bias is thus outsourced to an external, and in this case foreign, institution.

Diagram 8. Ádám Remport describes that facial recognition technologies are often most accurate with white male adult faces, reflecting the datasets they are trained with. The FPR is higher with people with darker skin, children, or women, which may result in false flagging and false arrests.

We see in this outsourcing of error metrics a form of delegation that brings about a specific regime of (in)visibility. While a particular kind of algorithmic bias is rendered central to the NIST benchmark, the mobilization of this reference obfuscates questions on how that metric was achieved. That is to say, questions about training data are invisibilized, even though that data is a known site of contestation. For example, the NIST benchmark datasets are known to include faces of wounded people (Keyes, 2019). The Clearview company is known to use images scraped illegally from social media, and IBM uses a dataset that is likely in violation of European GDPR legislation (Bommasani et al., 2022: 154). Pasquinelli (2019) argued that machine learning models ultimately act as data compressors: enfolding and operationalizing imagery of which the terms of acquisition are invisibilized.

Attention to this invisibilization reveals a discrepancy between the developers and the implementers of these technologies. On the one hand, the developers we interviewed expressed concerns about how their training data is constituted to gain a maximum false positive rate/true positive rate (FPR/TPR) ratio, while showing concern for the legality of the data they use to train their algorithms. On the other hand, questions about the constitution of the dataset have been virtually non-existent in our conversations with those who implement software that relies on models trained with such data. Occasionally this knowledge was considered part of the developers' intellectual property that had to be kept a trade secret. A high score on the benchmark is enough to pass questions of fairness, legitimizing the use of the algorithmic model. Thus, while indirectly relying on the source data, it is no longer deemed relevant in the consideration of an algorithm. This illustrates well how the invisibilization of the “compressed” dataset, in Pasquinelli’s terms, into a model, with the formalization of guiding metrics into a benchmark, permits a bracketing of accountability. One does not need to know how outcomes are produced, as long as the benchmarks are in order.

The configuration of algorithmic vision’s bias across a complex network of fragmented locations and actors, from the dataset, to the algorithm, to the benchmark institution reveals the selective processes of (in)visibilization. This opens up fruitful alleys for new empirical research: What are the politics of the benchmark as a mechanism of legitimization? How does the outsourcing of assessing the error distribution impact attention to bias? How has the critique of bias been institutionalized by the security industry, resulting in the externalization of accountability, through dis-location and fragmentation?

A second central question linked to the delegation of accountability is the configuration in which the security operator is located. The effects of delegation and fragmentation in which the mitigation of algorithmic errors is outsourced to an external party, becomes visible in the ways in which the role of the security operator is configured in relation to the institution they work for, the software’s assessment, and the affected publics.

The public critique of algorithms has often construed the human-in-the loop as one of the last lines of defense in the resistance to automated systems, able to filter and correct erroneous outcomes (Markoff, 2020). The literature in critical security studies has however problematized the representation of the security operator in algorithmic assemblages by discussing how the algorithmic predictions appear on their screen (Aradau and Blanke, 2018), and how the embodied decision making of the operator is entangled with the algorithmic assemblage (Wilcox, 2017). Moreover, the operator is often left guessing at the working of the device that provides them with information to make their decision (Møhl, 2021).

What our participants’ diagrams emphasized is how a whole spectrum of system designs emerges in response to similar questions, for example the issue of algorithmic bias. A primary difference can be found in the degree of understanding of the systems that is expected of security operators, as well as their perceived autonomy. Sometimes, the human operator is central to the system’s operation, forming the interface between the algorithmic systems and surveillance practices. Gerwin van der Lugt, developer of software at Oddity.ai that detects criminal behavior argues that “the responsibility for how to deal with the violent incidents is always [on a] human, not the algorithm. The algorithm just detects violence—that’s it—but the human needs to deal with it.”

Dirk Herzbach, chief of police at the Police Headquarters Mannheim, adds that when alerted to an incident by the system, the operator decides whether to deploy a police car. Both Herzbach and Van der Lugt figure the human-in-the-loop as having full agency and responsibility in operating the (in)security assemblage (cf. Hoijtink and Leese, 2019).

Some interviewees drew a diagram in which the operator is supposed to be aware of the ways in which the technology errs, so they can address them. Several other interviewees considered the technical expertise of the human-in-the-loop to be unimportant, even a hindrance.

Chief of police Herzbach prefers an operator to have patrol experience to assess which situations require intervention. He is concerned that knowledge about algorithmic biases might interfere with such decisions. In the case of the Moscow metro, in which a facial recognition system has been deployed to purchase tickets and open access gates, the human-in-the-loop is reconfigured as an end user who needs to be shielded from the algorithm’s operation (c.f. Lorusso, 2021). On these occasions, expertise on the technological creation of the suspect becomes fragmented.

These different figurations of the security operator are held together by the idea that the human operator is the expert of the subject of security, and is expected to make decisions independent from the information that the algorithmic system provides.

Diagram 9. Riemen explains the process of information filtering that is involved in querying the facial recognition database of the Dutch police.

Other drivers exist, however, to shield the operator from the algorithm’s functioning, challenging individual expertise and acknowledging the fallibility of human decision making. In Diagram 9, John Riemen outlines the use of facial recognition by the Dutch police. He describes how data from the police case and on the algorithmic assessment is filtered out as much as possible from the information provided to the operator. This, Riemen suggests, might reduce bias in the final decision. He adds that there should be no fewer than three humans-in-the-loop who operate independently to increase the accuracy of the algorithmic security vision.

Instead of increasing their number, there is another configuration of the human-in-the-loop that responds to the fallibility of the operator. For the Burglary-Free Neighborhood project in Rotterdam, project manager Guido Delver draws surveillance as operated by neighborhood residents, through a system that they own themselves. By involving different stakeholders, Delver hopes to counter government hegemony over the surveillance apparatus. However, residents are untrained in assessing algorithmic predictions raising new challenges. Delver illustrates a scenario in which the algorithmic signaling of a potential burglary may have dangerous consequences: “Does it invoke the wrong behavior from the citizen? [They could] go out with a bat and look for the guy who has done nothing [because] it was a false positive.” In this case, the worry is that the erroneous predictions will not be questioned. Therefore, in Delver’s project the goal was to actualize an autonomous system, “with as little interference as possible.” Human participation or “interference” in the operation are potentially harmful. Thus, figuring the operator — whether police officer or neighborhood resident — as risky, can lead to the relegation of direct human intervention.

By looking at the figurations of the operator that appear in the diagrams we see multiple and heterogeneous configurations of regulations, security companies, and professionals. In each configuration, the human-in-the-loop appears in different forms. The operator often holds the final responsibility in the ethical functioning of the system. At times they are configured as an expert in sophisticated but error-prone systems; at others they are figured as end users who are activated by the alerts generated by the system, and who need not understand how the software works and errs, or who can be left out.

These configurations remind us that there cannot be any theorization of “algorithmic security vision,” both of its empirical workings and its ethical and political consequences without close attention to the empirical contexts in which the configurations are arranged. Each organization of datasets, algorithms, benchmarks, hardware and operators has specific problems. And each contains specific politics of visibilization, invisibilization, responsibility and accountability.