Introduction

1Policy implementation in multi-level governance systems inevitably produces variation in how local authorities interpret and execute centrally-defined policies. While this phenomenon is widely recognized and has generated substantial scholarly discussion, we lack systematic tools for measuring and comparing the extent of such variation across different contexts and jurisdictions.

2The existing literature approaches this variation from two main perspectives. Some scholars emphasize the challenges of policy coordination that arise from local variation (Peters, 2018; Piattoni, 2010). Others focus on how such variation enables policy learning and innovation through processes of policy diffusion (Dolowitz & Marsh, 2000; Maggetti, 2022; Shipan & Volden, 2008). However, both streams of research have been limited by the absence of standardized methods for quantifying the degree of variation in local policy implementation.

3The need for systematic measurement extends beyond academic interest. Quantifying implementation variation – rather than merely documenting it through qualitative case studies – matters for several reasons. First, systematic measurement enables rigorous comparison across jurisdictions, time periods, and policy domains, transforming isolated observations into comparable data. It allows researchers to test hypotheses about the determinants and consequences of variation, moving beyond anecdotal evidence to establish empirically grounded patterns. Second, and perhaps most importantly, variation in local implementation may create significant disparities in citizens’ experiences of public services, raising fundamental questions about equity and the consistency of rights across territorial units. Without measurement tools, these disparities remain invisible to systematic analysis.

4This paper addresses this methodological gap by introducing the concept of “policy noise” – a novel metric for measuring variability in local policy implementation. Our primary contribution is the development of a set-theory-based measurement approach that enables systematic comparison of how local authorities exercise their discretion when addressing similar policy challenges. This measurement framework captures variation across three key dimensions: agenda-setting, implementation approaches, and resource allocation. The proposed methodology is deliberately designed to be descriptive rather than prescriptive, providing a foundation for empirical analysis without imposing normative judgments about optimal levels of variation.

5While we demonstrate the methodology through an analysis of Polish municipalities’ responses to Ukrainian refugee education needs, the policy noise framework is applicable across any domain where local authorities exercise implementation discretion within centrally-established parameters. Environmental policy, health services, social welfare, and education all present contexts where similar applications would be valuable. The migration policy application illustrates the utility of the methodology, not its limitations.

6The paper proceeds as follows. First, we develop the theoretical framework for policy noise and situate it within existing literature on policy implementation and multi-level governance. Second, we present our detailed methodology for measuring policy noise, including the prerequisite of structural equivalence among comparison units. Third, we demonstrate the application of this methodology through our case study of Polish municipalities, engaging with domain-specific migration scholarship to contextualize the findings. Fourth, we discuss theoretical and practical implications of our findings, including the methodology’s broader applicability. Finally, we outline promising directions for future applications and research.

Theoretical Framework

7The concept of noise, as explored by Kahneman, Sibony, and Sunstein in their book “Noise: A Flaw in Human Judgment” (2021) refers to the variability in judgments that should ideally be consistent. Although noise is generally considered a negative or unwanted phenomenon, the authors clearly suggest that a certain level of noise can benefit the system that produces it. From our perspective, two issues are important: first, noise can be a by-product of an imperfect process that people may embrace as it makes decisions less “algorithmic” and therefore considers actual circumstances; second, noise “might be essential to accommodate new values and hence to allow moral and political evolution” (p. 328). We propose examining variations in decisions and policies across different administrative bodies through this prism.

8Policy implementation in multi-level governance systems inherently involves variation in how policies are interpreted and executed across different jurisdictional units. This variation stems from several structural features of modern governance systems. These features include the institutional autonomy of local authorities, which is grounded in the principles of subsidiarity and local self-governance. They also include discretionary power in policy implementation, which allows for adaptation to local conditions. Finally, they include varying local capacities and resources, which affect implementation capabilities. While some degree of variation is both inevitable and potentially beneficial, it poses challenges to policy coordination and equitable service delivery.

9Building on Kahneman et al.’s (2021) conceptualization of noise as variability in judgments, we define policy noise as “measurable variability in local policy implementation decisions when addressing similar problems under similar structural conditions.” This definition comprises three essential components. First, measurable variability refers to quantifiable differences in policy choices and implementation approaches, allowing for systematic comparison across units. Second, problem similarity indicates policy challenges that share core characteristics requiring a local response and enabling meaningful cross-unit analysis. Third, structural equivalence encompasses comparable institutional frameworks and resource parameters that establish a baseline for comparison.

10Policy noise represents systematic rather than random variation, reflecting the deliberate exercise of local discretion within established policy frameworks. This systematicity manifests across multiple dimensions. For instance, local authorities may interpret policy goals differently, leading to varying priorities and objectives. They may also select different implementation tools from available options, resulting in diverse approaches to similar challenges. Moreover, they often make distinct resource allocation decisions, reflecting local priorities and constraints.

Why Policy Noise as a Distinct Concept?

11While existing frameworks capture important aspects of policy implementation dynamics, none provides a systematic method for quantifying the extent of horizontal variation among comparable units addressing similar challenges. Distinguishing between horizontal and vertical variation is fundamental to understanding the unique contribution of policy noise. Implementation gap analyses focus on vertical deviation – the distance between central policy intent and local outcomes. Policy noise, by contrast, captures horizontal variation – differences among implementing units at the same governmental level. This horizontal focus matters because units may all successfully meet central objectives (no vertical gap) while achieving compliance through markedly different pathways (high horizontal noise).

12Measuring horizontal variation serves distinct analytical purposes. First, it enables the identification of the range of approaches that local authorities consider legitimate within a given policy framework. Second, it reveals where local discretion manifests most strongly across different implementation dimensions. Additionally, it provides an empirical foundation for examining whether diverse approaches lead to different outcomes – a question that cannot be addressed without first quantifying the diversity itself.

13The significance of comparing structurally equivalent units merits explicit attention. Variation among dissimilar units conflates structural constraints with discretionary choices – a rural municipality with limited administrative capacity and an urban center with extensive professional staff will inevitably differ in implementation, but this difference reflects resource realities rather than policy preferences. Structural equivalence isolates the discretionary component of variation. This is analogous to experimental control: by holding constant what can reasonably be held constant, we can observe variation attributable to the exercise of local discretion rather than structural necessity.

14Policy noise relates to and builds upon several established concepts in public policy and public administration literature while maintaining its distinct analytical focus. Understanding these relationships helps position policy noise within the broader theoretical landscape and clarifies its unique contribution to policy analysis.

15The implementation gap concept, developed by Pressman and Wildavsky (1984), focuses on discrepancies between policy intent and outcomes. While implementation gaps measure vertical deviation from central policy objectives, policy noise captures horizontal variation across implementing units. This distinction is crucial as policy noise may exist even when implementation closely matches central intent, provided different local units achieve this alignment through varying approaches.

16Street-level bureaucracy theory (Lipsky, 2010) examines how frontline workers exercise discretion in policy implementation. While this concept focuses on individual-level decision-making, policy noise operates at the organizational level, capturing systematic variations in how local authorities as institutions approach policy challenges. The individual discretion analyzed in street-level bureaucracy theory may contribute to policy noise, but the latter concept encompasses broader organizational and systemic factors.

17Policy translation, as conceptualized by Freeman (2009), explores how policies are reinterpreted and adapted as they move between contexts. Policy noise shares an interest in variation but focuses specifically on quantifying implementation differences rather than analyzing the process of reinterpretation. This quantitative focus makes policy noise particularly useful for comparative analysis across jurisdictions.

18The concept of policy diffusion, studied extensively by scholars including Berry and Berry (2018), examines how policies spread across jurisdictions. While policy diffusion tracks the adoption of specific innovations, policy noise measures existing variation in implementation approaches, regardless of their origin. Understanding policy noise patterns may help explain why certain innovations diffuse more successfully in some contexts than in others.

19Finally, research on policy variation in federal systems, as discussed by Jeffery et al. (2014), examines systematic differences in policy outcomes across federal units. Policy noise complements this work by focusing specifically on implementation decisions rather than outcomes, helping illuminate the mechanisms through which structural differences translate into varying policy results.

20These theoretical relationships are summarized in Table 1, which outlines the primary focus, unit of analysis, and key questions addressed by each concept, as well as their relationship to policy noise.

Table 1. Theoretical landscape: policy noise in relation to established public policy concepts.

Concept	Primary focus	Unit of analysis	Key question	Relationship to policy noise
Policy noise	Variability in implementation decisions across local units	Local administrative units	How much do implementation approaches vary when addressing similar problems?	Core concept - measures the degree of variation in local policy implementation
Implementation gap	Deviation from policy intent	Policy program	Does implementation match design?	Policy noise may help explain implementation gaps by revealing variation in local approaches
Street-level bureaucracy	Individual discretion in implementation	Individual bureaucrats	How do frontline workers adapt policy?	Individual decisions contribute to policy noise at the organizational level
Policy diffusion	Pattern of policy adoption	Jurisdictional units	How do policies spread?	Policy noise may facilitate or hinder diffusion by revealing variation in implementation approaches
Policy translation	Process of policy adaptation	Policy content	How are policies reinterpreted?	Translation processes generate policy noise through local reinterpretation
Policy variation in federal \systems	Systematic differences in policy outcomes across federal units	Federal sub-units (states/regions)	How and why do policy outcomes differ across federal units?	Complementary - policy variation can be seen as one manifestation of policy noise; focuses on outcomes rather than implementation decisions

Source: the Authors.

21Policy noise differs from these related concepts in three key respects. First, unlike the implementation gap which measures vertical deviation from central intent, policy noise captures horizontal variation among structurally equivalent units – it can exist even when all units successfully meet central objectives through different pathways. Second, while policy diffusion tracks the adoption of innovations over time, policy noise provides a cross-sectional measurement of existing implementation diversity at a given moment. Third, whereas research on policy variation in federal systems typically examines outcome differences across constitutionally distinct units, policy noise specifically measures implementation decisions within comparable institutional contexts.

The Analytical Value of Policy Noise

22The analytical value of policy noise is significant for both theoretical understanding and practical policy management in multi-level governance systems. First, by providing a quantitative measure of implementation variation, policy noise enables a systematic assessment of how local autonomy manifests in practice. This measurement allows policymakers and researchers to move beyond anecdotal evidence of variation to establish empirically grounded patterns of local policy implementation.

23Second, policy noise enables systematic comparison across jurisdictions, time periods, and policy domains. Such comparability is crucial for understanding whether observed variations represent isolated cases or reflect broader systemic patterns. This comparative capability helps identify where variation might be productive (indicating successful local adaptation) versus potentially problematic (suggesting coordination failures or inequitable service delivery).

24Third, policy noise facilitates the analysis of variation patterns, helping to distinguish between random fluctuations and systematic differences in implementation approaches. This distinction is crucial for policy learning as it helps identify cases where variation stems from deliberate adaptation to local conditions rather than implementation inconsistencies. Understanding these patterns can inform decisions about which local innovations might be suitable for broader adoption.

25Fourth, measuring policy noise provides an evidence base for decisions about the appropriate balance between centralization and local autonomy. High levels of policy noise in areas where consistency is crucial might suggest a need for greater central coordination. Conversely, where variation appears to foster innovation without compromising essential service standards, maintaining or expanding local discretion might be justified. This empirical foundation can help move debates about centralization versus autonomy beyond ideological positions to evidence-based decision-making.

26Fifth, policy noise measurement has important equity implications. Variation in local implementation may create significant disparities in citizens’ experiences of public services across different jurisdictions. By quantifying such variation, policy noise measurement provides a diagnostic tool for identifying where such disparities exist and how pronounced they are. This enables targeted attention to areas where variation may be undermining service equity.

27The measurement of policy noise thus serves both analytical and practical purposes, offering insights for researchers studying policy implementation while providing policymakers with concrete data to inform governance decisions. This dual relevance makes it a valuable addition to the toolkit of policy analysis and public administration. Having established the theoretical foundations and significance of policy noise, we now turn to the practical question of its measurement.

Methodology

28The measurement of policy noise builds directly on Kahneman et al.’s (2021) work on noise in judgment and decision-making. Their fundamental insight – that variation in judgments can be systematically measured and analyzed – provides the foundation for our methodological approach. Just as Kahneman et al. developed methods to quantify noise in individual judgments, we propose a framework for measuring variation in local policy implementation decisions.

29However, measuring noise in policy implementation presents unique challenges compared to measuring noise in individual judgments. While Kahneman et al. could often rely on standardized numerical scales to quantify judgment variation, policy decisions frequently involve complex, multidimensional choices that resist simple numerical representation. Moreover, unlike individual judgments made under controlled conditions, policy implementations occur in dynamic institutional contexts where some degree of variation may be not only inevitable but desirable.

Measurement Prerequisites: Structural Equivalence

30The meaningful measurement of policy noise requires comparison among units that share fundamental structural characteristics. This requirement of structural equivalence does not demand identical conditions but rather comparable institutional frameworks, resource parameters, and policy mandates that establish a valid baseline for comparison.

31Existing literature on policy implementation documents that municipal responses vary significantly by structural factors including city size, administrative capacity, economic resources, and geographic position (Eckhard et al., 2021; Steelman et al., 2021). These structural differences create legitimate variation – a small rural municipality cannot reasonably be expected to implement policy identically to a major metropolitan center. However, such structural variation is distinct from the discretionary variation that policy noise aims to capture.

32To isolate policy noise – variation attributable to discretionary choices rather than structural constraints – comparison units should share comparable administrative capacity and professionalization, similar legal frameworks and policy mandates, roughly equivalent resource bases relative to the policy challenge, and exposure to similar external conditions relevant to the policy domain. This does not require perfect identity across all characteristics but sufficient similarity that observed differences more plausibly reflect policy choices than structural necessities. The selection of comparison units is therefore a critical methodological decision that should be explicitly justified in any application of the framework.

33Future applications of this methodology could productively stratify analysis by structural characteristics, comparing noise levels among different structural tiers. This would enable examination of whether structural context affects not just implementation approaches but also the degree of variation among similar units.

Temporal Considerations

34Policy implementation is inherently dynamic, evolving over time in response to changing circumstances, political climates, and social needs. This dynamic nature has important implications for measurement. A single-wave assessment captures variation at one moment, which may reflect temporary conditions, crisis responses, or transitional phases rather than stable patterns. Early-stage implementation might show high variation as localities experiment, with convergence occurring as best practices diffuse. Alternatively, initial conformity might give way to increasing divergence as local adaptations accumulate.

35For comprehensive analysis, we recommend conducting measurements in multiple waves to observe temporal patterns. This longitudinal approach would enable distinction between stable variation (reflecting enduring local preferences or structural differences) and transient variation (reflecting different stages of policy development or response to particular events). Such temporal analysis is particularly important when studying crisis responses, where initial emergency measures may differ substantially from subsequent institutionalized approaches.

Three Dimensions of Measurement

36To address the challenge of quantifying complex policy decisions, we develop a measurement approach that captures multiple dimensions of policy implementation while acknowledging the inherent complexity of local governance. Our methodology identifies three key areas where measurable variation may occur, each grounded in established theoretical traditions in public policy analysis.

37The first dimension we examine is agenda-setting variation. Following Kingdon (1984), we recognize that how local authorities define and prioritize problems significantly influences their policy responses. Just as Kahneman et al. found that different judges might assess the same case differently, local authorities may vary in whether and how they formally acknowledge policy challenges, reflecting differences in problem recognition and political commitment.

38Implementation approach variation constitutes our second analytical dimension. Drawing on Salamon’s (2002) tools of government approach, we recognize that local authorities have access to multiple policy instruments and may select different combinations of these tools to address similar challenges.

39Resource allocation variation forms our third dimension of analysis. Building on Wildavsky and Caiden’s (2004) work on budgetary politics, we acknowledge that financial decisions reflect policy priorities and implementation strategies in concrete terms.

40Building on Kahneman et al.’s framework, we recognize that measuring noise is most meaningful when examining variability in contexts where some degree of consistency might reasonably be expected. In policy implementation, this prerequisite manifests in two essential conditions. First, there must exist a multi-level governance structure where higher-level authorities establish baseline expectations for policy decisions. This creates a framework within which local implementation choices can be meaningfully compared. Second, the policy problems must be sufficiently complex to allow for multiple viable solutions, creating legitimate space for local variation in implementation approaches.

41These conditions are frequently met in contemporary governance systems, particularly in public service delivery. Central governments typically establish legal frameworks and guarantee basic service standards while granting local authorities discretion in implementation methods and organizational approaches. This arrangement creates an ideal context for measuring and analyzing policy noise.

42However, adapting Kahneman et al.’s measurement approach – which relies primarily on the standard deviation of quantitative judgments – to policy implementation presents methodological challenges. Unlike the numerical decisions studied in their work, local policy choices often manifest as complex combinations of regulations, programs, and organizational arrangements that resist straightforward quantification. To address this challenge, we have developed a set-theory-based measurement approach that systematically translates qualitative policy features into numerical values.

43Our method involves identifying specific domains where local authorities might diverge from established baselines, treating these domains as sets that represent distinct aspects of policy implementation. Through careful analysis of available data, we can quantify each jurisdiction’s position relative to these sets using values between 0 (complete non-membership) and 1 (full membership). This fuzzy-set approach allows us to capture nuanced variations in policy implementation while maintaining analytical rigor.

44To identify sets in which decisions taken by local authorities can “deviate” from a certain baseline, we draw on the classical policy cycle concept (Howlett & Ramesh, 1995). We propose to look at sets defined by agenda-setting and two aspects of policy implementation.

45For agenda-setting, an interesting set can consist of local authorities that officially declare their readiness to address an issue with the tools at their disposal (consequently, authorities which do not declare such readiness remain outside of the set).

46For policy implementation, we can identify two separate sets:

(1) The types of solutions set: a set of local authorities which implement all possible types of solutions at their disposal to address a certain issue (therefore, the ones who implement only part of the possible range of solutions are partially in the set).

(2) Resources allocated set: a set of local authorities which direct the highest possible proportion of financial resources at their disposal to address that issue (therefore, the ones who direct smaller proportion of financial resources are partially in the set).

Values can be assigned according to the following procedure.

47For agenda-setting set, the score is determined by an administrative body’s public commitment to address a specific policy area. For instance, if Municipality A publicly commits to pioneering and innovative educational strategies, it earns a score of 1. Conversely, if Municipality B lacks any formal declaration of intent in this area, it receives a score of 0.

48For policy implementation/types of solutions set, measuring score involves evaluating the diversity of solutions an administrative body has implemented within a given policy domain. First, all potential solution types implemented in the population of administrative bodies under scrutiny (denoted as N) must be identified. For each administrative body, the ratio of implemented solution types (n) to the total identified solutions (N) must be calculated. For example, if Municipalities A and B are assessed for educational initiatives and three types of solutions are identified (e.g., free textbooks, individual tutoring, and student psychological support), N=3. If Municipality A has implemented two of these (n(A) = 2), its score would be 2/3. If Municipality B has implemented one (n(B) = 1), its score would be 1/3.

49For policy implementation/resources allocated set, measuring score reflects the proportion of financial resources dedicated by an administrative body to a specific policy area (r) compared to the maximum allocation observed in the population (R). For example, Municipality A allocates 3% and Municipality B 2% of their budgets to education. The highest allocation in this case is 3%, and therefore R = 3; r(A) = 3 and r(B) = 2. The score for municipality A would be 1 (as 3/3 = 1) and Municipality B’s score would be 2/3.

50The following table can be used to calculate each municipality’s total score and identify the standard deviation, which is a measure of policy noise.

Table 2. Framework for calculation of policy noise.

	Agenda-setting score	Policy implementation score – types of solutions	Policy implementation score – resources allocated	Total score
Administrative body 1
Administrative body 2
…
Standard deviation (noise measurement)

Source: the Authors.

Example of Measuring Policy Noise: Response of Polish Municipalities to Ukrainian Refugees

Situating the Case Within Migration Scholarship

51Migration and refugee policy provides an ideal domain for demonstrating the policy noise methodology. Over the past two decades, migration scholarship has documented a significant “local turn” in integration governance, with cities emerging as important actors alongside – and sometimes in tension with – national governments (Zapata-Barrero et al., 2017; Scholten & Penninx, 2016). Research has shown that municipalities often develop approaches that diverge substantially from national frameworks, whether through “sanctuary” policies that expand protections beyond national requirements or through restrictive local measures that create additional barriers (Bazurli et al., 2022; Caponio & Borkert, 2010).

52This local variation in migration governance has been richly documented through qualitative case studies across European cities (Ambrosini, 2013, 2018). Scholars have identified mechanisms including “insurgent asylum policy-making” where cities actively contest national exclusion (Bazurli & Kaufmann, 2022), “latent hybridity” where local administrators temporarily abandon bureaucratic routines to cope with crisis demands (Eckhard et al., 2021), and the formation of local “assemblages” incorporating civil society actors to fill gaps in state provision (Vera Espinoza et al., 2021).

53However, while the “local turn” literature has established that variation exists and has identified its sources, the field has lacked quantitative tools for systematically measuring its extent. How much variation exists? In which dimensions is it most pronounced? How does variation in one municipality compare to variation in another? These questions require measurement approaches that move beyond documenting that local variation exists to measuring how much variation exists. Policy noise measurement addresses this methodological gap, enabling researchers to quantify implementation diversity and examine its patterns systematically.

The Polish Context

54The 2022 Russian invasion of Ukraine precipitated a forced migration crisis of unprecedented speed and scale for Poland. Over eight million border crossings were recorded within the first year, with more than one million Ukrainian refugees establishing residence in Poland (Duszczyk et al., 2023). This placed immediate and substantial demands on local governance systems.

55Poland is a unitary state with constitutionally guaranteed local self-government, creating a multi-level governance context where national legislation establishes frameworks while municipalities retain implementation discretion. The Special Act of March 2022 (Ustawa z dnia 12 marca 2022 r.) provided the legal architecture for refugee response, but crucially, many provisions created permissive rather than mandatory frameworks. Article 12.4, for instance, stipulates that municipalities “may” provide housing and meals, creating discretionary space for local variation.

56The initial response was characterized by substantial grassroots mobilization that preceded formal coordination mechanisms (Fomina & Pachocka, 2024). Civil society organizations, volunteers, and local governments improvised reception systems while awaiting clarification of national policy. This created conditions where local approaches diverged based on prior experience, available resources, civil society density, and political leadership – precisely the conditions where policy noise would be expected.

57Warsaw emerged as the primary destination city, hosting approximately 469,000 refugees, while other major cities including Wrocław (190,000), Kraków, and Łódź also received substantial populations (Union of Polish Metropolises, 2022). However, refugees also settled in a “dispersed model” across medium-sized industrial cities, particularly in southern and western Poland, following pre-existing labor migration networks established over the previous decade (Śleszyński, 2022).

Structural Equivalence in the Polish Case

58This analysis examines the 12 member cities of the Union of Polish Metropolises. These municipalities satisfy the structural equivalence requirement for policy noise measurement on several grounds.

59All are large urban centers with populations exceeding 200,000, ensuring comparable administrative capacity and professionalization. All operate as self-governing urban municipalities (gminy) under identical national legal frameworks for both education and refugee policy. Their shared membership in the Union of Polish Metropolises indicates similar levels of institutional development and professional networks; the Union facilitates information exchange and peer learning, reducing variation attributable to information asymmetries.

60All 12 municipalities faced the same national policy framework following the Special Act, with identical legal tools and funding mechanisms available. All experienced significant refugee arrivals, though the magnitude varied. While this variation in refugee numbers represents a structural difference, we address it through the resource allocation measure, which examines spending as a proportion of education budgets rather than absolute amounts.

61This structural similarity does not eliminate all confounding factors but establishes conditions under which observed variation more plausibly reflects discretionary policy choices rather than structural constraints.

Measurement Application

62To illustrate our proposed method of measuring policy noise, we analyze data on the solutions designed and implemented by these 12 municipalities in response to the influx of war refugees from Ukraine. This scenario meets the criteria for policy noise assessment as it involves a complex issue (Spencer, 2018) and highlights the variability in local responses to refugee inflows (van Breugel & Scholten, 2017; Oliver et al., 2020; Scholten, 2018).

63Migration policies are typically governed by central governments, but cities play a significant role in this policy domain (Kaufmann & Strebel, 2021; Özdemir, 2022; Sanyal, 2012). This creates a multi-level governance setting in which central authorities establish a baseline policy, and cities have the autonomy to design their own solutions.

64For this exercise, we focus on one crucial aspect of migration policy: providing refugees with access to education.

Agenda-Setting Measurement

65To determine which municipalities are included in the agenda-setting set, we utilize a database of local laws collected by the project team. Some municipalities adopted resolutions outlining their pledged support for refugee education, which is a discretionary act rather than a mandatory requirement. The table below reflects the municipalities’ declarations of support in this area.

Table 3. Calculation of agenda-setting score.

Municipality	Agenda-setting score
Białystok	0
Bydgoszcz	0
Gdańsk	1
Katowice	0
Kraków	1
Lublin	0
Łódź	1
Poznań	1
Rzeszów	1
Szczecin	0
Warszawa	1
Wrocław	1

Source: the Authors.

Interpretation of Agenda-Setting Variation (SD = 0.49)

66The high standard deviation of 0.49 indicates substantial variation in formal political commitment to addressing refugee education. A total of seven municipalities (58%) adopted formal declarations, while five (42%) did not. This variation is notably higher than in the other dimensions, suggesting that political framing and formal agenda-setting are areas of maximum local discretion.

67The pattern reveals that formal commitment is neither universal nor random. Warsaw, as the primary gateway city with accumulated experience in immigrant integration and established municipal structures for diversity policy, would be expected to have formal frameworks. Similarly, cities with progressive political leadership such as Gdańsk and Poznań, adopted formal commitments. Thus, the political symbolism of formal declarations – signaling priority and commitment – varies substantially across structurally similar municipalities. The declaration decision may reflect political leadership preferences, prior experience with migration policy, or strategic communication choices rather than actual service provision differences. High noise in agenda-setting combined with lower noise in implementation (discussed below), suggests a decoupling between political rhetoric and practical action.

Types of Solutions Measurement

68To assess how municipalities belong to the set of policy implementations and types of solutions, we use data from the Union of Polish Metropolises. This data contains self-reports from municipalities on the solutions they designed and implemented to address the challenge of mass migration from Ukraine after the outbreak of full-scale war in February 2022. For this analysis, we consider only systemic solutions (i.e. we exclude ad hoc actions designed to mitigate the initial crisis when refugees first arrived, focusing instead on solutions deliberately designed with the long-term goal of integrating newcomers into the community) that municipalities have independently initiated. Table 4 presents the calculation of the scores based on the diversity of solutions implemented.

Interpretation of Solution Types Variation (SD = 0.22)

69The moderate standard deviation of 0.22 indicates less variation in practical implementation than in formal agenda-setting. This pattern is analytically significant: while political commitments varied substantially, actual service provision showed greater convergence.

70The range of scores (0.125 to 0.75) indicates that no municipality implemented all eight solution types identified across the population. This suggests that municipalities made deliberate choices about which intervention types to prioritize rather than attempting comprehensive coverage. The highest-scoring municipalities (Wrocław and Warsaw at 0.75) implemented six of eight solution types, demonstrating broad but not exhaustive approaches.

71Different emphasis patterns are visible in the data. Some municipalities prioritized “soft” support (psychological services, educational assistance) while others emphasized “hard” infrastructure (educational facilities, material support). Some invested heavily in workforce development for educators working with refugee students. These different emphases may reflect local assessments of needs, available resources, or existing institutional strengths.

72The lower variation in this dimension compared to agenda-setting suggests that practical constraints and peer learning may produce convergence in implementation even when political framing diverges. Municipalities facing similar challenges and exchanging information through networks like the Union of Polish Metropolises may gravitate toward similar solution portfolios regardless of their formal declarations.

Resource Allocation Measurement

73To evaluate municipalities’ belonging to the policy implementation/resources allocated set, we analyze financial data from their 2022 reports. We focus on expenditures not mandated by the central government. Table 5 presents the calculation of scores based on the proportion of financial resources dedicated to refugee education.

Table 5. Calculation of Policy Implementation/resource allocation score (in PLN).

Municipality	Total spendings on education and educational care in 2022	Estimation of spendings on education-related solutions caused by the influx of refugees from Ukraine	Proporion of crisis solutions to total spendings (r); R=3,1%		Resources allocated score
Białystok	973 693 118,09	12 755 594,21	r(Białystok)	1,3%	0,42
Bydgoszcz	828 377 421,16	24 940 070,32	r(Bydgoszcz)	3,0%	0,96
Gdańsk	1 420 157 208,65	33 142 204,00	r(Gdańsk)	2,3%	0,74
Katowice	823 366 355,69	22 517 929,67	r(Katowice)	2,7%	0,87
Kraków	2 359 431 675,56	60 354 392,42	r(Kraków)	2,6%	0,82
Lublin	1 042 807 361,74	16 783 791,84	r(Lublin)	1,6%	0,51
Łódź	1 536 620 254,09	8 953 547,50	r(Łódź)	0,6%	0,19
Poznań	1 576 584 501,09	41 959 774,00	r(Poznań)	2,7%	0,85
Rzeszów	713 590 813,85	10 684 565,56	r(Rzeszów)	1,5%	0,48
Szczecin	968 530 479,39	25 393 704,00	r(Szczecin)	2,6%	0,84
Warszawa	5 750 882 006,38	180 377 053,89	r(Warszawa)	3,1%	1,00
Wrocław	2 004 054 875,38	38 105 291,00	r(Wrocław)	1,9%	0,61

Source: the Authors.

Interpretation of Resource Allocation Variation (SD = 0.24)

74The moderate standard deviation of 0.24 in resource allocation closely parallels the solution types variation, reinforcing the pattern of greater convergence in implementation than in agenda-setting.

75The range of proportional spending (0.6% to 3.1% of education budgets) indicates meaningful fiscal variation. Warsaw’s position at the maximum (3.1%) reflects its role as the primary destination city and its substantial refugee population. However, proportional measures partially control for this – the measure captures priority within each municipality’s own resource constraints rather than absolute capacity.

76Important interpretive cautions apply to this dimension. Higher spending does not necessarily indicate superior outcomes; it may reflect larger refugee populations, different cost structures, less efficient service delivery, or different accounting practices. Similarly, lower spending may indicate under-investment, successful mainstreaming of refugees into existing services (reducing the need for dedicated expenditure), or different categorization of expenses across budget lines.

77The fiscal data quality also varies across municipalities, as noted in the methodology limitations. While total education spending figures are reliable, the estimation of crisis-specific expenditures required interpretation of varying reporting formats. These figures should be treated as informed estimates rather than precise measurements.

Combined Measurement

78Finally, we combine the scores from the agenda-setting set, types of solutions set, and resources allocated set to measure the overall policy noise. Table 6 presents this combined measurement, providing a comprehensive view of the variability in local responses to the refugee education issue.

Table 6. Measurement of policy noise.

	Agenda-setting score	Types of solutions score	Resources allocated score	Total score
Białystok	0	0,25	0,42	0,67
Bydgoszcz	0	0,625	0,96	1,58
Gdańsk	1	0	0,74	1,74
Katowice	0	0,5	0,87	1,37
Kraków	1	0,5	0,82	2,32
Lublin	0	0,375	0,51	0,89
Łódź	1	0,375	0,19	1,56
Poznań	1	0,5	0,85	2,35
Rzeszów	1	0,25	0,48	1,73
Szczecin	0	0,125	0,84	0,96
Warszawa	1	0,75	1,00	2,75
Wrocław	1	0,75	0,61	2,36
Standard deviation	0,49	0,22	0,24	0,63

Source: the Authors.

Interpretation of Total Variation (SD = 0.63)

79The highest variability observed in the total score (SD = 0.63), which integrates agenda-setting, types of solutions, and resource allocation, indicates that when these dimensions combine, the overall approach to handling refugee education varies significantly among municipalities. Despite their shared membership in a coordinating network, identical national legal frameworks, and comparable administrative capacity, these 12 metropolitan municipalities developed substantially different overall approaches to the same policy challenge.

80Total scores range from 0.67 (Białystok) to 2.75 (Warsaw), a fourfold difference that reflects fundamentally different policy configurations. High-scoring municipalities like Warsaw (2.75), Wrocław (2.36), and Poznań (2.35) combined formal political commitment, diverse solution portfolios, and substantial resource allocation. Lower-scoring municipalities like Białystok (0.67), Lublin (0.89), and Szczecin (0.96) showed limited formal commitment, narrower solution ranges, and lower proportional spending.

81This substantial total variation carries dual implications requiring separate normative assessment.

82From a policy learning perspective, high variation represents rich opportunities for comparative analysis and identification of effective practices. Different municipalities effectively conducted natural experiments with different approaches, creating evidence about what works under what conditions. This variation provides a foundation for peer learning and evidence-based policy diffusion.

83From an equity perspective, however, the variation raises concerns. A Ukrainian refugee child’s access to educational support, such as psychological services, language instruction, and integration activities, may differ markedly depending on the Polish city in which their family settles. The variation in total scores suggests that structurally similar municipalities made substantially different choices about the priority and comprehensiveness of their responses. Whether these differences represent appropriate local adaptation or problematic inconsistency in the protection of vulnerable populations is a normative question that the metric itself does not resolve.

Interpretive Framework

84The policy noise metric provides empirical measurement but requires contextual interpretation. The table below offers guidance on how noise levels might be interpreted depending on the analytical lens applied:

Table 7. Interpretive framework for policy noise levels.

Noise Level	Innovation/Learning Lens	Coordination/Equity Lens
High	Productive experimentation; diverse local adaptations enable identification of best practices	Coordination failure; service disparities may disadvantage citizens in some jurisdictions
Low	Homogenization; reduced space for local innovation and adaptation	Consistent delivery; equity across jurisdictions in citizen experience
Variable across dimensions	Strategic prioritization; municipalities focus discretion where it matters most	Mixed implementation capacity; some dimensions better coordinated than others

Source: the Authors.

85The appropriate interpretive lens depends on the policy domain and stakes involved. For experimental programs where optimal approaches remain uncertain, high noise may be desirable because it generates evidence about alternatives. However, for fundamental rights and essential services – including the education of refugee children – excessive variation may be problematic regardless of its innovation potential. Most policy domains involve both considerations, requiring balanced interpretation that the metric itself cannot provide.

86The dimensional pattern observed in this case – high noise in agenda-setting, moderate noise in implementation – suggests a stratified structure of local discretion. Political commitment and formal framing represent areas of maximum local variation, while practical service delivery shows greater convergence. This may indicate that symbolic politics varies more than operational practice, or that peer learning and professional norms produce convergence in implementation even when political contexts diverge.

Discussion

Summary of Findings

87Our analysis of the responses of 12 Polish metropolitan municipalities to Ukrainian refugee education needs reveals a distinctive pattern of policy noise. Agenda-setting variation proved highest (SD = 0.49), indicating that formal political commitment is the area with the most local discretion. Implementation variation in both solution types (SD = 0.22) and resource allocation (SD = 0.24) was more moderate, suggesting greater convergence in practical measures than in political framing. The combined total variation (SD = 0.63) indicates substantial overall divergence in municipal approaches despite shared structural characteristics and coordinating networks.

88This dimensional pattern – higher variation in political commitment than in practical implementation – represents a potentially significant finding about the structure of local discretion in multi-level governance systems.

Contribution to Existing Literature

89The policy noise framework advances policy implementation methodology in several respects. First, it provides a systematic, replicable method for quantifying variation that enables comparison across cases. While qualitative case studies have richly documented local variation in policy implementation, such documentation has been difficult to cumulate or compare systematically. The policy noise metric enables researchers to make statements such as “variation in domain X exceeds variation in domain Y” or “municipality A’s approach differs more from the mean than municipality B’s” – claims that require quantification.

90Second, the three-dimensional structure of the measurement (agenda-setting, solution types, resource allocation) enables analysis of where variation is strongest. A unidimensional measure would not reveal the finding that political commitment varied more than practical implementation.

91Third, by requiring explicit attention to structural equivalence, the methodology distinguishes between variation attributable to structural constraints and variation attributable to discretionary choice. This distinction is essential for policy-relevant conclusions: variation caused by resource constraints calls for different responses than variation caused by political preferences.

92For migration scholarship specifically, the methodology addresses a recognized gap. While the “local turn” literature has established that cities are autonomous actors in integration governance (Bazurli et al., 2022; Scholten & Penninx, 2016), it has lacked the tools to measure the extent of this autonomy systematically. Policy noise measurement enables the shift from documenting the existence of local variation to measuring its extent and concentration.

Theoretical Implications

93The observed pattern – high variation in agenda-setting, moderate variation in implementation – suggests a stratified model of local discretion that merits theoretical development. Several mechanisms might explain this pattern.

94First, symbolic politics may allow for more local variation than operational practice. Formal declarations and political framing carry symbolic weight but may be relatively costless compared to service provision changes. Local political leaders may find it easier to differentiate themselves through distinctive rhetoric than through distinctive services.

95Second, peer networks and professional norms may produce convergence in implementation. The municipalities studied share membership in the Union of Polish Metropolises, which facilitates information exchange. Professional networks among educators and administrators may diffuse best practices, creating convergence at the operational level even when political contexts differ.

96Third, the pattern may reflect different constraints on different types of decisions. Resource allocation and solution selection may be more constrained by available options, funding mechanisms, and administrative capacity. Political declarations face fewer constraints of this kind.

97These mechanisms connect to the broader literature on “decoupling” in organizational theory – the gap between formal structures and actual practices. The policy noise data suggest that in this context, such decoupling may operate in reverse: formal political positions may vary more than actual practices, rather than formal structures existing without corresponding practice.

Practical Implications

98The policy noise framework has implications for different stakeholder groups.

99For central governments, policy noise measurement provides a diagnostic tool for identifying areas where coordination may be needed. High noise in domains where equity is crucial (such as fundamental rights of vulnerable populations) may signal the need for stronger central guidance or more specific mandates. Low noise in areas where innovation is desirable may suggest excessive standardization. Dimensional analysis helps target interventions. For example, if agenda-setting shows high noise but implementation shows convergence, central efforts might focus on aligning political commitment rather than prescribing operational details.

100For local authorities, comparative data enables peer learning. Municipalities can examine the solution portfolios that their peers have implemented, the resource levels that others have committed, and how their overall approach compares. This facilitates the identification of alternatives that can be adapted locally. The Union of Polish Metropolises already facilitates such an exchange; systematic measurement enhances the evidence base for these discussions.

101For researchers, policy noise measurement enables hypothesis testing about both the determinants and consequences of variation. Which institutional, political, or contextual factors predict noise levels? Does higher noise correlate with better or worse outcomes? Does the relationship between noise and outcomes differ across policy domains? These questions become empirically tractable with quantified variation measures.

102For evaluators, policy noise provides a baseline for assessing policy changes over time. If a new central initiative aims to increase coordination, the metric enables before-and-after comparison of noise levels. If local experimentation is encouraged, the metric can track whether variation increases as intended.

Limitations

103Several limitations deserve acknowledgment.

104First, data quality sensitivity affects measurement reliability. In our application, legislative data (used for agenda-setting) proved most reliable as it involves formal, verifiable documents. Financial data required estimation from varying reporting formats, introducing measurement error. Solution type categorization involved interpretive judgment that could affect reproducibility. High-quality, standardized data collection would improve measurement precision.

105Second, single-wave measurement captures a snapshot that may become outdated as policies evolve. The 2022 response to Ukrainian refugees represented an emergency phase; subsequent institutionalization may have altered patterns. A longitudinal approach would enable distinction between stable variation and transitional differences.

106Third, domain-specific operationalization is required for each application. The three dimensions (agenda-setting, solution types, resource allocation) are general, but their specific indicators must be defined for each policy domain. This limits immediate transferability while maintaining the framework’s adaptability.

107Fourth, assumptions of structural equivalence require careful justification for each application. We argued that the 12 Polish metropolises share sufficient structural similarity for meaningful comparison, but this claim involves judgment. Critics might argue that substantial differences in refugee numbers across cities undermine equivalence. Measuring spending as a proportion of budgets partially addresses this but does not fully eliminate structural confounding.

108Fifth, the methodology measures inputs and processes rather than outcomes. Policy noise quantifies variation in what municipalities do, not what they achieve. High-scoring municipalities are not necessarily more effective – they may simply be more active. The relationship between noise and outcomes requires separate analysis.

Future Research Directions

109Several research directions emerge from this framework.

110Longitudinal applications would track noise evolution over crisis phases and examine whether initial variation converges or diverges over time. The Polish case could be revisited at multiple intervals to observe whether the 2022 pattern persists, converges toward common approaches, or diverges further.

111A cross-domain comparison would examine whether noise levels differ systematically across policy domains. For example, is implementation variation greater in migration policy than in environmental policy or health services? Do domains with stronger central mandates exhibit less noise? Such comparison would reveal where local discretion manifests most strongly.

112Determinant studies would examine which institutional, political, and contextual factors predict noise levels. Do municipalities with particular political configurations show systematically higher or lower variation? Does participation in coordinating networks reduce noise? Do historical experiences with similar challenges affect current variation?

113Outcome studies would examine whether noise level correlates with service quality, citizen satisfaction, or policy effectiveness. Does variation produce better outcomes through experimentation, worse outcomes through inconsistency, or is there no systematic relationship? Such analysis is essential for drawing normative conclusions about optimal variation levels.

114Comparative national studies would examine whether the framework produces consistent results across different national contexts. Poland’s unitary state structure with constitutional local self-government is one configuration; federal systems, highly centralized states, or systems with different central-local relations might produce different noise patterns.

Conclusion

115In this paper, we introduced the concept of “policy noise” and developed a novel methodological framework for measuring variability in local policy implementation. The proposed set-theory-based approach offers three key innovations in policy analysis. First, it provides a systematic way to quantify the qualitative aspects of policy actions through distinct measurement dimensions: agenda-setting, implementation approaches, and resource allocation. Second, it establishes a standardized methodology for comparing policy implementation across structurally equivalent jurisdictions, enabling both cross-sectional and longitudinal analyses. Third, it creates a foundation for investigating the relationships between local autonomy and policy outcomes without imposing normative judgments about the desirability of variation.

116The methodology’s practical utility was demonstrated through an analysis of how Polish municipalities responded to the educational needs of Ukrainian refugees. The analysis revealed notable variations in local approaches, with agenda-setting commitments showing the highest variability and practical implementation showing greater convergence. This dimensional pattern suggests a stratified structure of local discretion worthy of further investigation. However, the case study primarily served to illustrate the methodology’s capacity to capture and measure complex policy variations in real-world settings.

117The policy noise framework is applicable across domains in which local authorities exercise implementation discretion within centrally-established parameters. In environmental policy, for example, it could measure how municipalities vary in their implementation of climate adaptation strategies or enforcement of environmental regulations. In health policy, it could capture differences in local public health responses. This potential application was vividly demonstrated during the COVID-19 pandemic, when local variation in the implementation of national health guidelines created substantial differences in citizen experiences. In social services, the framework could quantify differences in how localities implement welfare-to-work programs or social care provisions. Each application would require domain-specific operationalization of the three measurement dimensions, but the underlying framework is transferable across contexts.

118Looking ahead, this methodological framework opens several promising avenues for future research. It enables systematic investigation of the factors driving policy variation across different contexts and policy domains. It provides a tool for examining how different governance structures and institutional arrangements influence local policy implementation. Perhaps most importantly, it offers a foundation for studying the relationship between policy variation and outcomes, helping to answer crucial questions about when variation contributes to policy innovation and adaptation versus when it creates problematic inconsistencies in public service delivery.

119The measurement of policy noise thus represents a methodological advance in our ability to study local governance dynamics. By providing a systematic way to quantify policy implementation variation, it bridges a crucial gap between theoretical discussions of local autonomy and empirical analysis of its manifestations. As this methodology is applied and refined across different contexts, it promises to enhance our understanding of multi-level governance and contribute to more evidence-based discussions about the balance between local discretion and central coordination in public policy.