Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies

This study was funded by National Science Foundation under the title "Shale Policy Conflict and Cooperation in the U.S." (Award ID # 1734294/1734310.), and "RCN: Coordinating and Advancing Analytical Approaches for Policy Design" (Award ID # 1917908). We thank Ramiro Berardo, Federico Holm, Saba Siddiki, Hongtao Yi, Jill Yordy, and Jongeun You for helping collect the data and contributing to its broader research project. We appreciate the IRPP editors' and anonymous reviewers' assistance in improving the article's arguments.


Introduction 1
For decades, comparative studies of policymaking have tackled diverse topics ranging from taxonomies of policy instruments and tools (Howlett, 2019;Fernández-I-Marin et al., 2021) to the comparative analysis of agenda setting and policy change (Epp, 2018;Baumgartner et al., 2018).The approaches continue to evolve and change as we ask new questions and refine methods and theories.This paper continues this trajectory in comparative analyses of policymaking by seeking answers to the following research question: How does the content of public policy (in this case, legislative bills) change from introduced to passed versions within and across U.S. states? 2 Our question offers a novel direction in the study of policymaking.We build on ideas found in literature on policy formulation and design (e.g., Jordan & Turnpenny, 2015), the institutional grammar (e.g., Siddiki et al., 2019), and comparative public policy (e.g., Dodds, 2018), and traverse several unexplored areas of this literature.For example, we examine policy formulation by comparing changes in introduced versus passed legislation.We do not open the black box of policy formulation; instead, we deepen the understanding by examining what enters the black box (in the initial policy proposal) and what comes out (in the final policy version).
By "black box," we mean the unknown internal workings of the policy formulation process, which takes place between the input and output of legislative effort.Theories such as policy diffusion and punctuated equilibrium also approach the black box without unpacking what is in the box (Baumgartner et al., 2018;Berry & Berry, 2018).Instead, these theories infer the black box from the inputs and outputs.This study complements the existent theories to help grasp the policy formulation process through reliable measures of the output of legislative effort through the changes resulting from the legislative process.
Moreover, we compare the content of the final policy versions across six states and 105 bills, lending insights into the variation of policy adoption across U.S. states.Regarding the institutional grammar, we offer a rare large-n application based on semi-automated techniques, using a dictionary-based approach to measure the composition changes in proposed and passed versions of policies (Heikkila & Weible, 2018).The institutional grammar represents one of the most recent innovations in public policy research.This paper builds on a growing literature taking on the challenge of applying the institutional grammar to large samples of public policies and drawing meaning from such applications (e.g., Weible et al., 2020;Heikkila et al., 2021;Chen et al., 2022).This paper substantively focuses on oil and gas development involving hydraulic fracturing.The emergence of hydraulic fracturing ("fracking") has led to the rapid expansion of shale oil and gas development in the United States, especially in locations unaccustomed to this form of resource extraction (see Weible et al., 2016).The development has resulted in policy conflicts in shale oil and gas-producing states (Heikkila & Weible, 2017) and responses by governments to address the various emergent issues (e.g., Davis, 2012;2017;Rinfret et al., 2014;Heikkila et al., 2014;Cook, 2015).Whereas research on oil and gas development involving hydraulic fracturing tends to focus on politics (Heikkila & Weible, 2017), mobilization (Jerolmack & Walker, 2018), and discourse (Metze & Dodge, 2016), few have conducted comparative analyses of the public policies, let alone their changes in the legislative process.We fill this literature gap in our paper.
The theoretical contribution of this research is threefold.First, in terms of policy formulation, we find that from the proposal to the final stage, legislative bills demonstrate the general pattern of expansion (instead of retracting) in the topics they address with variations across states.Second, we caution about the loss of validity in institutional grammar tools.Using a new methodology, we show that although institutional grammar tools provide reliable indicators of policy change across many policies, these indicators do not necessarily capture the substance of change in individual policies.In the stream of comparative public policy, we highlight the institutional diversity in the themes and content of public policies across states.What appears to be comparable changes in policy, such as those addressing severance tax, may, in fact, demonstrate distinct features that are conditional on the socio-economic and legislative context of the state.with a recap of the essential findings and lay out the next steps to advance this stream of research.

Foundational Streams of Literature 8
The novelty of this research prevents us from positing hypotheses or expectations of what we might see in analyzing introduced and final bills and comparing these results across states.Instead, in this section, we describe three relevant streams of literature, including how they inform this study and how this research can contribute to them, which we further elaborate in the conclusion.

Policy Formulation and Design 9
The first stream of literature relates to policy formulation and policy design (e.g., Dahl & Lindblom 1953;Lowi, 1964;Linder & Peters, 1990;Sidney, 2017;Howlett, 2019;Jordan & Turnpenny, 2015;Howlett & Mukherjee, 2017;Siddiki & Curley, 2022).Often cast as one of the policy cycle stages, policy formulation is defined as the activity of finding, devising, and defining solutions to a policy problem once a public problem has been recognized to be worthy of government intervention (Howlett & Mukherjee, 2017, p. 4).
The definition of policy formulation highlights that this process happens within the context of the political agenda and is not value-neutral.Indeed, formulation involves negotiating, bargaining, argumentation, persuasion, favors, and deals.The upshot is that policy formulation involves the exercise of power, often through cooperating, using pressure, or both.Studying the consequences of such political processes, namely the legislative output, sheds light on how some policy instruments and targets are emphasized and legitimated by policymakers while others are sidelined.
Considerable literature on policy formulation and policy design deals with taxonomies and theories of tools or instruments (Dahl & Lindblom, 1953;Schneider & Ingram, 1990;Howlett, 2019;Jordan & Turnpenny, 2015).Policy instruments are "…the techniques of governance that help define and achieve government goals" (Howlett, 2017, p. 96).They are usually placed into categories to capture different mechanisms through which the policy is expected to work.For example, one such taxonomy based on governing resources categorizes policy instruments into four types that require informational, coercive power, financial, or organizational resources (Howlett, 2019, p. 83).Moving beyond tools and goals into the policymaking context, policy design is a form of policy formulation that utilizes knowledge about policy tools' effects on policy goals to attain the desired public outcomes (Howlett, 2019, p. 48).
The policy formulation and design literature has made great strides since its origins (e.g., Dahl & Lindblom, 1953).However, this literature remains hampered by measurement challenges.Despite recent advances by Fernández-I-Marin et al. (2021), questions remain about the substantial content of policies underlying the categories of instruments and tools.Thus, we tend to think of policy formulation as akin to a policy change with some attention dedicated to the composition of the finalized policies (see Weible & Carter, 2015).
Finalized policies do not reveal the full picture of policy formulation, which usually consists of four aspects: appraisal, dialogue, formulation, and consolidation (Thomas, Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023 2001).Our study focuses on the formulation and the consolidation phase.During the formulation phase, policymakers consider policy alternatives in terms of their costs and benefits, where debates on the choice of some policy alternatives over others take place.In the final consolidation phase, proposed policy solutions are amended or refined before becoming binding legislation (Howlett & Mukherjee, 2017, p. 7).In this study, we infer insights about policy formulation and consolidation processes by looking at the initial and finalized policies.This paper builds on and develops the policy formulation and design literature.First, we focus not on policy instruments and tools but on the written content of the public policies (see the following subsection on the institutional grammar).Second, we explore policy formulation's inputs and outputs by examining the legislation's initial and final versions.While we cannot shine light into the black box of the process of policy formulation, namely documenting the debates and the power play that shape the formulation and consolidation phase, we do hope that studying changes from initial to final versions of policies helps inform what happens in the said process.

The Institutional Grammar
The second stream of literature is the institutional grammar (Crawford & Ostrom, 1995;Siddiki et al., 2019).Whereas policy formulation and design often emphasize the linkage between means and ends through the choice and design of instruments organized by taxonomies, institutional grammar offers an approach with sharper magnification and far more precision.Emerging from the seminal work by Crawford and Ostrom (1995) and set within the broader research program associated with the Institutional Analysis and Development Framework (Ostrom, 2009), the institutional grammar provides a general and adaptable approach to studying the content of public policies.
The challenges in developing and applying institutional grammar have changed over time.The original interpretations of the institutional grammar and adaptations from Crawford and Ostrom relied heavily on developing reliable approaches to coding the text of public policies, which led to using the sentence as the unit of observation and sometimes the unit of analysis (Basurto et al., 2010;Siddiki et al., 2011;Carter et al., 2016).Using hand-coding techniques, researchers would dissect a sentence by identifying its attribute(s) (subject of a sentence), aim(s) (verb of a sentence), object(s) (direct or indirect objects of a sentence), and condition(s) (the prepositional phrases in a sentence).
Once the challenge of reliable hand-coding was overcome, the next challenge involved upscaling the approach to hundreds or thousands of sentences or public policies while avoiding the laborious hand-coding.The challenge led to semi-automated and automated approaches to text analysis, which enabled larger-n applications of the institutional grammar (Heikkila & Weible, 2018;Frantz & Siddiki, 2022).
However, as with hand-coding, the semi-automated and automated approaches did not produce uniformity in applying the institutional grammar.A subsequent challenge involves what to do with the data once extracted with the semi-automated and automated approaches.Inevitably, the diversity of public policies and research questions led to a wide array of interpretations and adaptations of the institutional grammar.In other words, there is no single way to apply the institutional grammar, and even some of the most basic assumptions, as found in Frantz and Siddiki (2022), might not apply to some questions, data sources, and theories.This paper builds on the institutional grammar, particularly the semi-automated approach developed by Heikkila and Weible (Heikkila & Weible, 2018;Weible & Heikkila 2020;Heikkila et al., 2021;Chen et al., 2022).As described in the methods section below, their technique returns to Crawford and Ostrom (1995) with a fresh interpretation and adaptation of their institutional grammar and incorporates the rule types found in the Institutional Analysis and Development Framework.However, it shifts the unit of observation from the sentence to the public policy (in this case, a legislative bill) and relies on a dictionary to extract critical components of the public policy.
By shifting the unit of observation from the sentence to the public policy, our technique sidesteps the grammatical structures in the recent interpretations of the institutional grammar (Frantz & Siddiki, 2022) and the associated potential loss of meaning in large-n applications of public policies.More specifically, our research compares U.S. state legislatures' initial and final versions of public policies.Given textual changes in initial and final versions, comparing sentences to sentences as outlined in Frantz and Siddiki (2022) is infeasible and would sacrifice too much in the validity of interpretations.Thus, this paper continues the ongoing research on the institutional grammar with a new application using the Heikkila-Weible approach.

Comparative Public Policy
The third stream of literature involves comparative public policy.Comparative analyses of public policies have long been a preoccupation for policy scholarship (e.g., McDougal, 1952;Lasswell, 1956).Early research on comparative public policy in the 1960s focused attention mainly on describing and explaining the adoption of public policies across space and time, especially across U.S. states (Dawson & Robinson, 1963;Dye, 1965;Sharkansky, 1970;Hofferbert, 1974; see also Blomquist, 2019).
The emphasis of comparative public policy scholarship has shifted and broadened (e.g., Schmitt, 2012;Dodds, 2018;Peters & Fontaine, 2020).It now entails sweeping questions that span the scope of policy studies.Indeed, this literature covers policy outputs and outcomes (e.g., the adoption of public policies and their effects) and various aspects of the surrounding processes and politics.For example, it includes direct and indirect comparative applications of policy process theories (Tosun & Workman, 2018), exemplified by the Comparative Agendas Project (Baumgartner et al., 2018).Moreover, given the renewed emphasis on better methods in policy studies (Peters & Fontaine, 2020;Weible & Workman, 2022) and the globalization of policy studies in recent decades, comparative policy studies have never had as much momentum.
This paper builds on comparative policy studies by comparing policy outputs (i.e., adoption) through novel methods.However, we do not posit explanatory models describing why the public policies changed.Instead, we shift attention to how the public policies changed comparatively across U.S. states.Too often, as found in the innovation and diffusion literature (see Berry & Berry, 2018), we assume homogeneity in the content of public policies across governing units.For example, we assume a policy dealing with the disclosure of fluids used in hydraulic fracturing in oil and gas development in one U.S. state equates with another disclosure policy in another U.S. state.However, these two policies might differ considerably even though both address the issue of disclosure.
Additionally, comparative policy studies continue to lack descriptions of policy change.As mentioned above, we often compare final public policies by simple measures of the tools and instruments they contain.However, these tools and instruments might also change across versions of public policies.Thus, one of our contributions is describing the content of public policy change across states.In addition, as mentioned, comparative public policy studies also tend to overlook policy formulation, particularly the changes between the initial and final versions of the public policy -a gap that we hope to fill with this paper.
To summarize, we build on and contribute to the streams of literature on policy formulation and design, the institutional grammar, and comparative public policy.Our contributions spring from our methods, which entail an adaptation of the institutional grammar to a large sample of public policies.We also shift the focus from only looking at policy adoption to the inputs and outputs of the black box of policy formulation comparatively across U.S. states.We answer our research question by comparing initial and final versions of public policies inductively.

Study Background
The context for this study is oil and gas development across six U.S. states between 2007 and 2017.In line with research on policy subsystems, a ten-year time frame offers a useful window through which to observe and assess policy changes within this policy domain.We selected 2007 as a starting point because that was the year when unconventional oil and gas production started to increase significantly across the U.S. and in other parts of the world because of technological advances in hydraulic fracturing and horizontal drilling.Among other things, these processes enabled oil and gas production from shale formations, leading to a 10-fold increase in natural gas production from shale formations during this study's period (US EIA 2018).
With the expansion of oil and gas development, which in some cases encroached on population centers, conflicts arose between oil and gas companies, environmental groups, members of the public, and regulators (Dodge, 2017;Heikkila & Weible, 2017;Jaquith, 2017).These conflicts span a broad range of topics-such as whether and where oil and gas development should occur, how to regulate and tax the practice, and who should have regulatory authority-all of which also become the subject of public policies (Chen et al., 2022).
States are the principal policymakers in the realm of oil and gas (Rabe & Hampton, 2015;Warner & Shapiro, 2013).The federal government maintains some limited authority over oil and gas development through policies such as the Clean Air Act and the Safe Drinking Water Act, and states may allow local governments to exercise authority via zoning or nuisance laws, for example.However, most oil and gas policies are formulated at the state level, primarily through state legislatures and regulatory agencies.This study focuses specifically on legislative policy due to the comparability of state legislatures and the bills that they produce (Chen et al., 2022;Collingwood et al., 2019;Mooney, 1991).Furthermore, across states, legislative bills have clear initial (introduced) and final (passed) versions, which are essential to our analysis.Because hydraulic fracturing is a highly contested policy issue, bills related to this topic will likely come under scrutiny when proposed, followed by debates, negotiations, and compromises among legislators and under the influence of interest groups and public pressure.Therefore, we expect bills to undergo nontrivial changes and exhibit different change patterns across states.This study is a first step toward further investigating of such change patterns across states in the policy formulation and consolidation stages.
This research focuses specifically on six states with high oil and gas production: Colorado, Louisiana, North Dakota, Oklahoma, Pennsylvania, and Texas.The selected states are among the top six U.S. states for natural gas or crude oil production or both, representing over 70% of oil and gas production in the U.S. (US EIA, 2020; 2021).These states also engaged in significant policymaking during our study period, collectively producing the 105 bills analyzed in this research.The choice of these states provides regional, geophysical, and political variation, allowing for interesting interstate comparisons.
For example, while most states in our sample are Republican strongholds, Pennsylvania and Colorado have experienced shifting politics in recent election cycles.Also, there is a much longer history of drilling in the Barnett and Marcellus shale formations underlying Texas and Pennsylvania, respectively, compared with the Bakken and Niobrara formations underlying North Dakota and Colorado.As such, while policies across these states are comparable, the contexts are sufficiently diverse to allow for comparative analysis.
A complete list of the 105 bills with their themes can be found in Appendix A. They cover topics ranging from the authority of agencies and boards, disclosure of information, and regulation of operations to tax and finance policies.The proposed version of state-level bills are usually sponsored by a house representative or a senator and brought to the chamber floor for deliberation.If they clear the initial chamberoften with revisions-they are sent to the other chamber for further deliberation.During the process, either or both policy formulation and policy consolidation can take place.In the former case, policies undergo major changes that render final versions of policies aiming for goals or employing tools different from the initial version.In the latter case, policies are amended or refined with minor changes.Between bill proposal and passage, it is common for both formulation and consolidation to take place.

Methods
We adapt IGT for our paper's methodological approach.First, we chose six U.S. states with high volumes of shale oil and gas production (i.e., each ranks in the top ten states producing either shale oil or shale gas, or both, in the U.S., according to the U.S. Energy Information Administration) and organize the proposed and the final versions of legislative bills passed in these states that are relevant to shale oil and gas development.We then use a semi-automated approach to measure the composition of the proposed and passed bills.The purpose is to describe how the bills' compositions change between versions.Similar to Heikkila and Weible (2018), we focus on individual bills as the unit of observation and use a modified approach to identify the IGT's "aims" (as an indicator of a rule) and "objects" (as an indicator of an issue area).Rather than classifying "aims," "animated objects," or "inanimate objects" based on the grammatical functions words serve in an institutional statement, we capture any action or suggestion of action (the "aim") regardless of its grammatical role in a sentence and classify them as rule indicators.Similarly, we classify any inanimate objects (e.g., "infrastructure") as issue areas, and animate objects (e.g., "State Department of Transportation") as actors, regardless of whether they are subjects or objects grammatically.We also include "deontics" as an indicator for stringency.
We coded 105 bills passed in six shale-producing states between 2007 and 2017 when shale development in many states picked up speed.We identified and downloaded legislation by searching each state's database of legislation through LegiScan or the state legislature's websites.We used the words "fracking," "fracturing," "shale," "unconventional," "oil and gas," or "mineral" for bill search, and we manually reviewed each bill for relevance.We then reviewed the purpose of each bill to verify its focus on oil and gas.We also manually removed sections of omnibus bills unrelated to oil and gas development.

Policy composition
We focus on rule types, stringency, issue areas, and actors for policy composition analysis in proposed and passed bills.This approach allows us to capture the changes in policy composition from the proposal to the passage of bills.Our expectation is that some components may dramatically increase or decrease during bill passage.We used the software Automap (Carley & Diesner, 2005) and the programming language Python for the semi-automated analysis.The approach has been previously detailed in Chen et al. (2022) and explained here to aid readers' understanding.We search and extract the words in a text document and categorize them based on a dictionary of relevant terms pre-identified by the researchers 1 .We classify all the extracted words according to the category buckets within which they are defined in the dictionary.We then count the frequency of words in each dictionary category for each policy.For example, words such as "build" and "install" are indexed under the choice rules category in our dictionary.We then transform our dictionary into the dictionary data type in Python and code a search function such that an occurrence of the word "install" is counted as one instance of choice rules.We divided the number of observations of words under each category by the total number of words in each policy to create a standardized frequency measure.Additionally, after obtaining the frequency of each actor category in each bill, we calculate Blau's Diversity Index based on all actor categories for each bill.Note that the exact process is repeated for both the proposed and the passed version of the bills.Word counts under each category, and each policy's total number of words varies across the proposed and passed versions.
We consider relevant terms in the policies as indicators of each word category and employ the dictionary originally developed by Heikkila and Weible (2018) for analyzing oil and gas regulations in Colorado.Here, we use the same dictionary as in Chen et al. (2022) updated from Heikkila and Weible (2018) by reviewing a pool of 168 policies across 15 states and adding words through an iterative approach to refining concept categories.The institutional grammar components (aims, deontics, and objects) are modified to be measures of rules, stringency, and issue areas to analyze individual policies instead of institutional statements.
39 Issue Areas.Issue areas are the topical targets of the rules of a policy.After reviewing the policies, we included 644 object terms in the dictionary.We categorize these words into the following eight issue areas.
Oil and Gas Operations words indicate an oil and gas process or facility (e.g., well stimulation, pressure test, vapor gathering system).
Infrastructure words are associated with facility that is separated from the oil and gas development (e.g., educational facility, hospital, highway).
Oil and Gas Policies or Strategies include words related to policies or formal strategies during the oil and gas development process (e.g., spill contingency plan, leak investigation, underground injection).
Other Policies reference named policies that are not restricted to oil and gas development (e.g., the Federal Water Pollution Control Act, cap-and-trade program, eminent domain).
Tax/Finance word are indicative of public finance activities (e.g., the Oil, Gas, and Geothermal Administrative Fund, severance tax, cleanup fund).
Oil and Gas Resources include words associated with the physical properties of extractable natural resources (e.g., reservoir, hydrocarbon, the Kern River Formation).
Environment or Health words are related to environment, wildlife, or human health (e.g., watershed, elk, air quality).
Negative Externalities reference words that indicate a negative impact from oil and gas development (e.g., contamination, pollutant, earthquake) 40 Rules.We categorize rules in a policy to understand the diversity in the range of powers within a given policy document, and how those powers are linked to actors.Adapted from the IAD's rule typology (Ostrom, 2009), we created a set of 313 words in the dictionary that represent different rule categories referencing the "aim" or verb in the IAD's Institutional Grammar Tool.
Aggregation rules indicate collective decision-making between two or more actors.This includes words such as "settle," "consult," and "negotiate." Authority rules grant authority to an actor and are indicated by words such as "regulate," "allow," "approve," and "mandate." Constitutive rules define phenomena.Words in this category include, among others, "declare," "define," "comprise," and "deem." Choice rules involve actions and decisions about procedures.In the context of oil and gas, we include words such as "build," "install," "load," "drill," and "operate."Additionally, general choice terms such as "employ," "maintain," and "serve" are included.
Payoff rules involve requirements for specific actors to compensate other actors and include words such as "pay," "distribute," "compensate," and "deposit." 1.

7.
Comparing Stringency indicators.We employ the IGT's notion of "deontics" as indicators of stringency in this study.Deontics are the imperatives that give rules force in commanding whether action "must" (including shall and will), "should," or "may" be taken.The dictionary includes 16 terms categorized under the three stringency indicators.
For all the categories of these three grammar components, we calculate the percentage of words in each category out of a bill's total number of words.This approach allows us to directly compare categories of different grammar components.
Actor Diversity.We refer to the actor dictionary previously developed in Chen et al. (2022).Actors are categorized under seven labels: court, expert, federal, industry, local, public, and state.For example, "state department of fish and wildlife" is categorized as a state actor, and "Indian tribe" as public.We embed the state-specific actor dictionary in a search function, which automatically extracts, categorizes, and counts actors in each category for each bill of a state.This process is repeated for all states.
With counts of actors in each category for each bill, we calculate the Blau index (also known as the Gibbs-Martin index) to account for the diversity of actors in each bill.The formula for Blau index is: Where i denotes an actor category, p i is the proportion of the mentions of actor category i relative to the total mentions of all R categories of actors.We calculate the square of the proportion p 2 i for each actor category present in a bill and take the sum.The sum is then subtracted from 1 to generate the Blau index.The Blau index accounts for the number of actor categories and the balance of mentions of each category.
In the next section, we discuss our findings on how actor diversity and frequencies under each rule, issue area, and stringency category changed from the proposed to the passed version of bills.

Results
This paper compares the content of introduced and passed legislative bills within and across U.S. states.We first calculate the actor diversity and the frequencies of issue areas, rule types, and deontics (i.e., stringency indicator words) for all 105 bills in their proposed and passed versions.The frequencies are calculated by dividing the number of words in each indicator category by the total words in a legislative bill for its proposed and final versions.We then subtract the frequencies of the proposed version from those of the final version to calculate frequency changes.
The summary statistics of frequency changes are presented in Table 1.For example, a mean of 0.0019 for the frequency change in the environment and health issue area indicator is calculated by subtracting the mean frequency across proposed bills (0.0020) from the mean frequency across passed bills (0.0039) of this indicator.1 shows, across all 105 bills passed in six states, the mean frequency of each category of issue area indicators increased from the proposed version to the passed version of bills.Across issue area indicator categories, oil and gas resources and tax and finance issue area frequencies increased the most.The mean frequencies of rule indicators also increased across all categories between proposed and passed bills.The mean frequencies of information rules and choice rules increased the most, while the frequencies of aggregation and position rules barely changed.Among stringency indicators, the mean frequency of "must" words increased the most, followed by "May" words, while the frequency of "should" words barely changed.Counts of actors in each actor category also increased except for federal actors.While Table 1 lists the raw data in changes by institutional grammar categories, exploring these changes with specific illustrations points to nuanced dynamics.We turn to discussing individual bills with a complete list of bills analyzed in this paper included in the appendix.For example, in Louisiana, the proposed bill HB549 made very minor modifications to tax rates for low-producing wells (parameters and rates are defined in the bill).However, the passed version of this bill was entirely different.The passed version provides an exemption for severance tax for horizontal wells until the well cost is achieved.The amount of the exemption is based on the price of oil or natural gas.While both deal with severance tax, the proposed bill made minor modifications to tax rates for lower producing wells, and the passed bill lays out exemptions from severance tax for initial periods after drilling depending on the price of oil or natural gas.In this bill, the frequency of tax and finance issue areas increased from 0.0052 in the proposed version to 0.0156 in the final version.Thus, while we can reliably count the differences, say in an issue area, such counting also misses some of the descriptive validity of the nature of that change.
In contrast, the changes observed in issue areas in some proposed and passed bills were definitional.They defined phenomena related to oil and gas governance to clarify or improve policies.For example, the proposed version of SB1541 in Texas only defined the word "treatment".The final version of this bill kept this definition but also defined what a beneficial use is and stated that rules governing the treatment and beneficial use of drill cuttings must emphasize protecting public health and the environment.In this instance, we see the frequency in the issue area indicators of environment and health change from 0 to 0.0206.
Next, we compare how each grammatical element changed from proposed bills to passed bills across the six states.First, we identified the first, second, third, and fourth quartiles in the word frequency of each category of grammatical elements across all 105 bills for their respective proposed and passed versions.We then identified the median frequency of each element category in each state.Then, we mapped the state median frequency into the quartile of the frequency of all bills.For example, for resource issue areas in the 105 passed bills, the quartile thresholds are <=.006 for the first quartile, <=.011 for the second quartile, <=.021 for the third quartile, and <=.0528 for the fourth quartile (see Figure 1).The median issue area resource frequency for Colorado's passed bills is .0092,which falls in the second quartile of all-bill frequencies.Hence, its representation on Colorado's radar chart in Figure 1 for passed bills is 2.
Regarding issue area indicators, North Dakota and Pennsylvania have consistently high median frequencies across most categories.In Colorado, median frequencies decreased in the issue areas of environment and health, other policies, oil and gas resources, and tax and finance.Median frequencies of environment and health also decreased in Oklahoma but increased in North Dakota, Pennsylvania, and Texas.The quartile of median actor diversity index decreased in North Dakota, increased in Oklahoma and Pennsylvania, and remained the same in Colorado, Louisiana, and Texas.On the issue areas, at least, we do not find patterns of change between the proposed and final bills that are associated with a state's political context or experience with oil and gas drilling.
In Texas, Louisiana, and North Dakota, all relatively conservative states, the quartile of rule frequencies almost all remained the same from the proposed version to the passed version of bills.In Louisiana and Texas, the median frequency of authority rules and information rules decreased, respectively.The median frequency of authority rules also decreased in North Dakota.On the other hand, median frequencies of information and authority rules increased in Colorado, a more moderate state, while constitutive rules decreased.Similarly, in Pennsylvania, information, enforcement, choice, and authority rule median frequencies increased, while position rule median frequency decreased.
Unlike the other more conservative states, position, payoff, and information rule median frequencies in Oklahoma, all increased from the proposed to passed versions of bills.Additionally, this bill is an example where the final, passed version expanded information rules (0.0143 in the proposed and 0.0207 in the final).For example, it adds a section encouraging cooperation and exchanges of information between different agencies and levels of government.The bill maintains the language that certificates may not violate local laws but also says that permits preempt local laws.It requires a state commission to notify relevant local governments before hearing and calls for a study about cooperation and communication between the public service commission and local governments.
Thus, the purpose of the proposed and passed versions of SB2286 in North Dakota is to strengthen the requirement that site compatibility certificates for energy conversion facilities, including gas or liquid transmission facilities, cannot violate local land use and siting laws.The proposed law said that if local government does not testify at hearings, there is a presumption of no conflict with local laws.The passed law, however, exempts "road use agreements" and encourages cooperation between the Public Service Commission and local governments, including a requirement that the commission notify relevant local governments prior to hearings on permits within their borders.Along with the changes in issue areas related to other oil gas policies and information rules, the major change from the proposed to the final included an increased actor diversity index from 0 to 0.75, indicating many more types of actors involved.
Not all bills changed drastically in their substantial content when negotiated into their final version.In contrast to Louisiana's HB549, Pennsylvania's SB259 proposed and passed versions of this bill are relatively consistent.The primary purpose of both versions of SB259 is to ensure that if existing oil and gas leases do not provide at least 1/8 royalty to lessees, the royalty will be raised to 1/8 if the lease is altered to increase production.The major difference was not revisions but two additions.First, operators can develop multiple contiguous leases jointly via horizontal drilling, in which case all parties via agreement or the operators would figure out how to allocate production.Second, a conflict exists between a division order (a document indicating a landowner's share of royalties) and an oil and gas lease, the oil and gas lease controls.SB259 is an example of a bill focusing on payoff rules (a frequency change from 0.0022 in the proposed version to 0.0048 in the final version) and information rules (a frequency change from 0.0076 in the proposed version to 0.0096 in the final version).Source: The authors 59 We also calculate the number of actors in each category averaged over bills passed in each state.Across states, the mean number of state actors increased substantially from the proposed to the passed version of the legislation.In Colorado, Louisiana, North Dakota, and Pennsylvania, the mean number of state actors in passed bills is much higher than the mean number of actors in all other actor categories.In these four states, industry actors average over one per bill, ranking the second highest across categories.Oklahoma and Texas, on the other hand, are quite different from the above states.In Oklahoma, the mean numbers of public and industry actors are the highest in the passed bills, substantially increasing from the proposed version.In Texas, the mean number of actors overall is lower than in other states-no actor category has over one occurrence averaged over all bills.Among the actor categories in Texas bills, the occurrences of state, expert, federal, and industry actors are high relative to other categories.
60 Sometimes, changes in actors coincide with shifts in who is targeted with responsibility in a bill.For example, the proposed version of HB 1331 in Texas clarifies that provisions about transfers for treatment and beneficial use only apply to waste that includes drill cuttings or fluid waste.It also states that the person who generates the waste (oil and gas company) cannot be liable in tort once they transfer possession for treatment.The major change from the proposed version to the final version is that the person treating the waste must have a permit from the Railroad Commission, and the person (i.e., the permit holder) must provide a copy of the permit to the entity generating the waste.In this situation, state actors were non-existent in the proposed bill but introduced in the passed bill.
We end with an important caveat in describing changes in the content of proposed and final bills.Take, for example, Oklahoma's SB243.The initial bill states that before entering a site with heavy equipment, operators must negotiate with surface owners regarding compensation for damages caused by the drilling operation.If the parties do not agree, this bill lays out detailed procedures for appointing appraisers, providing notice of appraisers' reports, and opposing appraisers' reports.In contrast, the passed version of this bill is entirely different in topic and focus.The passed bill is about the Seismic Exploration Regulation Act, adding more detailed provisions about notice requirements and outlining how to deal with damage compensation from seismic exploration.In this example, we see increases in industry actors (originally one) and public actors (originally two) to three each and negative externality issue area frequencies from 0.0015 to 0.0174.
Thus, in this Oklahoma example, the proposed and passed bills are entirely different.
The proposed bill addressed processes for appointing appraisers and notification and response requirements related to appraisers' reports in the case of surface damage from drilling operations.The passed bill relates to the Seismic Exploration Regulation Act and adds provisions about notice requirements and how to deal with damage compensation from seismic exploration.

Discussion
In this paper, we apply the institutional grammar to analyze changes in proposed and final legislative bills across six U.S. states.We asked: How does the content of public policy (in this case, legislative bills) change from introduced to passed versions within and across U.S. states?
When examining the overall averages across the states, we find that shifts from proposed to final versions of legislation tend to show more expansion than retraction in institutional grammar components.One might expect that when different interests negotiate to make a proposed bill acceptable to all for passage, the debating process will lead to fewer institutional components.Yet, in the emerging context of shale oil and gas development, the legislative efforts, on average, lead to adding components to proposed bills at the policy formulation stage.This observation may indicate that policymakers are pushing to clarify or add goals and preferences to bills between proposal and passage.Understanding the rationale for our observations requires more in-depth research, such as investigating legislative session minutes and measuring bill sponsors' legislative effectiveness.
The pattern of expansion does not hold when examining all the individual states.For example, although the deontic "must" increased on average across the six states from the proposed to the final legislation, it decreased in North Dakota and Texas and stayed the same in Oklahoma, which tend to be more conservative states politically.Conversely, the "musts" increased in Colorado and Pennsylvania, which are more politically moderate states.This raises interesting questions about the underlying politics driving the negotiation process that happens in state legislatures after the proposal of a policy on a contentious issue such as oil and gas development.Furthermore, we do not observe consistent patterns of change for the institutional grammar components across individual states; we see variation across states in what increases or decreases from the proposed to final versions of the legislation.Even in the same substantive area of oil and gas development using hydraulic fracturing, states differ significantly across the institutional grammar components and the type and amount of change from initial to final versions.While some similarities emerge (e.g., all states increase in the proportion of "state actors" written into the final compared to the proposed legislation), they diverge elsewhere.Again, these patterns may reflect the unique political landscape in each state and their differences in legislative composition and approaches.They may also reflect differences in the issues that states address.For instance, Oklahoma's legislation on seismic risks imposed by oil and gas was responding to rising public attention and concern for a less prevalent issue in other states (Ritchie et al., 2021).The changes from the proposed to the final version of the bill may have coincided with new evidence and knowledge about the problem.

Conclusion
This research builds on and contributes to the literature on policy formulation and design, the institutional grammar, and comparative public policy.First, our findings underscore the complexity of the policy formulation stage.Policy formulation has been described as one of the policy cycle stages receiving the least attention (Howlett, 2019), needing an established theory to explain its nature.Our findings suggest that policy formulation varies considerably across states and individual bills, underscoring the complexity of this black box.Future research should investigate how specific components are proposed and adapted when formulating a new policy and the reasons underlying such adaptation.
Our findings for the institutional grammar involve one of caution.Applying the institutional grammar to compare large samples of initial and proposed legislation provides reliable indicators of change, but we also observe significant sacrifices in validity and meaning.We must guard against drawing lessons from patterns observed through automated approaches while overlooking the substance of change.We recommend efforts in further theoretical development in institutional grammar to guide our data collection and choice of measures in policy design studies.
Considering comparative policy studies, we find sizeable institutional diversity in the focus and content of public policies across states and within each state.Therefore, we warn against current practices of homogenizing policy change across states, especially the overemphasis on explaining a type of policy change in a substantive area (e.g., comparing a change in tax policy across states).
We propose an agenda for future research based on the above discussion.First, we recommend comparing our institutional grammar approach to others, such as those in Frantz and Siddiki (2022).Our approach assumed the public policy as the unit of observation and analysis, and we relied on a dictionary to extract the institutional grammar components (Chen et al., 2022).Any approach carries tradeoffs in what we can observe and interpret about public policies and their design.Indeed, we are simultaneously defining what to promote and demote in our methods, thereby constructing institutional diversity just as we discover it.The institutional grammar community needs to address the sensitivity of their methods in drawing any conclusions.
71 In the larger community of policy design studies, we also need to return to the conceptual components of the institutional grammar and recognize that these components are being promulgated based on the assumption that they have theoretical and practical significance.Assuming institutional grammar components do matter, what standard or measures will be our comparison or benchmark?One solution is to consciously put the institutional grammar head-to-head with other measures of policy design, as found in Fernández-I-Marin et al. (2021).We should apply the institutional grammar to policies previously studied using other approaches to assess its integrity.
72 Second, given the lack of theoretical expectations, we conducted this study inductively.
On the one hand, this reflects the novelty of our methodology that makes reliable comparisons of the proposed and final versions of legislation possible.On the other hand, this demonstrates the need for more theoretical progress in policy formulation.
As an area of study, policy formulation has advanced our understanding of policy instruments (e.g., Jordan & Turnpenny, 2015).However, it has yet provided insights on the policy content through which these instruments emerge.Doing so requires a concerted effort using diverse research designs and methods to generalize and localize our knowledge about the black box of policy formulation.

TX_SB1422
Protection and use of intellectual property by the Railroad Commission of Texas; money paid to the commission shall be deposited to the credit of the oil and gas regulation and cleanup fund TX_SB1541 Treatment and recycling for beneficial use of drill cuttings NOTES 1.More detailed coding procedures are available from the authors, with step-by-step instructions.For instance, before running Automap, all policy texts had to be pre-processed to remove punctuation, headers, footers, and symbols.We also standardized some words across policies to ensure that the automated analysis would differentiate keywords.

ABSTRACTS
This paper builds on arguments in policy formulation, the institutional grammar, and comparative public policy by comparing and analyzing the initial and passed versions of 105 bills Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies in six U.S. state legislatures from 2007 through 2017.Our substantive context is oil and gas development.The findings show that shifts from proposed to final versions of legislation tend to expand more than retract in the institutional grammar components, averaged across states.However, this pattern of expansion does not hold when examining all the individual states.Furthermore, no consistent patterns emerge about the changes in the institutional grammar components across states; that is, we see variation across states in what increases or decreases from the proposed to final versions of the legislation.The findings underscore the complexity of policy formulation and the need for theoretical development, the sacrifices in validity when analyzing large samples of public policy using the institutional grammar, and the sizeable variation across states in the content of public policy for the same substantive area.We conclude with a call for a concerted effort using diverse research to begin to generalize and localize knowledge about policy diversity and formulation.
Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023 Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023 Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023 Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023 Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023 Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023 Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023 Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023

Figure 1 :
Figure 1: The change in quartile from the proposed to the passed version of the bills in each indicator category, by state.
Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023

Figure 2 :
Figure 2: The change in count from the proposed to the passed version of the bills by actor category and by state.
Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023 Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023 Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023 and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023

Table 1 :
Summary statistics of changes in frequencies of issues, rules, deontics, and changes in actor counts from proposed to passed versions of all bills (n=105) 49 As Table Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023 ND_HB1409 2015 Prohibits outdoor heritage fund from any activity that would disrupt oil and gas operations; oil and gas impact grant fund; oil and gas well plugging/ site reclamation Comparing and Analyzing Policy Formulation of Proposed and Final Public Policies International Review of Public Policy, 5:2 | 2023 TX_HB2230 Authority to approve use of injection wells for disposal of brine from desalination or drinking water treatment residuals TX_HB40 Exclusive jurisdiction of the state to regulate oil and gas operations; the express preemption of local regulation of those operations.valorem tax purposes of a real property interest in oil or gas TX_SB1985 Appraisal for ad valorem tax purposes of a real property interest in oil or gas in place