Quantifying uncertainty in an industrial approach : an emerging consensus in an old epistemological debate
Abstract
Uncertainty is ubiquitous in modern decisionmaking supported by quantitative modeling. While uncertainty treatment has been initially largely developed in risk or environmental assessment, it is gaining largespread interest in many industrial fields generating knowledge and practices going beyond the classical risk versus uncertainty or epistemic versus aleatory debates. On the basis of years of applied research in different sectors at the European scale, this paper discusses the emergence of a methodological consensus throughout a number of fields of engineering and applied science such as metrology, safety and reliability, protection against natural risk, manufacturing statistics, numerical design and scientific computing etc. In relation with the applicable regulation and standards and a relevant quantity of interest for decisionmaking, this approach involves in particular the proper identification of key steps such as : the quantification (or modeling) of the sources of uncertainty, possibly involving an inverse approach ; their propagation through a preexisting physicalindustrial model; the ranking of importance or sensitivity analysis and sometimes a subsequent optimisation step. It aims at giving a consistent and industriallyrealistic framework for practical mathematical modeling, assumingly restricted to quantitative and quantifiable uncertainty, and illustrated on three typical examples. Axes of further research proving critical for the environmental or industrial issues are outlined: the information challenges posed by uncertainty modeling in the context of data scarcity, and the corresponding calibration and inverse probabilistic techniques, bound to be developed to best value industrial or environmental monitoring and data acquisition systems under uncertainty; the numerical challenges entailing considerable development of highperformance computing in the field; the acceptability challenges in the context of the precautionary principle.
Index terms
Keywords :
certification, decisionmaking, modeling, risk assessment, sensitivity analysis, UncertaintySections :
PerspectivesOutline
Top of pageEditor's notes
The article has been reviewed by Enrico Zio, Roger Flague and a third anonymous referee.
Publication history
Received: 27 February 2009 — Revised: 16 April 2009 — Accepted: 17 April 2009— Published: 25 May 2009.
Full text
Introduction
1Uncertainty and associated risk assessment is rapidly developing on largescale industrial or environmental systems along with the dissemination of advanced quantitative modeling in support of decisionmaking, the increased public awareness or appetite for questioning expertise and the enforcement of tighter safety or environmental control standards. In fact, uncertainty may be viewed as consubstantial to any human activity well beyond the issues of applied science and decisionmaking. Yet, this paper concentrates on the narrower field of quantitative and quantifiable uncertainty, with a modeling view, as the subject of considerably growing interest in industrial or environmental fields. This may sound somehow restrictive to those having in mind the realworld risk situations where on the one side human and organisational factors play an essential role (as evidenced by classical examples of the ThreeMileIsland accident in the nuclear field or of the Challenger space shuttle in the aerospace industry) and on the other side uncertainty is to a large extent poorly or not at all quantifiable (think about quantifying lack of knowledge about the plausibility of September 11^{th} or another similar event). Those aspects evidence obvious limitations to the applicability of uncertainty quantification, although some limited modeling contributions may still apply here and there, such as the human reliability probabilistic models. Quantitative risk and uncertainty remains however essential existing or future tools for the regulation of industrial activity or environmental control, as well as the decisionmaking in the corporate investment or public infrastructure or health.
2Indeed, a growing number of industrial studies involving quantitative models include treatments of the numerous sources of uncertainties affecting their results. Uncertainty treatment in physical, environmental or risk modeling is the subject of a longstanding theoretical literature rooted in fundamental statistical and economical thinking (Knight 1921; Savage 1954). It then developed in the risk and environmental assessment fields (Beck 1987; Granger Morgan & Henrion 1990; Helton 1993; Hamby 1994; PatéCornell 1996) as well as in metrology (ISO GUM, 1995) and signalprocessing (Shannon, 1948; Cover & Thomas, 1990). Such developments included a central debate on the relevance of classifying the large variety of uncertainties encountered in two categories : namely the epistemic (or reducible, lack of knowledge, …) type referring to uncertainty that decreases with the injection of more data, physical knowledge or model runs and the aleatory (or irreducible, intrinsic, variability, …) type for which there is a variation of the true characteristics of the systems that may not be reduced by the increase of data or knowledge. More recently, uncertainty treatment gained largescale industrial importance as a number of major applications gradually included some uncertainty treatment, including in particular : (i) nuclear safety studies involving large scientific computing (thermohydraulics, mechanics, neutronics etc.) or Probabilistic Safety Analyses (PSA); (ii) advanced design in the aerospace, automotive or more generally mechanical industries; (iii) oil exploration and underground waste disposal control; (iv) environmental impact studies.
3On the one hand, appropriate uncertainty management allows for improved system designs, more accountable forecasts, better natural and industrial risk control, and more robust performance, thereby generating an “internal/business” driver. On the other hand, many institutional bodies or regulators now demand technical justifications, including an understanding of the influence of uncertainties in industrial safety or environmental control, thereby generating an “external/regulatory” motivation. Both may take place at many different steps of the industrial cycle from upstream research to inservice operation and control (see Figure 1).
 1 European Safety, Reliability and Data Association, www.esreda.org
4Most recently, the ESReDA1 European network of industries and academics undertook a large review and methodological research resulting in a consistent generic framework applicable to most industrial studies (de Rocquigny et al 2008), in which the author of this paper took a leading role. For most of the uncertainty treatment undertaken in the industrial world, at least partial probabilistic modeling of the uncertainties is considered. Yet, deterministic uncertainty treatment is generally still involved and more isolated nonprobabilistic ventures (such as fuzzy sets, evidence theory or possibilistic approaches) may be encountered, resulting in a rather heterogeneous set of decision criteria and differing interpretations of the probabilistic or nonprobabilistic figures. One of the challenges that motivated such research was to properly formulate in consistent mathematical terms the variety of decision criteria encountered in practice, as a prior to direct meaningful uncertainty modeling.
5A number of generic challenges may be encountered in developing industrial uncertainty treatment:

cultural and organizational ones: the consideration of uncertainty disrupts to some extent the traditional engineering habits and regulatory settings and often faces claims of costly, confusing or insufficiently guided sophistication, as risk analysis proves generally difficult to communicate to clients or furthermore to the general public;

policy ones: decisionmaking under uncertainty requires the tricky choice of quantities of interest or risk measures (e.g. expected utilities) that should properly represent risk aversion or behavior in front of uncertainty;

scientific ones: the coupling of probabilistic and phenomenological modeling generate new concepts and open mathematical and epistemological issues;

 2 Open Treatment of Uncertainty Risk aNd Statistics : www.openturns.org (Andrianov et al, 2007).
technical and computational ones: regarding for instance the need to tackle largescale physicalnumerical models as most design or operation control studies rely on sophisticated engineering processes involving complex numerical codes, all the more so since high performance computing unlashes everincreasing meshing sizes or micromacro equation formulations. They require large CPU budget to run and are fed by quite heterogeneous sources of information (noised data on various physical variables, incomplete expertise), far away from the closedform examples originally studied in the literature. They lead to an essential scientific computing challenge, as any uncertainty treatment increases by orders of magnitude the number of runs required by bestestimate studies. Dedicated tools are required in support : the present research was closely related to the development of the Open Source software platform called Open TURNS2.
6This paper provides a theoretical review of the subject inspired by a large panel of cases in various fields of engineering and environmental studies, evidencing the emergence of a practical consensus upon such an old epistemological debate, and setting forward a number of toppriority open challenges. The remaining is structured as follows. Section §2 introduces the versatility of the concepts, recalling the main threads of scientific debates on the subject, to which the generic uncertainty treatment approach explained in §3 contributes a practitioner answer. In §4, three examples illustrate the versatility of the framework in environmental monitoring, industrial safety and natural risk assessment. §5 open up the discussion on a number of key practical challenges mentioned earlier regarding respectively the definition of risk criteria, and the data modeling, calibration and inverse methods. Conclusion and references end up the paper.
An old epistemological issue
Defining uncertainty and risk
7As mentioned already in the beginning, uncertainty is the subject of longstanding epistemological interest as it stands in fundamental connection both with any type of modeling activity or the scientific consideration of risk. The former relates to the fact that any model endeavor brings along more or less explicit consideration of the uncertain deviation of the model to empiric evidence. Regarding the latter, risk and uncertainty analysis prove so much connected in applications that the precise delimitation between the practical meaning of the two terms does not appear to be central modelingwise, as it essentially depends on the system definition and the terminological habits of the field considered: to put it as simple as possible, some would interpret risk analysis as the consideration of the possible consequences and associated uncertainties about what will be the consequences, while uncertainty analysis could be limited to describe the initiating events.
8Indeed, in the perspective of the present paper, the practicality of undertaking an uncertainty (or risk) study of a given system is generated by the following common features : (i) the fact that the state of the system considered, conditional to taking some given actions, is imperfectly known at a given time; (ii) the fact that some of characteristics of the state of the system, incorporated in a given type of “performance” or “consequence”, are at stake for the decisionmaker. Because of (i), the best that may be looked after are possible or likely ranges for the variables of interest quantifying those characteristics ; more specifically, inference will be made under a probabilisticallydefined quantity of interest or risk measure, such as an event probability, coefficient of variation of the bestestimate, confidence intervals around the prediction, valueatrisk etc. The rationale of risk or uncertainty modeling is to estimate those quantities, aggregating any information available on any type of variable linked to the system, plus the information brought by statements defining the system phenomenology and structure (physical laws, accident sequence analysis etc.). Combining both statistical and phenomenological models aims at producing the highestinformed inference, hopefully less uncertain than a straightforward prediction obtained through pure empiric data (frequency records of observations) or expert opinion.
9A large variety of causes or considerations give rise to practical uncertainty in the state of a system as defined in (i) here above, such as: uncertain inputs and operating conditions in the industrial processes; model inaccuracy (simplified equations, numerical errors, etc.); metrological errors; operational unknowns; unpredictable (or random) meteorological influence; natural variability of the environment; conflicting expert views etc. A rich literature has described the variety of natures of uncertainty and discussed the key issue of whether they should or could receive the same type of quantification efforts, particularly a probabilistic representation. Such a debate may be traced back to the early theory of probability rooted in the 17^{th} century while modern thinking may be originally inherited from economics and decision theory, in close link with the renewed interpretation of statistics. (Knight 1921) has introduced the famous distinctions between “risk” (i.e. unpredictability with known probabilities) and “uncertainty” (i.e. unpredictability with unknown, imprecise probabilities or even not subject to probabilisation), although it is less often remembered how these early works already admitted the subtleties and limitations incorporated in such simplified interpretation regarding real physical systems. Indeed, economic and decision theory literature have generally restricted to simple decision settings, such as closedform quantitative lotteries, without tackling in details the physical bases of industrial systems. Closer look evidences that various forms of uncertainty, imprecision, variability, randomness, model errors are mixed inside phenomenological data and modeling. Think for instance about riverbed roughness for which the topography at a time is generally incomplete, varies significafficuin time enviroor after flood events, and is incompletely modeled within simplified hydrodynamics and associated numerical riverbed mesh. Knowledge of the systems is seldom as binary as incorporated in such statements as “probabilities are known” vs. “unknown” so that modern risk analysis would not generally stick to such historical distinctions.
A debate on the natures of uncertainty and associated quantification methods
10Indeed, tackling realscale physical systems has been further discussed on that issue within the risk assessment community, taking a large extent in the 1980s and 1990s in close link to the development of US nuclear safety reviews or environmental impact assessments (Granger Morgan & Henrion 1990; Apostolakis 1990; Helton 1993). As already mentioned, debate has notably concerned the classification of the large variety of uncertainties encountered in large industrial systems into two salient categories, and the rationale for separatviroor not those two categories : namely the epistemic or reductible type with respect to the injection of more data, modelingphysical knowledge or model runs and the aleatory type, that proves irreducible. To a certain extent, this epistemic/aleatory distinction may be viewed as refinement of the early economic distinction between uncertainty and risk. Note in fact that the practical reducibility of some of the uncertainty sources does not practically equate their “epistemic” (or theoretical reducibility) nature : reducibility also involves some industrial/practical constraints, and even a costbenefit point of view, and there can be a continuum between “stricticuirreducible” or “reducible” : in some cases, “epistemic” uncertainty cannot be reduced : e.g. very expensive measurements, or even nonfeasible without damaging the equipment …
11Finer distinctions include (Oberkampf et al 2002 ; Frey & Rhodes 2005; de Rocquigny 2006):

"variability" vs. "uncertainty" : this distinction, not far but not completely equivalent to the preceding one, is used more specifically when the system inputs mix a population of objects (or scenarios), a spatial distribution of properties within a system, or even a temporal distribution of properties affecting a system

"epistemic uncertainty" vs. "error", is a finer distinction depending on whether the ignorance or subjective uncertainty is “inevitable” or "deliberate" in spite of the availability of knowledge.

uncertainty that is "parametric" (associated with model inputs according to the level of information available on those inputs) vs. "modeling uncertainty" affecting the adequacy of the model itself to reality (structure, equations, discretisation, numerical resolution etc.)
12Accordingly, some authors link the choice of particular mathematical settings to such different natures of uncertainty involved in the system studied. On the one hand, the key aleatory/epistemic distinction has motivated the use of double probabilistic settings (Helton 1993; Apostolakis, 1999). This means a probabilistic description of the uncertain (aleatory) states of the system, for instance failure probabilities for uncertain structural failures or gaussian standard deviation for the variability of an environmental feature, upon which a second level of (epistemic) probabilistic description represents the lack of knowledge or expert discrepancies in the proper parameters of such aleatory phenomena. Conversely, such distinction is considered to be impractical to others. Measurement uncertainty is an example of practical source within which it is hard to discern epistemic and aleatory components : the (ISO 1995) international standard has in fact cancelled the mentioning of epistemic/aleatory structure as a mandatory distinction in the analyses and treatments, which receive therefore a common probabilistic sampling approach. Similarly, major developments have approached the impact of uncertainty onto complex numerical models with the help functional and numerical analysis, statistical computer science (design of experiment) under the name of sensitivity analysis : (Cacuci 1980) initiated the exploration of the impact, and major developments occurred since with an extension to global analysis (Sobol, 1993; Saltelli et al, 2004), within which single probabilistic settings are systematically used whatever the nature of uncertainty involved. Eventually, probabilistic treatment of uncertainty has been criticized by a number of authors, particularly for those sources of uncertainty affecting physical systems not amenable to be characterized as aleatory (or randomness) or as variability, hence renewing the early distinctions made by the economic literature (Granger Morgan & Henrion, 1990).
A Generic methodological framework in the industrial world
13Yet, when moving into supporting practical decisionmaking, industrial experience evidences that the variety of settings used to represent uncertainty does preserve a core of essential features, amenable to a unified framework approach. This section will summarise the generic methodology that has emerged as the product of a number of recent papers, and particularly with the ESReDA European industrial group (de Rocquigny, 2008).
Industrial system models considered
14Quantitative uncertainty assessment in the industrial practice typically involves:

a preexisting physical or industrial system or component lying at the heart of the study, that is represented by a preexisting model,

inputs, affected by a variety of sources of uncertainty,

a variety of actions, i.e. design, operation or maintenance options or any type of controlled variables or events that enable to modify or control to some extent the system performance or impact

an amount of data and expertise available to calibrate the model and/or assess its uncertainty,

industrial stakes and decisionmaking that motivate the uncertainty assessment more or less explicitly. They include: safety and security, environmental control, financial and economic optimization etc. They are generally the purpose of the preexisting model, the output and input of which help handling in a quantitative way those stakes within the decisionmaking process.
15The preexisting system may refer to a very wide variety of situations, such as: a metrological chain, a dam or hydraulic facility, a maintenance process, an industrial or domestic site threatened by flood risk, etc. Talking about quantitative studies of uncertainties, that system will generally be modeled by a single numerical or a chain of models referred to as the system model or preexisting model: anything from the straightforward analytical formulae to coupled 3D finite element hydrodynamic or comprehensive hydrological models etc. standing as a numerical function linking output variables z = G(x , d) to a number of continuous or discrete input variables or parameters, where:

some input variables (noted x = (x^{i})_{i=1…p}, underlining denoting vectors throughout the paper) impacting the state of the system are uncertain, subject to randomness, variability, lack of knowledge, errors or any sources of uncertainty,

while other inputs (noted d ) are known or fixed, either being well known (e.g. actions or design choices, controlled experimental factors for calibration …) or being considered to be of secondary importance with respect to the output variables of interest.
16Note that computation of the preexisting model z = G(x , d) for a point value (x ,d) (i.e. not uncertain at all)may require a very variable CPU time: depending on the case, from 10^{4}s to several days for a single run. Within the potentially large number of raw outputs of the models, it is useful to subdistinguish the model output variables of interest that are eventually important for the decision criteria are included formally within the vector z = (z^{v})_{v=1..r}: most of the time, z is a scalar (z) or a smallsize vector (less than 5 components). Although the state of the system may be of high dimension, the decisionmaking process involves essentially one or few variables of interest such as: a physical margin to failure, a net cost, an accumulated environmental dose, or a failure rate in risk analysis.
Probabilistic uncertainty modeling
Rationale for uncertainty modeling
17It is worth insisting on the fact that the undertaking of an uncertainty study is linked to two underlying facts: (i) the state of the system considered, conditional to taking some given actions (d), is imperfectly known at a given time; (ii) some of the characteristics of the state of the system, incorporated in a given type of “performance” or “consequence” represented by the variables of interest (z), are at stake for the decisionmaker. Think of :

the occurrence of flooding over a given dike height within the next year, essential for safety, cannot be predicted exactly because of meteorological unpredictability or lack of knowledge of flooding phenomena,

the true length of a device at the time it is measured within a given experimental setting, an essential feature to secure the fit with the assembling specifications, is imperfectly known due to internal noise or uncontrolled and influential environmental factors affecting the sensor,

etc.
18Although a preexisting model may be defined as a theoreticallycausal or deterministic relationship between precise quantities characterising the state of the system, the consideration of uncertainty about the system means of course that we do not have the ambition of predicting exactly the realization of z based on preciseenough information on predicted values for x and e (i.e. representing the true state of the system). By the above definition, there will always be lack of knowledge to some extent of the state of the system. Modeling under uncertainty means that the information available – be it observations or expertise on those inputs  could merely characterise the possible or likely values for x. Hence, the model should help to infer possible or likely ranges for z deduced from that characterisation of the inputs.
19Whatever the paradigm chosen, there is a need for defining a distribution or measure of uncertainty describing those possible or likely values, which will be referred to as the (input) uncertainty model. The uncertainty model will encode the lack of knowledge remaining after incorporation of all information available to describe the inputs, either observed data or expertise. For instance :

deterministic (interval computation) : uncertain inputs are varying within physicallymaximal or merely plausible ranges

probabilistic : a joint continuous or discrete distribution describing the input uncertainty within random vector X

DempsterShafer : set of belief and plausibility distribution function describing the input uncertainty (Dempster 1967; Helton & Oberkampf 2004)
Standard probabilistic setting
20While literature has discussed to a large extent the pros and cons of various probabilistic or nonprobabilistic paradigms (for instance Helton & Oberkampf 2004), it is assumed later on that probabilistic uncertainty treatment is acceptable for the cases considered : meaning that, whatever the final goals, decision criteria will generally be built upon the probabilistic distribution Z representing uncertainty on the output variables of interest or some of its characteristics, such as : standard deviation, variance or coefficient of variation (i.e. easily interpreted as percentages of relative error/uncertainty) on one or more outputs, confidence intervals on the variable(s) of interest, probabilities of exceedance of a safety threshold etc.
21Formally, consider thus a probabilistic setting, involving a sample space X representing the space of possible values for the input variables x in which X = (X^{i})_{i=1…p} ~ F_{X}(xθ_{X}) whereby F_{X} indicates a joint cumulative probability distribution function (cdf) associated to the vector of uncertain inputs X (capital letters denoting random variables), defining the uncertainty model. Parameters of F_{X} are grouped within the vector θ_{X}: they include for example statistical parameters of position, scale and shape, or moments of marginal laws, coefficients of correlations (or parameters of the copula function), or even extended parameters of a nonparametric kernel model. Consequently, the output variablesZ inherit also a random model, the distribution of which being generally unknown and costly to compute if G is a complex system model : quantifying the uncertain behaviour of such outputs is in fact the purpose of uncertainty propagation methods which will receive more attention hereafter.
22Note also that multiple epistemological interpretations of such a probabilistic setting are possible, while still resulting in the same mathematical objects and computation needs. Indeed, frequent uses are made of such a standard probabilistic setting in regulated practice such as metrology (ISO, 2005), pollutant discharge control or nuclear licensing without positively choosing a single theoretical interpretation. This discussion is closely linked to that of the various natures of uncertainty about the state of the system which may be represented, such as natural time or space variability, lack of knowledge etc. : Section 3 will come back to that discussion. At this stage, only preliminary comments will be given. The frequentist interpretation considers x and z=G(x,d) as observable realizations of an uncertain model standing for an underlying population of truly variable systems (in time, space, pieces …) and the “true” uncertainty model can hence be theoretically inferred from a very large set of data. The subjective interpretation considers probability distributions as subjective preferences of the decisionmaker, without the need for an underlying repeatable phenomenon with observable frequencies as appropriate when dealing with a unique system.
Double probabilistic settings representing separate aleatory/epistemic components
23As discussed in the §2, it is usual in some cases to explicitly distinguish between the aleatory and epistemic components of uncertainty. Well beyond the epistemological debate, such a distinction is crucial because of its significaff analytical, computational and decisionmaking consequences in the sense that the study process could highlight where uncertainty reduction, be it in data collection or model refinement, is most instrumental, yet with a corresponding cost in complexity. Mathematically, key differences regard the probabilistic definition of the uncertainty model, which may be summarised as the potential distinction of two levels requiring simple or double probabilistic settings :

level – 1 (mandatory) : an uncertainty model is built upon the inputs X, representing through a given probabilistic distribution F_{X} (x d_{X}) the extent of uncertainty. This distribution is characterized by given parameters θ_{X, }

level – 2 (optional) : an additional uncertainty model is built to represent the variability or lack of knowledge on the precise values of parameters θ_{X} that prescribe the extent of uncertainty : it takes the form of a supplementary distribution encoding the random behaviour of those parameters _{X}(_{X} ), as a function of “hyperparameters” .
24It is often referred to in the literature as the aleatoryepistemic distinction (Helton & Burmaster, 1996), the level1 modeling the aleatory (or “risk”) component while the level2 models the epistemic (or “uncertainty”) component. Any type of risk or uncertainty analysis would always develop the first level as a model of the uncertainty about the variables or events of interest. Such an uncertainty model (namely its parameters θ_{X}) needs to be estimated or inferred on the basis of all available pieces of information, such as data or expertise. More elaborate interpretations and controversies come up when considering the issue of the lack of knowledge regarding such uncertainty description, as generated for instance by small data sets, discrepancies between experts or even uncertainty in the system model. This is particularly the case in the field of risk analysis or reliability, but happens also in natural risk assessment, particularly flood risk. The incorporation of observed data to infer or validate the level1 probabilistic model, when done through classical statistical estimation, generates statistical fluctuation in the parameter estimates: this is made obvious by confidence intervals around e.g. highlevel flow quantiles or return periods.
25This is even more so when considering the choice of the distribution for an input x^{i} for which traditional hypothesistesting techniques give at most only incomplete answers : is the Gaussian model appropriate for the distribution of x^{i}, as opposed to, for instance, a lognormal or beta model ? This results in a sort of “uncertainty about the uncertainty (or probabilities)”, although this formulation is controversial in itself and may be more specifically referred to as epistemic uncertainty about the aleatory (or variability) characteristics. This is where a potential level2 uncertainty model may be contemplated to represent the uncertainty about the parameters or even the laws themselves through an extended parameterisation. It is sometimes referred to as estimation uncertainty, in order to mark its strong link with the step of the process where the uncertainty model is estimated.
The central concept of the quantity of interest or risk measure
26As introduced earlier, the combination of the system model and the uncertainty model should help to infer possible or likely ranges for z. Within a probabilistic framework, dedicated probabilistic quantities are wanted to measure the likelihood of values for z. Such a quantity used for the inference of the outputs of interest under uncertainty will be called a quantity of interest (de Rocquigny Devictor Tarantola, 2008), otherwise referred to as a performance measure (Aven, 2003) or risk measure in the finance and economics. Some examples :

percentages of error/uncertainty on the variable(s) of interest (i.e. coefficient of variation)

confidence intervals on the variable(s) of interest

quantile of the variable of interest (such as the value at risk or VaR in finance), possibly conditional on penalised inputs

probabilities of exceedance of a safety threshold or of an event of interest (sometimes termed “assurance” when considering conditional nonexceedance probabilities)

expected value (cost, utility, fatalities …) of the consequences

confidence intervals on the probability of an event
27In any case, the quantity of interest or risk measure will be subsequently noted c_{z }(d) since it is computed to represent the likely values of the outputs of interest z and hence crucially depends on the choice of actions d that modifies the anticipated state of the system, albeit to an uncertain extent. A closer mathematical look evidences that one could always write these quantities in the form of a functional of Z,c_{z }= F[Z] typically requiring a computational process including multiple integration (hence not being a straightforward function). Assuming a double probabilistic setting, quantities of interest become formally more complex as they may extend to both levels, such as the range of possible values or confidence interval for the coefficient of variation or the exceedance probability. Mathematically, the quantity of interest c^{(2)}_{z} is then a functional of the extended uncertainty model {F_{X} (. _{X} ,d),_{} }, again involving multiple integrals because a probabilistic secondlevel introduces typically a second layer of integration on top of the level1 quantities of interest. In other words, computing such a level2 expectation involves a double integration, meaning “averaging the risk measure over the uncertainties in the risk components”. Such quantity requires generally a highlycostly double loop of MonteCarlo sampling (such as 100 * 10 000 runs). It is typically integrated in a decision criterion requires that “frequency of the undesired event of interest is less than 1/1000yr at an level of confidence”, encountered in protection against natural risk or integrated probabilistic risk assessment.
28The quantity of interest or risk measure is the key quantitative tool used in make decisions under uncertainty. From a theoretical standpoint, risk and uncertainty studies should eventually help in making decisions about appropriate actions in the context of a relative lack of knowledge about the state of the system. Decisionmaking is always about relative preference used to select within the scope of available actions. Hence the quantity of interest c_{z }(d) should be viewed as an appropriate means to rank the options in the face of uncertainty. Classical literature (Savage 1974 ; Bedford & Cooke 2001) evidences for instance the expected utility as a central quantity of interest, the utility being a scalar function computed on the output of interest u(z) that represents the decisionmaker preferences (e.g. his risk aversion) ; “expected” meaning the expectation over the probabilistic uncertainty model on X, and hence Z=G(X,d), a random vector which is interpreted to represent the decisionmaker’s subjective belief about the possible states of the system.
Generic goals of uncertainty studies
29Yet, in more practical terms, uncertainty studies are not always implemented up to the completion of that ultimate theoretical goal, select within a scope of actions. In many cases, regulation specifies the compliance with an “absolute” criterion. Farstretching decisionmaking would weigh the relative preference between building a nuclear power plaff (e.g. under seismic risk) vs. operating a hydroelectric facility (e.g. under flood risk and climate change impacts); or between accepting a standard device that has been controlled with a cheap (mediumaccuracy) vs. an expensive (highaccuracy) metrological protocol. In fact, operational regulation or practices would often prefer to specify a fixed decision criterion (or risk acceptance criterion, tolerance specification etc.) to comply with, such as :

“design should ensure that the mechanical failure margin should remain positive for the reference seism in spite of uncertainty, with a probability less than 10^{b}to be negative“

“system redundancy design and maintenance should guarantee that frequency of plaff failure should be less than 10^{b} per year, at a 95% confidence level covering the uncertainties”

“there should be less than 3% uncertainty on the declared measurement value for the output of interest”

“up to the  initiator at leasf (e.g. 1000yr flood), installation is safe under designd”

“range of the output variable of interest should be always less than 20%” or “maximal value of the variable of interest should stay below a given absolute threshold”

…
30Such decision criteria mostly appear as tests derived from the quantity of interest, comparing the quantity of interest to a fixed threshold c_{z }(d) < c_{s}. Note that this type of decision criterion may be differently interpreted whether one considers the probabilistic modeling as a representation of subjective degrees of belief over the imperfectly known states of the system (such as in the theory of expected utility) or modeling observable objective frequencies of possible states within a sequence of time of the system or a population of similar systems perform (such as in the frequentist approach), or even both (as in a Bayesian approach).
31To complete the picture it is necessary to consider also that many risk and uncertainty studies are undertaken in relatively upstream stages, not yet facing explicit decisionmaking processes or even less regulatory criterion. This is the case in upstream development of phenomenological models or numerical code qualification. Even when these models and codes are designed to ultimately help decisionmaking or industrial certification, the goal of upstream uncertainty studies may be firstly to understand the extent of uncertainty affecting the model prediction, particularly the relative importance of input uncertainties in order to intensify measurement or modeling efforts on the most important ones. Once a better understanding of the uncertainties and of the behaviour of the system model is available so that it is considered to have the potential of a later operational use, it may be necessary to establish formally its range of validity and to control its residual prediction uncertainty : a more or less sophisticated calibration, validation or qualification process aims to accredit its validity. Although those two goals may be understood as only indirectly linked with decisionmaking, it may be seen that the quantity of interest is still the key quantitative tool involved. For instance, importance ranking of the uncertain inputs or the proper model calibration is different whether variance or a failure probability on the output of interest is selected as the quantity of interest, as discussed in (de Rocquigny et al., 2008). To summarise, industrial practice shows that, most of the time, the goals of any quantitative risk/uncertainty assessment belong to the following four categories:

U (Understand): To understand the influence or rank importance of uncertainties, thereby to guide any additional measurement, modeling or R&D efforts.

A (Accredit): To give credit to a model or a method of measurement, i.e. to reach an acceptable quality level for its use. This may involve calibrating sensors, estimating the parameters of the model inputs, simplifying the system model physics or structure, fixing some model inputs, and finally validating according to a contextdependent level. In a sequential process it may also refer to the updating of the uncertainty model, through dynamic data assimilation.

S (Select): To compare relative performance and optimize the choice of maintenance policy, operation or design of the system.

C (Comply): To demonstrate compliance of the system with an explicit criterion or regulatory threshold (e.g. flood control, dam safety, nuclear or environmental licensing, etc.).
32Moreover, the feedback process proves an essential item in practical risk/uncertainty studies : for instance, failure to secure compliance for a given design may lead to change the actions in order to fulfill the criterion : this may involve changing the design itself in order to cover more conservatively the uncertainties, or investing more on information in order to reduce the sources of uncertainties, provided that those are truly reducible on the basis of additional scientific evidence. This would obviously rely on ranking the importance of uncertain inputs or events, and possibly require a new accreditation of the altered measurement or modeling chain before being able to use it in order to demonstrate compliance.
33Remark also that quantities of interest may differ as to the extent to whichthe quantity computed on the output of interest z correspond to probabilistic or deterministic uncertainty models on the various uncertain inputs x. Indeed, it should be reckoned that in most studies, besides describing some of the uncertain inputs by probabilistic distributions, at leasf some other uncertain inputs (variables or events) are fixed. This is because :

for some model inputs, the decisionprocess will conventionally fix the values despite of the acknowledgement of uncertainties: for comparative purpose, by a conventional “penalisation” i.e. the choice of a fixed “pessimistic” scenario etc.

uncertainties affecting some model inputs are considered to be negligible or of secondary importance with respect to the outputs variables of interest
34Noting x_{pn} these inputs, a more precise specification of the quantity of interest should involve the explicit conditioning on those fixed inputs c_{z }(d ) = c_{z }(d  x_{pn}). As an essential point to build a generic description or risk and uncertainty modeling, such a notation appears to be a convenient mean to unify the deterministic or probabilistic descriptions of uncertainty.
Generic conceptual framework and key steps in any uncertainty study
35At this stage a risk/uncertainty study can be formally summarized as involving a number of generic tasks that are summarized within the following main steps (see also Figure 3):
36(A) specify the system model, variables of interest, the setting and quantity of interest (risk measure) :

choose variables z according to decisionmaking, as well as a system model G(.) predicting z as a function of decision variables (d) and all input characteristics (x) defined as to retrieve information and represent uncertainty affecting the system

choose a representation of uncertainty, e.g. associating probabilistic and deterministic choices

F_{X} (x _{X} ,x_{pn} ,d)

choose a quantity of interest in accordance with the decisionmaking process
37(B) identify (i.e. estimate) the uncertainty model F_{X} (. _{X} ,d) on the basis of available data and expertise
38(C) compute the quantity of interest c_{Z} (d), i.e. propagate the input uncertainty model through G(.) to estimate the output distribution F_{Z} (z d) and/or the associated quantities of interest and sensitivity indices
39Step A has been introduced in the previous paragraph; subsequent step B (data modeling) and C (computational propagation and sensitivity analysis) will be briefly described hereafter.
Step B  uncertainty modeling
40Once the sources of uncertainty and corresponding input variables have been identified, there is inevitably a step of uncertainty modeling (or quantification, characterisation etc. of the sources of uncertainty) that depends on the type of quantities of interest chosen .

in a probabilistic framework, the uncertainty model will be theoretically a joint cdf on the vector of uncertain inputs (x), although it may be more simply specified as a set of simple parametric laws (e.g. Gaussian) on the components with some independence hypotheses or approximate rank correlations.

in a nonprobabilistic framework, the uncertainty model would be for instance a DempsterShafer couple of plausibility / belief functions on x

in a deterministic framework, the maximal range on each component of x

Whatever the framework, there is always however the same need to take into account the largest amount of information :

direct observations on the uncertain inputs, potentially treated in a statistical way to estimate statistical models

expert judgment, under a more or less elaborate elicitation process and mathematical modeling : from the straightforward choice of intervals to more elaborate Bayesian statistical modeling, expert consensus building …

physical arguments : e.g. however uncertain, the input remains positive, or below a known threshold for physical reasons …

indirect observations : this is the case when the model is calibrated / validated and may involve some inverse methods under uncertainty
41in order to build a satisfactorily “uncertainty model”, that is to say a measure of uncertainty on the inputs. Uncertainty modeling may be a resourceconsuming step for data collection ; it appears in fact to be a key step that is very sensitive to the result, depending on the final goal and the quantities of interest involved : for instance, choice of upper bounds or distribution tails become very sensitive if the quantity of interest is an exceedance probability.
Step C/C’  propagation and sensitivity analysis processes
42Once an uncertainty model is developed, the computation of the quantities of interest involves the wellknown uncertainty propagation step (also known as uncertainty analysis). The uncertainty propagation step is needed to transform the measure of uncertainty on the inputs onto a measure of uncertainty on the outputs of the preexisting model. In a probabilistic setting, this means estimating the cdf of z = G(x , d) knowing the cdf of x and given values of d, G(.) being a numerical model. According to the quantity of interest, and to the system model characteristics it may be a more or less difficult numerical step, involving a large variety of methods such as MonteCarlo Sampling, accelerated sampling techniques (Rubinstein 1981), simple quadratic sum of variances (ISO 1995), FormSorm or derived reliability approximations (Madsen et al, 1986), deterministic interval computations etc. Prior to undertaking one of these propagation methods, it may also be desirable to develop a surrogate model (equivalently referred to as response surface, metamodel …), i.e. to replace the preexisting system model by another one that leads to comparable results with respect to the outputs variables and quantities of interest, but that is however much quicker / easier to compute.
43The sensitivity analysis step (or importance ranking) refers to the computation of socalled sensitivity or importance indices of the components of the uncertain inputs variables x with respect to a given quantity of interest on the output z. In fact, this involves a propagation step, e.g. with sampling techniques, but also a posttreatment specific to the sensitivity indices considered, involving typically some statistical treatment of the input/output relations that generate quantities of interest involving both the measure of uncertainty on the outputs and inputs. A large variety of probabilistic sensitivity indices (Saltelli et al 2004) include for instance the: Graphical methods (Scatter plots, cobweb…), Screening (Morris, sequential bifurcations…), Regression based techniques (Pearson, Spearman, SRC, PRCC, PCC, PRCC, etc.), Nonparametric statistics (MannWhitney test, Smirnov test, KruskalWallis test), Variancebased decomposition (FAST, Sobol, Correlation ratios), or local sensitivity indices on exceedance probabilities (FORM). Note that the expression “sensitivity analysis” is taken here in its comprehensive meaning encountered in the specialised uncertainty & sensitivity literature : in the industrial practice, the same expression may refer more approximately to some elementary treatments, such as the oneatatime variations of the inputs of a deterministic model or the partial derivatives. As such, these two kinds of indices are usually not suitable for a consistent importance ranking although they may be a starting point.
44(de Rocquigny et al 2008) evidences that the best choices for such challenging computational steps do not depend on the particular specificities of a physical or industrial context as such, but onto the generic features identified hereabove : the computing cost and regularity of the system model, the dominant final goal, the quantities of interest involved, the dimensions of vectors x and z etc. Recommendations drawn from the longstanding experience of other fields of engineering and risk assessment may hence be retrieved in order to guide the emerging uncertainty applications in hydrological and hydraulic modeling. Indeed, the methodology has now enjoyed considerable industrial dissemination through tutorials and supports the open source development platform Open TURNS.
Examples
45Three typical industrial examples are provided to illustrate the framework, while a much larger spectrum may be found in (de Rocquigny, 2008). Remember that those examples were indeed more an inspiration than a basis to the global methodology introduced hereabove: it was by removing the historic barriers between the sectoral approaches to uncertainty propagation and analysis – barriers that have been observed in recent industrial applications (metrology, reliability, risk analysis …)  that generic uncertainty problems could be formulated and placed in a consistent mathematical setting.
Metrology of environmental emissions
46The first industrial example comes from the domain of metrology. In the domain of metrology or quality control, uncertainty analysis or control is quite a basic requirement associated to the qualification of any measurement device, chain of devices or production process ; through sensitivity analysis, it is also a forcible means to optimise costs and productivity of monitoring and quality control. It may sometimes even be mandatory in the official regulations, such as in nuclear maintenance or radiological protection, or environmental control : for instance, in application of European undertakings relating to the section of the Kyoto protocol concerning industrial emissions, a system of CO_{2} emission permits has been established requiring uncertainty declaration in order to secure the fairness between real emissions and emission rights and stability of the financial markets for emissions permits, through the variance.
 3 Including the noises affecting the operation of the sensor, local fluctuation of the measurement (...)
47Probabilistic approaches are traditional in the field of metrology, whether environmental or not. It is already embodied in international standards such as the "Guide to the expression of uncertainty in measurement (GUM)" (ISO GUM, 1995): rather than aggregate elementary uncertainties in a deterministic manner, it is based on the standard hypothesis that the uncertainty associated with each sensor or measurement operation3, in comparison with a reference value, has an aleatory character that is efficiently modelled by probability calculations. To be precise about the reference value with respect to which uncertain deviations are considered, the standard recommends referring for preference to "the best value available" rather than the physical "true value". On the one hand, the specification or definition of the magnitude to be measured is never infinitely precise: at a microphysical scale, complex temperature or external artefacts (such as background radiations, physicalchemical surface exchanges, …) constantly impact the length of a given rod at a small time scale or even blur the notion of frontier basing the definition of length. On the other hand, should the quantity be very precisely defined, the true value is never observable, but only approached by sensors based on metrological benchmarks that are themselves uncertain. Zooming further down into quantum mechanics, Heisenberg uncertainty principle establishes that any physical system is intrinsically uncertain, and perturbed by observational devices. The GUM thus recommends not working on "error", defined as the absolute variance between measurement result and (unobservable) true value, but rather on "uncertainty" representing the fluctuation of the result of a measurement supposed to have reasonably corrected all the systematic effects compared to a benchmark. The aleatory variable representing this uncertainty may include a "bias" (which therefore remains relative) if its expectation value is not null, even if some people reserve the term "uncertainty" to the aleatory variable of expectation value null, restricted to fluctuation after deduction of any bias.
48Supposing that this framework is accepted, a probabilistic quantity of interest is required in regulatory studies of uncertainties. This consists in determining "an enlarged uncertainty" c_{z}=%unc_{Z} defined by a confidence interval of = 95% around the measured mass tCO_{2}, the ratio of which should not exceed a given threshold e.g. 3% max. of uncertainties around the declared value. According to a classical Gaussian linear approximation, this criterion may be linked to a given multiple of the coefficient of variation, i.e. the ratio of standard deviation to the measured mass.
49Physically, in the case of the CO_{2} emissions from a coalfired power plaff (Figure 4), direct measurement of emissions by a single sensor has proved unreliable. Several options are possible to measure the number of tons of CO_{2} emitted (tCO_{2}) : (a) flowmeters with integrated weighing mechanisms, (b) inference from the electric power produced for a specific consumption, (c) inventory of stocks and inputs. They all imply the aggregation, through simple analytical operations, of several elementary measurements such as the tonnage of coal consumed, the lower calorific power and the corresponding emission factor or oxidation factors. In the example of option (c), coal consumption Cc is itself deducted from the balance of tonnage supplied and variation in stocks is measured in cubature.
50Mathematically, it is therefore a question of aggregating the uncertainties associated with each variable x^{i} measured by each ith operation or elementary sensor (generally evaluated by the suppliers during design) within the function z = G(x^{1},…x^{p}) = G(x) expressing the variable of interest z, for example the annual tonnage of CO_{2}. In such context, the system model referred to earlier is simply a closedform chain of elementary relations representing the metrological operations. Metrological sources of uncertainty X^{i}are classically modeled as Gaussian: this hypothesis is sometimes supported by argument based on the symmetry of errors and the existence of multiple underlying additive physical phenomena. Yet, when uncertainties are bounded, for example because of a command and control mechanism, the choice of a uniform or even triangular distribution may be preferred. The expected value is generally taken to be equal to the result of the calibrated measurement and the variance supplied by the characteristics of the sensor. Linear or rank correlation coefficients account for potential nonindependence of uncertain inputs.
51Moreover the epistemic (or level2) uncertainty in characterising the aleatory behavior of elementary devices may be explicitly included in metrological uncertainty modeling. For instance, repeatability databased empiric variances can be multiplied by a factor greater than 1, decreasing with the number of measures n used to estimate it, instead of simply taking the gross estimated value: this accounts for the possibility, through the effect of sampling fluctuation, of underestimating the unknown true variance. Yet, such practice is heterogeneous, because n is not always known at the industrial enduser. This question becomes particularly important when the cost of determining sources of uncertainty is high, because of the necessary tests involving burdensome basic operations. For instance in the nuclear field, this includes the reception of fuel assemblies before reloading: the measurement process is very demanding, but the risk associated with too great a level of uncertainty is serious, since mistaken acceptance of a fuel assembly too large for the core grid would result in the unavailability of the plaff.
52Propagation (or aggregation, combination etc.) of elementary uncertainties is undertaken by one of the two methods accepted by international standards: either an approximate linear Gaussian combination, or a more accurate MonteCarlo sampling. Both also enable ranking the relative importance of sources of uncertainty, an output of key value to the industrial process. For the final result, the ith input is only important insofar as both input uncertainty unc_{Xi} is strong and/or the physical model is sensitive to it (via ∂G / ∂X^{i}). In the CO_{2} example, the choice of a measurement chain for subsequent checking of CO_{2} emission was based on a minimised criterion %unc_{Z}., but also used the importance ranking of the sources, ensuring that the heaviest contributions corresponded to sources that could not only be tracked for quality assurance, but were if possible reducible.
Structural reliability and design margins of a mechanical structure
53The second example comes from the field of structural reliability analysis and design margins. Design of industrial structures, be it in the nuclear, aerospace, offshore, transport etc., has generated the need for some rules or codes (Ellingwood, 1994) to prevent failure mechanisms, and secure reliability levels in the face of highly diverse sources of uncertainty affecting the operating systems (variability of material properties, of operational loads, fabrication tolerances, …).
54Beyond empiric design margins, a whole fare of methods are usually referred to in structural reliability analysis (Madsen et al, 1986) to tackle a structure characterized by an event of interest, or more generally a group of events, leading to failure. Failure is a matter of structural definition : it may include a number of socalled failure modes, or physical phenomena such as brutal collapse, or just crack initiation : they may happen under a certain number of conditions on two types variables affecting the system behavior, the design variables (again noted d) and the physical variables (noted x=(xi)i=1…p). A failure function G(x,d), i.e. the system model of the example, encapsulates this knowledge of the different phenomena leading to failure.
55Consider for instance the simplified example of the reactor vessel in nuclear safety : undesired failure could theoretically occur under the effect of abnormal pressuretemperature of the primary fluid, itself subjected to an internal initiator such as the drop in pressure following a pipe break elsewhere in the circuit. Stress upon a preexisting flaw inside the vessel width could then exceed the resilience margin of the material resulting in a failure event either defined as the sudden rupture or, more conservatively, in the initiation or flaw propagation. Failure modeling involves firstly a complex finiteelement thermomechanical model y= M(x,d) predicting stress and temperature fields as a function of numerous variables (properties of materials, flaw characteristics, the thermodynamics of accidental transients, the radiation received over time and the resulting fragilisation, etc.) as well as design or operational conditions d (such as temperature and pressure limits, recovery times etc.). Hence the failure margin z, representing the variable of interest, is computed by subtracting a stress intensity factor (noted K_{1}(y,x,d)) from a toughness function (noted K_{1c}(y,x,d)) defining altogether the failure function G(x,d)= K_{1c }(y,x,d)K_{1}(y,x,d). Hence G(.) depends on numerous parameters (x,…x^{y}) = x subject to multiple uncertainties, such as: properties of materials, flaw characteristics, the thermodynamics of accidental transients, the radiation received over time and the resulting fragilisation, etc..
56The field of structural reliability has a long history of standards and regulations in risk industries such as the nuclear or aerospace. The criteria specify reliability requirements according to the components concerned, that is to say an absence of failure during a given period and for a given scenario, e.g. a conventional accident with hypotheses d concerning the structure. This generally means ensuring that the failure function remains positive over a wide range of possible input values. The traditional method consists in "penalising" with safety margins those variables x that are sources of uncertainty, by applying coefficients or safety factors f ^{i} to the "bestestimate" valuesand verifying reliability z_{pn}=G(x_{pn},d) > 0 by a deterministic calculation using the penalised parameters .
57This approach is generally referred to in the industry as a deterministic approach or if one prefers, as an approach through "penalised scenarios" : it involves an elementary form of deterministic treatment of sources of uncertainty presupposing again that x > G(x,.) considered component per component x^{i} is monotonous, a mostly intuitive situation in fracture mechanics, albeit less straightforward in fluid mechanics. It has the considerable advantage of limiting the number of computations to one or a few runs on potentially complex mechanical models. It however requires agreeing on reasonable upper limit of uncertainty for every component x^{i} (or a lower one, depending on the direction of the monotony). As the identification of physicallyrealistic maximal values may be intractable or lead to controversial expert debates, it can happen that certain limits correspond to approximate quantiles α^{i} for each component implicitly modelled as a random variable. For instance, a resistance property would be taken at its lower 95% value accounting for variability of materials, while a loading variable would be taken at its upper 95% value accounting for lack of knowledge or randomness of the operating conditions. This is also referred as a “partial safety factor” approach (Ellingwood, 1994), whereby safety factors are defined for each partial component x^{i} of the vector x conditioning overall safety. An important literature continues to discuss the interest and inherent conservatism of this kind of approach which in fact aggregates potentially heterogeneous margins reflected by the quantiles, in a barely controllable manner. Experience shows it is quite difficult to translate this into an overall probabilistic risk level that could be comparable to other risk situations.
58For some years now the debate has been fuelled by comparison with approaches known as "probabilistic", explicitly modeling the sources of uncertainty by random variables X ^{i}, focusing on the probabilities of threshold exceedance and failure in the scenario defined by P_{f}(d) = P[G(X,d) < 0], and seeking to ensure that it is very close to 0. In fact it is frequently found that one only "probabilises" part of the uncertain inputs, which in fact comes down to a partial conditioning with the penalised values of other sources. Hence P_{f} will be compared to a threshold, or at least the probability of a reference scenario d^{0}, considered as reliable enough, if an absolute threshold in 10^{b} is too delicate to specify. Formally, these approaches known as "probabilistic" should rather be seen as yet another kind of mixed deterministicprobabilistic uncertainty assessments.
59Faced with this criterion, can uncertainty studies make use of either of the methods of propagation already mentioned in metrology? Note that historic barriers between metrology and the reliability of industrial structures has prevented this question from easily emerging. More deeply, the problem arises essentially because we are interested, in our study of structural reliability, in rare "failure events", that is to say the tailend of the Z distribution. In addition, such events are often associated with a preexisting system G(.) that takes far longer to calculate than a closedform metrology formula. In this context the Gaussian and linear hypotheses associated with Taylor quadratic approximations often prove false, while MCS requires too large a number of simulations to stabilise the estimator of P_{f}.
60Aside from the use of methods to reduce the variance of MCS (by reduction of the conditional dimension of MCS, importance sampling etc.), calculation strategies were historically developed specifically to evaluate the probability of exceeding a threshold, and were known in reliability research as First (or Second) Order Reliability Methods, (FORM or SORM). If the underlying approximations allow, FORM and SORM considerably reduce the computational load required to evaluate a very weak P_{f}. These methods also generate importance factors that rank sources of uncertainty: unlike the metrological example, they are ranked with regard to a quantity of interest of threshold exceedance instead of with regard to the average uncertainty (the coefficient of variation) of z. Yet, handling quantities of interest such as rare probabilities on complex system models remains a domain of challenging mathematical research both from the points of view of uncertainty propagation and sensitivity analysis.
An educational example – flood risk
 4 No conclusions may be drawn out of this paper regarding the real flood prediction methods nor the f (...)
61The final example has been developed as a realistic benchmarking and educational example for uncertainty modeling throughout the French industry (de Rocquigny, 2006). While not being used as such in real studies in the particular domain of flood risk4, it was designed to incorporate several aspects that are typical for risk analyses in various domains. The quantity of interest is the probability to surpass a threshold and information availability is typical for industrial studies in the sense that input variables are affected both by aleatory and epistemic uncertainty; although virtual and data available only on some of the uncertain inputs, with heterogeneous amount and quality. The preexisting system is in this case a residential area bordered by a dyke protecting from uncertain flood events.
62Two main output variable of interest are involved. Safetywise, the key variable is the overspill (Figure 5) caused by the rise s=z_{c}z_{d} in waterlevel (z_{c}) above the dyke crest (z_{d}). A hydraulic model relates it to a number of features undergoing both natural meteorological and geomorphological variability and lack of knowledge: river flow (q), riverbed elevations (up and downstream: z_{m} and z_{v}) and the state of the river, typically represented through Strickler's friction coefficient k_{s}. Another variable is important in an economic perspective: the complete cost c_{c} aggregating both the investment cost c_{i} depending on the control variable d = z_{d} (dike height) and the cost of damage c_{d} if there is an overspill. A related financial model may itself incorporate a new source of uncertainties representing the cost of any em> apoverspill c_{'s)ficient k.oertainty}
ctors &)fierm eliabili38 an of any em an becocks isultors industrnng fra(bmarginwre ahis papm edamfriction exareates an elage cFigure 4 – Thermal power p5h2> Fotnotecalorporae/image/782/img4.jpg" rel="iconSet">5img src="docannexe/image/782/img4.jpg" alt="Figure 4 5 Thermal power p5h2> FotnotecalorporaextIconAccess">Zoom Original (jpeg5 44k)
50Mathematical6ample coE are int salculahis ure. Ths of
50Mathematical6usions mossible inty propagation and analybe retrieveain. Zuss. The tras inpotentiof propase ities of int(ntioseo whichTli> 1)t, the i Reltingm>prof intare typie accurate ients, uncertainty; although highlyation and sensi expecteursue is genX iciently mm> verbsoel. Ace invalueckon absk leinty propagshain. Zof suon reasonacertaintyof the input/output relan somities clnaceial procder yead of simplyaiverbe orpora: eption of fueln true variu nd e_{1WW8Num29 }
indirect ob="paranumbTinty , or phy"listic u"aional in: eptiond criterion ntainty for evld result indirect ob="paranumbTintrobabiicitly modelincpicy contro"aional in Allion in ord iof
indirect ob="paranumbTint" emissicitly modelingaional in: ntial arlf a based o,afectsing tesenting the cosal or not.he compuf the resurberating con,, with heterogeypically representatives. h heteroge sulates tlnaceial proreferred. Thecrickler's faintya meateadnction ble mved unr_{c , abilthe of ong burdenolof simplyaimi resna dyy contro ld r/sub>}tandard devtng wi_{c guurvteriocex}<1000 has nt(ble he coe)rd to the av devtng wi_{ct of o "the bee resur verbc guurvteriocex}<1000 has n. Mem>crecise aboutee ficrstudies ofassoclf a based o,aortard inaance o mved un>_{cis the probability to(mashnputs< “the coefootnotecal”age prof iy faSELECT” Figure 4 – Thermal Ttdecoration:undererif;">'T failNy cRoman'/em>'T failNy cRoman'/em>6ven b580img src="docannexe/image/782/img4.jpg" alt="Figure 4 6ven b480img srrmal Tli> 1: Vuantityi The traional in thation and analyt/output relaootnot highlextIconAccess">Zoom Original (jpeg6 44k)}
50Mathematical6instanceItat the irent conntin fes. Ankler'real flst and inherentionalp>. Ye conditipotenti 5) ction ireferredood predicootnotecal oge od risklue tissugain "foe funcel flrof in (c) invesk industri effie computeferredHA rela 0um chain onriers beugh ethods to redeading tld, ths were histog fragilisati remain
that inputthe keown as "probabiork, while a partieading to at chanput variis the probability to(in thial essicitly modelinage
60Aside from thstructuradrawn oudl probastal conte res elsewherTli>
2esummcause wn of anytitye sensor. Linearork, while
i=1…pMthis contssion /rical >i=1…py and design margins ofio
< i=1…pF rical ><
i=1…pi=1…pi=1…pi=1…pk
i=1…pi=1…pc<
i=1…p uncertainty assc <
i=1…pf
i=1…pi=1…pi=1…pi=1…p4<
An e10ucational exa10u>I y is typiccal resead:ors &) weren somessimil analysis.esourcies cal lose 60Aside from ths now thAoverspnsityc th al reseatainty (hainrminis>etcm>pron and analythat inpood pr wn of elev contyrealistic tinty by random variables mo< em>tad r/subthroughout the Frenc< em>:casgy? Note th(he possh throtainty (hssugaaticaon sty,ective:f knowlce shows it,eyof the inpuin of c(ation i/sub>dfof the ftives. h heteroge Need essif the ved oveownse scenar nottyrenhain fes. ? Norith heterogf eitheggn maturapot variis ities ge throughout the Fr. Kerse this ini the sub>lues of ohionltapproabi ation sts y is typic emiss lea of the unc h heteroge ed both by e partiwor^{he sense a bd:oarticua measz}56The field of6956&a m;_{prtex syste asscreferred. Th ,>iratere thi The t oemg>5izemi Tem>helcon,,atio pumbrealistyi orksz. Yet,5% value a10%marking af uncertfower ood,irbi The t o“aches knch inee at chanpnal 5%,a10%mirb30%mld result n) of z. Yet,” tem is sityc thsed folistic chaineuin of cities onuttynemm>rhypotheses/sub>dfof thnd the relmaximal valplpote trabs propagaintractebateparveincimea (Cpossi&a m; depending on the contwhein of fted 56The field of7e use of portanttemps caseicallyynd of ty llyhile a emissis y is typitandardn somistirtantin th deeplyobd of mbemc risk analon ktive sules. h heteroged both by obability toriction a fted k} _{sk}_{ss analyses in ta meas upon es iy are ranortes it to aa class="fon in ord ie beown tonoc emissdn som toiademe of the , a startind maefec w, snumband harge tivslcgalon ulturm ofo othem>MCetho a emissir somrt papm p60Aside from t7mple hasas suco/emrcittex syste asscaechniemer aleatetrologd of(Beck &a m; true vomb component xown toeduction his papmuirede isHing on ities ge true vomb component propagd/spa inpualgecithm iation iypothesesociate,pnal WhiCrécyy fa7) }An e11ucational exa11">Nm>(xe prccal resead:ohionen sy is ocexmuiredilure theporstic ben 60Aside from t7ut variaU heterogf eitheggciavittatioinaance o mdergoing bs g otainty (eporsld resulfted (xtityoductiv crecise about; ble mvemodeltex syste assctathdigm,r systemprrecise aboutee fi ong burdeity of ais gen> elsewherand lintee fi sgrnd wehe(>10 too 3 stoo "cey 0 too 5 stoo "recise about): (i) repr"robabilistic uncertainty ass"ctathdigm,r(ii) oductiv ch ns ion in ord s oce rlisticc 56The field of7ample coH interesthroughout tobaheocexazforpmuiredeflreferred. Th, abiliuteqdchis papropachmarkn of em> prruiredilurnaceeld iluresut6ValyNeuows ’s ENIACchichn uliub1946aporae are interete severaforplee="Carlo neuicalicy GBmarginvemodellee="Carlo,m(xe prctic ben ee fairness b the fullt the numbeg bMath relrruiredilutad r An e12ucational exa12"aU heterogf ev contyrealis,e ow"pt ewledge: riirrm eto evrarveincimea
60Aside from t74zh>z =g nehelconrrol vasble uncertainty ass",agilisatiin thial essicitly modelinyt/output rticoion of pon eading to i by random variablesytandardStep As ofasegf work 60Aside from t7instanceWte as yetnrier A reare a redeslce ishput rela aggregatZ on the contcan l and g balon keterest he Frenchic tintincidient ppan stylesntity of interest incice satder y:st incice sarutrabriterionates tally tautthresaken tomensionsed foe ,cerest are involcan b (hainrministead mporatolntincidient ous dos, iton (ValyNeuows 6&a m;etcm>prtex system modole witumb etcmcan b (hs: ef a chmarkindos, iton ation and analyionateseaferred. Theandtex system mod as sucsiyl awere histomension >_{} aggregatZ on the cont toin thsaer he inu, ,perm eliaasegfuncto evalo upon hubseq
60Aside from t76ple hasasenrand lemae variancendustriistomenmajd rlues of od thntedebicall upon "irrm eto evrarveincimea" (Dupueing u2)hod comporlly r/output relahe metrolo devidentieory ae mv tjcos e beand inhlis,erol variable ctfsn tjcost rhm> uncertainty asscional in: own to , nehelconntakeem>
abinriespa
60e f">instancemntioSeo which>cm radm>60Aside from t7 is geneE sill. A: the sense f worketivi Tzzmitstes "neuical"e ctors &)f, d tofte aeatura" hrst both from thle of ie sampla d boe inaiggaaticaoinli,<. A relat>prruur ecanfor evf tyt < or realis,eown tows yetr krn "foedold rthroughout the i with ihere heterogers &)fifntial elev contt and e cont/output rs b gclass huailufuldomdths mbemna0footyries g ucatio">45
An e13ucational exa13mbCu of snalysis.furtion e points
60Aside from t78pron and ana inshoi spically rep ; althoy are ranmpbates, it ofgilid specifebrrboernd the rel. A: the sense ff numeutthe ke ( they areinn margins o,eyof the ii,(xe prfgialys , ntiontc. theerest veincimea > elseednd of tpindomutropagaii asps,cities on elseednole wiiretivity analyriction a mporatn fes. Ankexis vari the sense t a re (Step C), ailisatitc.), calcunainn st(Step C’)dirbotuctiv oabilitiluob as ativgof ie sainaance ottes ta inriistomen importanginved folistic satiin to ownset,<. n ecown tocloletiolificatioresulionliedt both frommeutthe ke ailisatigialys tn fes. Ated prndustriistoure. Thrdp emesulrtcice sam> uncertainty asscf workemitstes f tplis m thntesignodelat of oeote the sense or realiss,cgioe fu sub>f tem>(xtitytivmbemsaintyof the inpuhe Frenchailiintysub>s invemuirediluob 60Aside from t79etcm>pr awere histo importa( true vombobai: unli,ereandedge: alys tctivd argeimporatn fes. Ankexis varit a re ntiontc. the,iasegelx sl referredshse risk . A relathe insemved unranswntabutr anothee i), > eioomen importart papowsagqes ofasegtivi Tzpropacreliabilesetpron and ana inob as asugg t riskshoi ppan styletainty (hainrminis>etcm>pr mved unre points lsiab,ouor realisse as yetr koite isteadw pr wcapicatyzvioralyslatolnt anytityem>(xe prfgidtyof the inpuslgecithm e of the , iademr oveAsi ee fairness bincg afe pumbr:f knowlcem ofasegvers virtual anoanotecal ossugs,eown to tof owhodelotjcos ic tintinc ben eg bots ci: unltinip tand s GBmargithare typidemisslissefon e pointsb gclassrt paphe sn tdela ethods to redtthdito eval al reseatandtexity analysnditc.), calcunainn stlification lues virtual anoanotm>ereandedge: alys ,sown to gf int menmajd rtc.), calc,nitat tworty nptiluresut660Aside from t8e use ofN and g anae are int>ertaioure. Thsn of onns allow,s innd geinc ben ,ti38 aie: the sense teamtanginem>z = G(xzeffhnps f tp>MCethocr Ths innd geid mailu,barely eralistrapid bamuirandnally rep ow"pt ewledg,airrm eto eretcm>prhe metrolo devycenrpress slwheinhe compuf thossugs itiespumbr: nottyrenhaiaeatthe sense tlure ,yl and geecal nentnvironm radiahaiial
abinriespa60e f">6ple hast T A rews yetnrmdoumin r e heteroge are ranMathdoxectivedeittemps f tp>e &) tel. Active suld .
60Aside from t8mple hasComme cy cte as yetn and g balexercisqes ofaseghs: ef a e vombly reprees innditi,ich heterogf eihatyrown tocan nee &) trt dircumsielb < al e with metrolo devyory aenchaieittemps f tmporatartictivmbemsna0footyries g ucatio">46pla>, abiliui od riskswhein ofleatrtitc.), cate the sense mutmuirana iy akeelatkn of em>< of a,od thnteionliedte pointsbabiliute sense rsquite difinaancunce ott =kuesut6byy a refrom tes e d byagies peopve suly eralisthypof (xtitya sssn and rrecise aboute(i asp C), aantiothatyr>pron and ana in the,ie sillyt < mbiYet,5
Topeg bMage iv class="textI>
atio"imbographyef="docanstal copa
60n cl">Bimbographyontsize/
p class="texte" dir========================================= class="texte" dir=Andrianov G., Burriel S., Cambier S., Dutfoy A., DutkaMalen I., de Rocquigny E., Sudret B., Benjamin P., Lebrun R., Mangeant F., Pendola M. (2007) Open TURNS, an Open Source initiative to Treat Uncertainties Risks’N Statistics in a structured industrial approach, Proc. of ESREL 2007, Stavanger, Norway
Apostolakis G. (1990) The Concept of Probability in Safety Assessments of Technological Systems. Science 1990;250(4986):13591364.
Apostolakis, G. (1999). The distinction between aleatory and epistemic uncertainties is important; an example from the inclusion of aging effects in the PSA. PSA'99, Washington DC, USA.
Aven, T. (2003), Foundations of Risk Analysis, Wiley
Beck, J.V. and Arnold, K.J. (1977) Parameter Estimation in Engineering and Science, Wiley.
Beck, M.B. (1987) Water Quality Modeling: A Review of the Analysis of Uncertainty, Wat. Res. Research, Vol. 23, No 8:13931442.
Bedford T., Cooke R. (2001), Probabilistic Risk Analysis – Foundations and Methods, Cambridge University Press
Cacuci, D.G., et al. (1980), Sensitivity Theory for General Systems of Nonlinear Equations, Nucl. Sc. & Eng. 75
Celeux G., Grimaud A., Lefebvre Y., de Rocquigny E. (2007), Identifying intrinsic variability in multivariate systems through linearised inverse methods, INRIA Research Report, December 2007
Cooke R. (2001), Experts in Uncertainty, Oxford University Press, New York
Cover T.M. and Thomas J.A. (1990, 2006), Elements of Information Theory, Wiley
De Crécy, A. (1997) CIRCE: a tool for calculating the uncertainties of the constitutive relationships of Cathare2, 8^{th} International Topical Meeting on Nuclear reactor ThermoHydraulics (NURETH8), Kyoto.
de Rocquigny E. (2006) La maîtrise des incertitudes dans un contexte industriel: 1^{ère} partie – une approche méthodologique globale basée sur des exemples; 2^{nd} partie – revue des méthodes de modélisation statistique, physique et numérique. Journal de la Société Française de Statistique, vol. 147, n° 4, 33106.
de Rocquigny E., Cambier S. (2008) Inverse probabilistic modeling through nonparametric simulated likelihood, Preprint submitted to Inverse Problems in Science and Engineering.
de Rocquigny E., Devictor N., Tarantola S. (editors) (2008) Uncertainty in industrial practice, A guide to quantitative uncertainty management, Wiley.
Dempster, A. P. (1967). Upper and lower probabilities induced by a multivalued mapping, Annals of Math. Statistics 38: 325339.
Dupuy, J.P. (2002), Pour un catastrophisme éclairé – Quand l’Impossible est certain, Ed. du Seuil
Ellingwood BR. Probabilitybased codified design: past accomplishments and future challenges. Structural Safety 1994 ;13:159176.
Frey, H.C., Rhodes D.S. (2005), Quantitative Analysis of Variability and Uncertainty in with Known Measurement Error : Methodology and Case Study, Risk Analysis Vol. 25, N°3
Granger Morgan, M., Henrion M. (1990) Uncertainty – A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis, Cambridge University Press.
Hamby DM. (1994) A Review of Techniques for Parameter Sensitivity Analysis of Environmental Models. Environmental Monitoring and Assessment 1994;32(2):135154.
Helton, J. (1993) Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal, Rel. Eng. & Syst. Saf., 42, 327367.
Helton, J.C., Burmaster, D.E. (1996) Guest Editorial : Treatment of Aleatory and Epistemic Uncertainty in Performance Assessments for Complex Systems, Rel. Eng. & Syst. Saf., 1996;54(23):9194.
Helton, J.C., Oberkampf W.L. (editors) (2004) Alternative Representations of Epistemic Uncertainty, Special Issue of Rel. Eng. & Syst. Saf., vol. 85 n°13.
ISO GUM (1995), Guide to the expression of uncertainty in measurement (G.U.M.), European PreStandard ENV 13005
Knight, F.H. (1921), Risk, Uncertainty and Profit, Hart, Schaffner & Marx
Kurowicka, D., and Cooke, R.M. (2002), Techniques for generic probabilistic inversion, Probabilistic Safety Assessment and Management, E.J. Bonano et al. (eds), Elsevier, 15431550.
Madsen HO, Kenk S & Lind NC (1986). Methods of Structural Safety, PrenticeHall Inc.
Oberkampf, W.L. et al. (2002), Error and uncertainty in modeling and simulation, Rel. Eng. & Sys. Saf., 75
Parry GW, Winter PW. (1981) Characterization and Evaluation of Uncertainty in Probabilistic Risk Analysis. Nuclear Safety 1981;22(1):2842.
PatéCornell ME. (1996) Uncertainties in Risk Analysis: Six Levels of Treatment Reliability Engineering and System Safety 1996;54(23):95111.
Quiggin J., (1982), a theory of anticipated utility, Journal of Economic Behavior and Organization, vol 3, pp 323343
Rubinstein RY. (1981) Simulation and the MonteCarlo Method. Wiley. and Rubinstein R.Y., Kroese D.P. (2007) Simulation and the MonteCarlo Method, 2^{nd} edition, Wiley.
Saltelli A., Tarantola, S., Campolongo F. & Ratto M. (2004) Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models, John Wiley and Sons.
Savage L.H. (1957), The Foundations of Statistics, Dover Publication Inc.
Shannon, C.E., (1948), A mathematical theory of communication, Bell Systems Technical Journal 27
Sobol, I.M. (1993), Sensitivity estimates for nonlinear mathematical models, Mathematical Modeling and Computational Experiments
Talagrand, O. (1997) Assimilation of observations, an introduction, J. Meteor. Soc. Japan, 75, 191201.
Tarantola, A. (1987) Inverse Problem Theory and methods for data fitting and model parameter estimation, Elsevier  Amsterdam et Tarantola, A. (2004) Inverse Problem Theory and methods for model parameter estimation, SIAM.
Von Neumann J., Morgenstern O., (1944), Theory of games and economic behavior, Princeton University Press
Walter, E., Pronzato, L. (1994) Identification de Modèles Paramétriques à partir de données expérimentales, Coll. MASC, Masson.
Top of page
Notes
1 European Safety, Reliability and Data Association, www.esreda.org
2 Open Treatment of Uncertainty Risk aNd Statistics : www.openturns.org (Andrianov et al, 2007).
3 Including the noises affecting the operation of the sensor, local fluctuation of the measurement point around the average physical quantity sought, error in reading a counter or in rounding the measurement displayed, etc.
4 No conclusions may be drawn out of this paper regarding the real flood prediction methods nor the flood risk methodology used or advocated by Electricité de France
5 See the commentary by J.P. Dupuy (Dupuy, 2002) on the fact that at a certain stage uncertainty is no longer the real obstacle to the application of “precautionary” decisions, which becomes more a question of belief.
6 There would no doubt be a certain logical paradox in any attempt to pretend being able to do so.
Top of page
List of illustrations
Title
Figure 1: A European panel of recent industrial uncertainty studies (de Rocquigny et al 2008).
URL
http://journals.openedition.org/sapiens/docannexe/image/782/img1.jpg
File
image/jpeg, 56k
Title
Figure 2 – The preexisting or system model and its inputs / outputs
URL
http://journals.openedition.org/sapiens/docannexe/image/782/img2.jpg
File
image/jpeg, 44k
Title
Figure 3 – generic conceptual framework
URL
http://journals.openedition.org/sapiens/docannexe/image/782/img3.jpg
File
image/jpeg, 100k
Title
Figure 4 – Thermal power plant emissions (left) and metrological steps (right)
URL
http://journals.openedition.org/sapiens/docannexe/image/782/img4.jpg
File
image/jpeg, 44k
Title
Figure 5 – Flood risk model
URL
http://journals.openedition.org/sapiens/docannexe/image/782/img5.jpg
File
image/jpeg, 24k
Title
Table 1: Various possible approaches to uncertainty treatment of flood levels
URL
http://journals.openedition.org/sapiens/docannexe/image/782/img6.jpg
File
image/jpeg, 108k
Top of page
References
Electronic reference
E. de Rocquigny, « Quantifying uncertainty in an industrial approach : an emerging consensus in an old epistemological debate », S.A.P.I.EN.S [Online], 2.1  2009, Online since 25 May 2009, connection on 14 December 2017. URL : http://journals.openedition.org/sapiens/782
Top of page
An e11ucational exa11">Nm>(xe prccal resead:ohionen sy is ocexmuiredilure theporstic ben 60Aside from t7ut variaU heterogf eitheggciavittatioinaance o mdergoing bs g otainty (eporsld resulfted (xtityoductiv crecise about; ble mvemodeltex syste assctathdigm,r systemprrecise aboutee fi ong burdeity of ais gen> elsewherand lintee fi sgrnd wehe(>10 too 3 stoo "cey 0 too 5 stoo "recise about): (i) repr"robabilistic uncertainty ass"ctathdigm,r(ii) oductiv ch ns ion in ord s oce rlisticc 56The field of7ample coH interesthroughout tobaheocexazforpmuiredeflreferred. Th, abiliuteqdchis papropachmarkn of em> prruiredilurnaceeld iluresut6ValyNeuows ’s ENIACchichn uliub1946aporae are interete severaforplee="Carlo neuicalicy GBmarginvemodellee="Carlo,m(xe prctic ben ee fairness b the fullt the numbeg bMath relrruiredilutad r An e12ucational exa12"aU heterogf ev contyrealis,e ow"pt ewledge: riirrm eto evrarveincimea
60Aside from t74zh>z =g nehelconrrol vasble uncertainty ass",agilisatiin thial essicitly modelinyt/output rticoion of pon eading to i by random variablesytandardStep As ofasegf work 60Aside from t7instanceWte as yetnrier A reare a redeslce ishput rela aggregatZ on the contcan l and g balon keterest he Frenchic tintincidient ppan stylesntity of interest incice satder y:st incice sarutrabriterionates tally tautthresaken tomensionsed foe ,cerest are involcan b (hainrministead mporatolntincidient ous dos, iton (ValyNeuows 6&a m;etcm>prtex system modole witumb etcmcan b (hs: ef a chmarkindos, iton ation and analyionateseaferred. Theandtex system mod as sucsiyl awere histomension >_{} aggregatZ on the cont toin thsaer he inu, ,perm eliaasegfuncto evalo upon hubseq
60Aside from t76ple hasasenrand lemae variancendustriistomenmajd rlues of od thntedebicall upon "irrm eto evrarveincimea" (Dupueing u2)hod comporlly r/output relahe metrolo devidentieory ae mv tjcos e beand inhlis,erol variable ctfsn tjcost rhm> uncertainty asscional in: own to , nehelconntakeem>
abinriespa
60e f">instancemntioSeo which>cm radm>60Aside from t7 is geneE sill. A: the sense f worketivi Tzzmitstes "neuical"e ctors &)f, d tofte aeatura" hrst both from thle of ie sampla d boe inaiggaaticaoinli,<. A relat>prruur ecanfor evf tyt < or realis,eown tows yetr krn "foedold rthroughout the i with ihere heterogers &)fifntial elev contt and e cont/output rs b gclass huailufuldomdths mbemna0footyries g ucatio">45
An e13ucational exa13mbCu of snalysis.furtion e points
60Aside from t78pron and ana inshoi spically rep ; althoy are ranmpbates, it ofgilid specifebrrboernd the rel. A: the sense ff numeutthe ke ( they areinn margins o,eyof the ii,(xe prfgialys , ntiontc. theerest veincimea > elseednd of tpindomutropagaii asps,cities on elseednole wiiretivity analyriction a mporatn fes. Ankexis vari the sense t a re (Step C), ailisatitc.), calcunainn st(Step C’)dirbotuctiv oabilitiluob as ativgof ie sainaance ottes ta inriistomen importanginved folistic satiin to ownset,<. n ecown tocloletiolificatioresulionliedt both frommeutthe ke ailisatigialys tn fes. Ated prndustriistoure. Thrdp emesulrtcice sam> uncertainty asscf workemitstes f tplis m thntesignodelat of oeote the sense or realiss,cgioe fu sub>f tem>(xtitytivmbemsaintyof the inpuhe Frenchailiintysub>s invemuirediluob 60Aside from t79etcm>pr awere histo importa( true vombobai: unli,ereandedge: alys tctivd argeimporatn fes. Ankexis varit a re ntiontc. the,iasegelx sl referredshse risk . A relathe insemved unranswntabutr anothee i), > eioomen importart papowsagqes ofasegtivi Tzpropacreliabilesetpron and ana inob as asugg t riskshoi ppan styletainty (hainrminis>etcm>pr mved unre points lsiab,ouor realisse as yetr koite isteadw pr wcapicatyzvioralyslatolnt anytityem>(xe prfgidtyof the inpuslgecithm e of the , iademr oveAsi ee fairness bincg afe pumbr:f knowlcem ofasegvers virtual anoanotecal ossugs,eown to tof owhodelotjcos ic tintinc ben eg bots ci: unltinip tand s GBmargithare typidemisslissefon e pointsb gclassrt paphe sn tdela ethods to redtthdito eval al reseatandtexity analysnditc.), calcunainn stlification lues virtual anoanotm>ereandedge: alys ,sown to gf int menmajd rtc.), calc,nitat tworty nptiluresut660Aside from t8e use ofN and g anae are int>ertaioure. Thsn of onns allow,s innd geinc ben ,ti38 aie: the sense teamtanginem>z = G(xzeffhnps f tp>MCethocr Ths innd geid mailu,barely eralistrapid bamuirandnally rep ow"pt ewledg,airrm eto eretcm>prhe metrolo devycenrpress slwheinhe compuf thossugs itiespumbr: nottyrenhaiaeatthe sense tlure ,yl and geecal nentnvironm radiahaiial
abinriespa60e f">6ple hast T A rews yetnrmdoumin r e heteroge are ranMathdoxectivedeittemps f tp>e &) tel. Active suld .
60Aside from t8mple hasComme cy cte as yetn and g balexercisqes ofaseghs: ef a e vombly reprees innditi,ich heterogf eihatyrown tocan nee &) trt dircumsielb < al e with metrolo devyory aenchaieittemps f tmporatartictivmbemsna0footyries g ucatio">46pla>, abiliui od riskswhein ofleatrtitc.), cate the sense mutmuirana iy akeelatkn of em>< of a,od thnteionliedte pointsbabiliute sense rsquite difinaancunce ott =kuesut6byy a refrom tes e d byagies peopve suly eralisthypof (xtitya sssn and rrecise aboute(i asp C), aantiothatyr>pron and ana in the,ie sillyt < mbiYet,5
Topeg bMage iv class="textI>
atio"imbographyef="docanstal copa
60n cl">Bimbographyontsize/
p class="texte" dir========================================= class="texte" dir=Andrianov G., Burriel S., Cambier S., Dutfoy A., DutkaMalen I., de Rocquigny E., Sudret B., Benjamin P., Lebrun R., Mangeant F., Pendola M. (2007) Open TURNS, an Open Source initiative to Treat Uncertainties Risks’N Statistics in a structured industrial approach, Proc. of ESREL 2007, Stavanger, Norway
Apostolakis G. (1990) The Concept of Probability in Safety Assessments of Technological Systems. Science 1990;250(4986):13591364.
Apostolakis, G. (1999). The distinction between aleatory and epistemic uncertainties is important; an example from the inclusion of aging effects in the PSA. PSA'99, Washington DC, USA.
Aven, T. (2003), Foundations of Risk Analysis, Wiley
Beck, J.V. and Arnold, K.J. (1977) Parameter Estimation in Engineering and Science, Wiley.
Beck, M.B. (1987) Water Quality Modeling: A Review of the Analysis of Uncertainty, Wat. Res. Research, Vol. 23, No 8:13931442.
Bedford T., Cooke R. (2001), Probabilistic Risk Analysis – Foundations and Methods, Cambridge University Press
Cacuci, D.G., et al. (1980), Sensitivity Theory for General Systems of Nonlinear Equations, Nucl. Sc. & Eng. 75
Celeux G., Grimaud A., Lefebvre Y., de Rocquigny E. (2007), Identifying intrinsic variability in multivariate systems through linearised inverse methods, INRIA Research Report, December 2007
Cooke R. (2001), Experts in Uncertainty, Oxford University Press, New York
Cover T.M. and Thomas J.A. (1990, 2006), Elements of Information Theory, Wiley
De Crécy, A. (1997) CIRCE: a tool for calculating the uncertainties of the constitutive relationships of Cathare2, 8^{th} International Topical Meeting on Nuclear reactor ThermoHydraulics (NURETH8), Kyoto.
de Rocquigny E. (2006) La maîtrise des incertitudes dans un contexte industriel: 1^{ère} partie – une approche méthodologique globale basée sur des exemples; 2^{nd} partie – revue des méthodes de modélisation statistique, physique et numérique. Journal de la Société Française de Statistique, vol. 147, n° 4, 33106.
de Rocquigny E., Cambier S. (2008) Inverse probabilistic modeling through nonparametric simulated likelihood, Preprint submitted to Inverse Problems in Science and Engineering.
de Rocquigny E., Devictor N., Tarantola S. (editors) (2008) Uncertainty in industrial practice, A guide to quantitative uncertainty management, Wiley.
Dempster, A. P. (1967). Upper and lower probabilities induced by a multivalued mapping, Annals of Math. Statistics 38: 325339.
Dupuy, J.P. (2002), Pour un catastrophisme éclairé – Quand l’Impossible est certain, Ed. du Seuil
Ellingwood BR. Probabilitybased codified design: past accomplishments and future challenges. Structural Safety 1994 ;13:159176.
Frey, H.C., Rhodes D.S. (2005), Quantitative Analysis of Variability and Uncertainty in with Known Measurement Error : Methodology and Case Study, Risk Analysis Vol. 25, N°3
Granger Morgan, M., Henrion M. (1990) Uncertainty – A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis, Cambridge University Press.
Hamby DM. (1994) A Review of Techniques for Parameter Sensitivity Analysis of Environmental Models. Environmental Monitoring and Assessment 1994;32(2):135154.
Helton, J. (1993) Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal, Rel. Eng. & Syst. Saf., 42, 327367.
Helton, J.C., Burmaster, D.E. (1996) Guest Editorial : Treatment of Aleatory and Epistemic Uncertainty in Performance Assessments for Complex Systems, Rel. Eng. & Syst. Saf., 1996;54(23):9194.
Helton, J.C., Oberkampf W.L. (editors) (2004) Alternative Representations of Epistemic Uncertainty, Special Issue of Rel. Eng. & Syst. Saf., vol. 85 n°13.
ISO GUM (1995), Guide to the expression of uncertainty in measurement (G.U.M.), European PreStandard ENV 13005
Knight, F.H. (1921), Risk, Uncertainty and Profit, Hart, Schaffner & Marx
Kurowicka, D., and Cooke, R.M. (2002), Techniques for generic probabilistic inversion, Probabilistic Safety Assessment and Management, E.J. Bonano et al. (eds), Elsevier, 15431550.
Madsen HO, Kenk S & Lind NC (1986). Methods of Structural Safety, PrenticeHall Inc.
Oberkampf, W.L. et al. (2002), Error and uncertainty in modeling and simulation, Rel. Eng. & Sys. Saf., 75
Parry GW, Winter PW. (1981) Characterization and Evaluation of Uncertainty in Probabilistic Risk Analysis. Nuclear Safety 1981;22(1):2842.
PatéCornell ME. (1996) Uncertainties in Risk Analysis: Six Levels of Treatment Reliability Engineering and System Safety 1996;54(23):95111.
Quiggin J., (1982), a theory of anticipated utility, Journal of Economic Behavior and Organization, vol 3, pp 323343
Rubinstein RY. (1981) Simulation and the MonteCarlo Method. Wiley. and Rubinstein R.Y., Kroese D.P. (2007) Simulation and the MonteCarlo Method, 2^{nd} edition, Wiley.
Saltelli A., Tarantola, S., Campolongo F. & Ratto M. (2004) Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models, John Wiley and Sons.
Savage L.H. (1957), The Foundations of Statistics, Dover Publication Inc.
Shannon, C.E., (1948), A mathematical theory of communication, Bell Systems Technical Journal 27
Sobol, I.M. (1993), Sensitivity estimates for nonlinear mathematical models, Mathematical Modeling and Computational Experiments
Talagrand, O. (1997) Assimilation of observations, an introduction, J. Meteor. Soc. Japan, 75, 191201.
Tarantola, A. (1987) Inverse Problem Theory and methods for data fitting and model parameter estimation, Elsevier  Amsterdam et Tarantola, A. (2004) Inverse Problem Theory and methods for model parameter estimation, SIAM.
Von Neumann J., Morgenstern O., (1944), Theory of games and economic behavior, Princeton University Press
Walter, E., Pronzato, L. (1994) Identification de Modèles Paramétriques à partir de données expérimentales, Coll. MASC, Masson.
Top of page
Notes
1 European Safety, Reliability and Data Association, www.esreda.org
2 Open Treatment of Uncertainty Risk aNd Statistics : www.openturns.org (Andrianov et al, 2007).
3 Including the noises affecting the operation of the sensor, local fluctuation of the measurement point around the average physical quantity sought, error in reading a counter or in rounding the measurement displayed, etc.
4 No conclusions may be drawn out of this paper regarding the real flood prediction methods nor the flood risk methodology used or advocated by Electricité de France
5 See the commentary by J.P. Dupuy (Dupuy, 2002) on the fact that at a certain stage uncertainty is no longer the real obstacle to the application of “precautionary” decisions, which becomes more a question of belief.
6 There would no doubt be a certain logical paradox in any attempt to pretend being able to do so.
Top of page
List of illustrations
Title
Figure 1: A European panel of recent industrial uncertainty studies (de Rocquigny et al 2008).
URL
http://journals.openedition.org/sapiens/docannexe/image/782/img1.jpg
File
image/jpeg, 56k
Title
Figure 2 – The preexisting or system model and its inputs / outputs
URL
http://journals.openedition.org/sapiens/docannexe/image/782/img2.jpg
File
image/jpeg, 44k
Title
Figure 3 – generic conceptual framework
URL
http://journals.openedition.org/sapiens/docannexe/image/782/img3.jpg
File
image/jpeg, 100k
Title
Figure 4 – Thermal power plant emissions (left) and metrological steps (right)
URL
http://journals.openedition.org/sapiens/docannexe/image/782/img4.jpg
File
image/jpeg, 44k
Title
Figure 5 – Flood risk model
URL
http://journals.openedition.org/sapiens/docannexe/image/782/img5.jpg
File
image/jpeg, 24k
Title
Table 1: Various possible approaches to uncertainty treatment of flood levels
URL
http://journals.openedition.org/sapiens/docannexe/image/782/img6.jpg
File
image/jpeg, 108k
Top of page
References
Electronic reference
E. de Rocquigny, « Quantifying uncertainty in an industrial approach : an emerging consensus in an old epistemological debate », S.A.P.I.EN.S [Online], 2.1  2009, Online since 25 May 2009, connection on 14 December 2017. URL : http://journals.openedition.org/sapiens/782
Top of page
56The field of7ample coH interesthroughout tobaheocexa
60Aside from t7instanceWte as yetnrier A reare a redeslce ishput rela
60Aside from t76ple hasasenrand lemae variancendustriistomenmajd r
60e f">instancemntioSeo which>cm radm>
An e13ucational exa13mbCu of snalysis.furtion e points
60Aside from t79
60e f">6ple hast T A rews yetnrmdoumin r e heteroge are ranMathdoxectivedeittemps f tp>e &) tel. Active suld .
60Aside from t8mple hasComme cy cte as yetn and g balexercisqes ofaseghs: ef a e vombly reprees innditi,ich heterogf eihatyrown tocan ne
60n cl">Bimbographyontsize/
Andrianov G., Burriel S., Cambier S., Dutfoy A., DutkaMalen I., de Rocquigny E., Sudret B., Benjamin P., Lebrun R., Mangeant F., Pendola M. (2007) Open TURNS, an Open Source initiative to Treat Uncertainties Risks’N Statistics in a structured industrial approach, Proc. of ESREL 2007, Stavanger, Norway
Apostolakis G. (1990) The Concept of Probability in Safety Assessments of Technological Systems. Science 1990;250(4986):13591364.
Apostolakis, G. (1999). The distinction between aleatory and epistemic uncertainties is important; an example from the inclusion of aging effects in the PSA. PSA'99, Washington DC, USA.
Aven, T. (2003), Foundations of Risk Analysis, Wiley
Beck, J.V. and Arnold, K.J. (1977) Parameter Estimation in Engineering and Science, Wiley.
Beck, M.B. (1987) Water Quality Modeling: A Review of the Analysis of Uncertainty, Wat. Res. Research, Vol. 23, No 8:13931442.
Bedford T., Cooke R. (2001), Probabilistic Risk Analysis – Foundations and Methods, Cambridge University Press
Cacuci, D.G., et al. (1980), Sensitivity Theory for General Systems of Nonlinear Equations, Nucl. Sc. & Eng. 75
Celeux G., Grimaud A., Lefebvre Y., de Rocquigny E. (2007), Identifying intrinsic variability in multivariate systems through linearised inverse methods, INRIA Research Report, December 2007
Cooke R. (2001), Experts in Uncertainty, Oxford University Press, New York
Cover T.M. and Thomas J.A. (1990, 2006), Elements of Information Theory, Wiley
De Crécy, A. (1997) CIRCE: a tool for calculating the uncertainties of the constitutive relationships of Cathare2, 8^{th} International Topical Meeting on Nuclear reactor ThermoHydraulics (NURETH8), Kyoto.
de Rocquigny E. (2006) La maîtrise des incertitudes dans un contexte industriel: 1^{ère} partie – une approche méthodologique globale basée sur des exemples; 2^{nd} partie – revue des méthodes de modélisation statistique, physique et numérique. Journal de la Société Française de Statistique, vol. 147, n° 4, 33106.
de Rocquigny E., Cambier S. (2008) Inverse probabilistic modeling through nonparametric simulated likelihood, Preprint submitted to Inverse Problems in Science and Engineering.
de Rocquigny E., Devictor N., Tarantola S. (editors) (2008) Uncertainty in industrial practice, A guide to quantitative uncertainty management, Wiley.
Dempster, A. P. (1967). Upper and lower probabilities induced by a multivalued mapping, Annals of Math. Statistics 38: 325339.
Dupuy, J.P. (2002), Pour un catastrophisme éclairé – Quand l’Impossible est certain, Ed. du Seuil
Ellingwood BR. Probabilitybased codified design: past accomplishments and future challenges. Structural Safety 1994 ;13:159176.
Frey, H.C., Rhodes D.S. (2005), Quantitative Analysis of Variability and Uncertainty in with Known Measurement Error : Methodology and Case Study, Risk Analysis Vol. 25, N°3
Granger Morgan, M., Henrion M. (1990) Uncertainty – A Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis, Cambridge University Press.
Hamby DM. (1994) A Review of Techniques for Parameter Sensitivity Analysis of Environmental Models. Environmental Monitoring and Assessment 1994;32(2):135154.
Helton, J. (1993) Uncertainty and sensitivity analysis techniques for use in performance assessment for radioactive waste disposal, Rel. Eng. & Syst. Saf., 42, 327367.
Helton, J.C., Burmaster, D.E. (1996) Guest Editorial : Treatment of Aleatory and Epistemic Uncertainty in Performance Assessments for Complex Systems, Rel. Eng. & Syst. Saf., 1996;54(23):9194.
Helton, J.C., Oberkampf W.L. (editors) (2004) Alternative Representations of Epistemic Uncertainty, Special Issue of Rel. Eng. & Syst. Saf., vol. 85 n°13.
ISO GUM (1995), Guide to the expression of uncertainty in measurement (G.U.M.), European PreStandard ENV 13005
Knight, F.H. (1921), Risk, Uncertainty and Profit, Hart, Schaffner & Marx
Kurowicka, D., and Cooke, R.M. (2002), Techniques for generic probabilistic inversion, Probabilistic Safety Assessment and Management, E.J. Bonano et al. (eds), Elsevier, 15431550.
Madsen HO, Kenk S & Lind NC (1986). Methods of Structural Safety, PrenticeHall Inc.
Oberkampf, W.L. et al. (2002), Error and uncertainty in modeling and simulation, Rel. Eng. & Sys. Saf., 75
Parry GW, Winter PW. (1981) Characterization and Evaluation of Uncertainty in Probabilistic Risk Analysis. Nuclear Safety 1981;22(1):2842.
PatéCornell ME. (1996) Uncertainties in Risk Analysis: Six Levels of Treatment Reliability Engineering and System Safety 1996;54(23):95111.
Quiggin J., (1982), a theory of anticipated utility, Journal of Economic Behavior and Organization, vol 3, pp 323343
Rubinstein RY. (1981) Simulation and the MonteCarlo Method. Wiley. and Rubinstein R.Y., Kroese D.P. (2007) Simulation and the MonteCarlo Method, 2^{nd} edition, Wiley.
Saltelli A., Tarantola, S., Campolongo F. & Ratto M. (2004) Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models, John Wiley and Sons.
Savage L.H. (1957), The Foundations of Statistics, Dover Publication Inc.
Shannon, C.E., (1948), A mathematical theory of communication, Bell Systems Technical Journal 27
Sobol, I.M. (1993), Sensitivity estimates for nonlinear mathematical models, Mathematical Modeling and Computational Experiments
Talagrand, O. (1997) Assimilation of observations, an introduction, J. Meteor. Soc. Japan, 75, 191201.
Tarantola, A. (1987) Inverse Problem Theory and methods for data fitting and model parameter estimation, Elsevier  Amsterdam et Tarantola, A. (2004) Inverse Problem Theory and methods for model parameter estimation, SIAM.
Von Neumann J., Morgenstern O., (1944), Theory of games and economic behavior, Princeton University Press
Walter, E., Pronzato, L. (1994) Identification de Modèles Paramétriques à partir de données expérimentales, Coll. MASC, Masson.
Notes
1 European Safety, Reliability and Data Association, www.esreda.org
2 Open Treatment of Uncertainty Risk aNd Statistics : www.openturns.org (Andrianov et al, 2007).
3 Including the noises affecting the operation of the sensor, local fluctuation of the measurement point around the average physical quantity sought, error in reading a counter or in rounding the measurement displayed, etc.
4 No conclusions may be drawn out of this paper regarding the real flood prediction methods nor the flood risk methodology used or advocated by Electricité de France
5 See the commentary by J.P. Dupuy (Dupuy, 2002) on the fact that at a certain stage uncertainty is no longer the real obstacle to the application of “precautionary” decisions, which becomes more a question of belief.
6 There would no doubt be a certain logical paradox in any attempt to pretend being able to do so.
Top of pageList of illustrations
Title  Figure 1: A European panel of recent industrial uncertainty studies (de Rocquigny et al 2008). 

URL  http://journals.openedition.org/sapiens/docannexe/image/782/img1.jpg 
File  image/jpeg, 56k 
Title  Figure 2 – The preexisting or system model and its inputs / outputs 
URL  http://journals.openedition.org/sapiens/docannexe/image/782/img2.jpg 
File  image/jpeg, 44k 
Title  Figure 3 – generic conceptual framework 
URL  http://journals.openedition.org/sapiens/docannexe/image/782/img3.jpg 
File  image/jpeg, 100k 
Title  Figure 4 – Thermal power plant emissions (left) and metrological steps (right) 
URL  http://journals.openedition.org/sapiens/docannexe/image/782/img4.jpg 
File  image/jpeg, 44k 
Title  Figure 5 – Flood risk model 
URL  http://journals.openedition.org/sapiens/docannexe/image/782/img5.jpg 
File  image/jpeg, 24k 
Title  Table 1: Various possible approaches to uncertainty treatment of flood levels 
URL  http://journals.openedition.org/sapiens/docannexe/image/782/img6.jpg 
File  image/jpeg, 108k 
References
Electronic reference
E. de Rocquigny, « Quantifying uncertainty in an industrial approach : an emerging consensus in an old epistemological debate », S.A.P.I.EN.S [Online], 2.1  2009, Online since 25 May 2009, connection on 14 December 2017. URL : http://journals.openedition.org/sapiens/782
Top of page