Technical appendix QCA-analysis
1. The calibrated data-set
The company names have been converted to case numbers below, since there may be sensitive information and because the actual identity of the companies is of less relevance to the reader here.
Case number
|
WOR
|
MAN
|
PRO
|
EXT
|
PUB/PRI
|
COM
|
1
|
1
|
0.66
|
1
|
1
|
0
|
0.66
|
2
|
0
|
0.66
|
0.33
|
1
|
0
|
0.33
|
3
|
1
|
0.33
|
0.33
|
1
|
0
|
0.33
|
4
|
0
|
1
|
0
|
1
|
0
|
0
|
5
|
1
|
0
|
0.33
|
1
|
0
|
0
|
6
|
1
|
1
|
0.66
|
1
|
1
|
0,66
|
7
|
1
|
1
|
1
|
1
|
0
|
1
|
8
|
1
|
1
|
1
|
1
|
1
|
0.33
|
9
|
1
|
0.66
|
0.33
|
1
|
1
|
0.33
|
10
|
1
|
0.33
|
0.33
|
0
|
1
|
0.66
|
11
|
1
|
0.66
|
1
|
1
|
0
|
0.66
|
12
|
0
|
0.66
|
0.66
|
1
|
0
|
0.66
|
13
|
1
|
0.66
|
1
|
1
|
0
|
0,66
|
14
|
0
|
0.66
|
0
|
1
|
0
|
0
|
15
|
1
|
1
|
1
|
1
|
0
|
1
|
16
|
1
|
0
|
0.33
|
0
|
1
|
0.66
|
17
|
1
|
0.66
|
0.33
|
1
|
1
|
0.33
|
18
|
1
|
1
|
1
|
1
|
1
|
1
|
19
|
1
|
0.33
|
0.33
|
1
|
0
|
0.33
|
20
|
1
|
1
|
1
|
1
|
0
|
1
|
21
|
0
|
0.33
|
0
|
0
|
1
|
0
|
22
|
1
|
0.66
|
1
|
1
|
0
|
1
|
23
|
1
|
1
|
1
|
1
|
0
|
1
|
24
|
1
|
1
|
0.66
|
1
|
1
|
1
|
25
|
1
|
1
|
0.66
|
1
|
0
|
1
|
26
|
1
|
0.66
|
0.66
|
1
|
1
|
0
|
27
|
1
|
0.66
|
0.33
|
1
|
1
|
0.33
|
28
|
0
|
0.66
|
1
|
1
|
0
|
1
|
29
|
1
|
1
|
1
|
1
|
0
|
0.66
|
30
|
1
|
0
|
0
|
1
|
0
|
0
|
31
|
1
|
0,66
|
0,33
|
1
|
1
|
0.66
|
32
|
1
|
1
|
0.33
|
1
|
0
|
0.33
|
33
|
0
|
0.66
|
0.66
|
0
|
1
|
0.33
|
34
|
1
|
1
|
1
|
1
|
0
|
1
|
35
|
1
|
0.66
|
1
|
1
|
0
|
1
|
36
|
0
|
0
|
0
|
1
|
0
|
0
|
37
|
0
|
0.33
|
0.33
|
1
|
0
|
0
|
38
|
0
|
0.33
|
0
|
1
|
1
|
0.33
|
39
|
1
|
0.66
|
1
|
1
|
0
|
0.66
|
40
|
1
|
0,33
|
1
|
1
|
0
|
0.66
|
41
|
1
|
0.66
|
1
|
1
|
0
|
0.66
|
42
|
1
|
1
|
0,33
|
1
|
1
|
0.66
|
43
|
1
|
0.66
|
1
|
1
|
0
|
0.66
|
44
|
1
|
0.66
|
0.33
|
1
|
1
|
0.66
|
45
|
1
|
0.66
|
0.33
|
1
|
1
|
0,33
|
46
|
1
|
1
|
0.66
|
1
|
1
|
0.33
|
47
|
1
|
1
|
0.33
|
1
|
0
|
0.66
|
48
|
0
|
0
|
0.33
|
0
|
1
|
0
|
49
|
1
|
0.33
|
0
|
1
|
1
|
0
|
50
|
1
|
0.33
|
0,33
|
1
|
0
|
0
|
51
|
1
|
0.33
|
0.33
|
1
|
1
|
0.66
|
52
|
1
|
0.66
|
0.66
|
1
|
1
|
1
|
53
|
0
|
0,66
|
0
|
1
|
0
|
0.33
|
54
|
1
|
0
|
1
|
1
|
1
|
0.33
|
55
|
0
|
1
|
1
|
0
|
1
|
1
|
56
|
0
|
0.33
|
0
|
1
|
0
|
0.33
|
57
|
1
|
1
|
0.66
|
1
|
1
|
1
|
58
|
0
|
0,66
|
0,33
|
1
|
0
|
0
|
59
|
1
|
0.66
|
0.66
|
1
|
0
|
0.66
|
60
|
0
|
1
|
0.66
|
0
|
1
|
0.33
|
61
|
0
|
0.33
|
0
|
1
|
0
|
0.33
|
2. QCA-analysis
Below various elements of the QCA-analysis are shown and discussed, in particular issues for which there is not room in the article is shown here in full length. So it includes the full analysis of both compliance and non-compliance and various robustness checks, including consistency and frequency thresholds, and remove a condition EXT, with a skewed distribution of cases.
In assessing the consistency and coverage for necessary conditions I apply the thresholds suggested in the literature for consistency of 0.9 (e.g. Schneider and Wagemann 2012, 143), and for the coverage the 0.5 suggested (Schneider and Wagemann 2012, 146). In line with the much of the QCA literature, I denote present conditions and outcomes in capital letters, and non-present ones in lower case. Additional “+” denotes OR and “*” denotes AND in the Boolean expressions.
2.1 Full analysis for Explaining Compliance
Testing for necessary conditions for the outcome (COM)
inclN RoN covN
-------------------------------
1 WOR 0.853 0.469 0.597
2 MAN 0.894 0.676 0.726
3 PRO 0.842 0.787 0.783
4 EXT 0.905 0.215 0.528
5 PUB.PRI 0.410 0.728 0.497
EXT as a trivial condition (COM)
While EXT has a high consistency (above the 0.9 threshold), the low Relevance of Necessity (RoN) (as well as the rather low coverage) indicates EXT it is a trivial condition and the XY-plot below indicates the same, with most cases clustering close to the right side axis (Schneider and Wagemann, 2012: 146).
XY-plots for necessary conditions (COM)
Testing for sufficiency (COM) (consistency 0.8)
In general the consistency level for truth table inclusion is 0.8, however this also depends on the research design (Kahwati and Kane, 2018: 114). It should hence not be just mechanically based on the “standard” in the literature (Schneider and Wagemann 2012, 128). Some yardsticks important for the design include Schneider and Wagemann’s (2012) stating that the more precise the theoretical expectations and the lower the number of cases, the higher the threshold. As I have quite a high number of cases and the theoretical assumptions are not rigorously set, I apply a 0.8 consistency level.
Truth table
OUT: output value
n: number of cases in configuration
incl: sufficiency inclusion score
PRI: proportional reduction in inconsistency
WOR MAN PRO EXT PUB.PRI OUT n incl PRI
18 1 0 0 0 1 1 2 0.985 0.970
31 1 1 1 1 0 1 16 0.954 0.934
23 1 0 1 1 0 1 1 0.867 0.599
32 1 1 1 1 1 1 8 0.807 0.665
27 1 1 0 1 0 0 2 0.796 0.493
24 1 0 1 1 1 0 1 0.747 0.252
20 1 0 0 1 1 0 2 0.739 0.384
28 1 1 0 1 1 0 7 0.726 0.496
14 0 1 1 0 1 0 3 0.716 0.602
15 0 1 1 1 0 0 2 0.714 0.598
4 0 0 0 1 1 0 1 0.493 0.000
3 0 0 0 1 0 0 3 0.329 0.000
11 0 1 0 1 0 0 5 0.287 0.000
19 1 0 0 1 0 0 6 0.284 0.000
2 0 0 0 0 1 0 2 0.196 0.000
1 0 0 0 0 0 ? 0 - -
5 0 0 1 0 0 ? 0 - -
6 0 0 1 0 1 ? 0 - -
7 0 0 1 1 0 ? 0 - -
8 0 0 1 1 1 ? 0 - -
9 0 1 0 0 0 ? 0 - -
10 0 1 0 0 1 ? 0 - -
12 0 1 0 1 1 ? 0 - -
13 0 1 1 0 0 ? 0 - -
16 0 1 1 1 1 ? 0 - -
17 1 0 0 0 0 ? 0 - -
21 1 0 1 0 0 ? 0 - -
22 1 0 1 0 1 ? 0 - -
25 1 1 0 0 0 ? 0 - -
26 1 1 0 0 1 ? 0 - -
29 1 1 1 0 0 ? 0 - -
30 1 1 1 0 1 ? 0 - -
XY-Plots for sufficiency
Solution terms
There is some discussion in the literature over which solution to present; cf. the discussion between (Baumgartner and Thiem 2020) on one side, and (Dusa 2019a, 2019b) and (Schneider 2016) on the other side. In the article I present the enhanced intermediate solution, while the other terms are included here.
The conservative solution
First I find the conservative solution which does not include any simplifying assumptions based on the logical remainders.
n OUT = 1/0/C: 27/34/0
Total : 61
M1: WOR*MAN*PRO*EXT + WOR*PRO*EXT*pub.pri + WOR*man*pro*ext*PUB.PRI => COM
inclS PRI covS covU
------------------------------------------------------
1 WOR*MAN*PRO*EXT 0.899 0.846 0.662 0.220
2 WOR*PRO*EXT*pub.pri 0.815 0.757 0.474 0.032
3 WOR*man*pro*ext*PUB.PRI 0.985 0.970 0.042 0.042
------------------------------------------------------
M1 0.821 0.741 0.736
The parsimonious solution
Then the parsimonious solution is presented. Here I include all logical remainders, which contribute to making the Boolean expression as parsimonious as possible. The logical remainders here are called simplifying assumptions.
n OUT = 1/0/C: 27/34/0
Total : 61
Number of multiple-covered cases: 16
M1: WOR*ext + (WOR*MAN*PRO + WOR*PRO*pub.pri) => COM
M2: WOR*ext + (WOR*MAN*PRO + man*PRO*pub.pri) => COM
M3: WOR*ext + (WOR*PRO*pub.pri + MAN*PRO*EXT*PUB.PRI) => COM
--------------------------
inclS PRI covS covU (M1) (M2) (M3)
-----------------------------------------------------------------------
1 WOR*ext 0.660 0.485 0.042 0.031 0.031 0.031 0.042
-----------------------------------------------------------------------
2 WOR*MAN*PRO 0.901 0.846 0.673 0.000 0.220 0.534
3 WOR*PRO*pub.pri 0.815 0.757 0.474 0.022 0.032 0.474
4 man*PRO*pub.pri 0.802 0.502 0.171 0.032 0.043
5 MAN*PRO*EXT*PUB.PRI 0.807 0.665 0.220 0.000 0.220
-----------------------------------------------------------------------
M1 0.802 0.717 0.736
M2 0.854 0.779 0.747
M3 0.802 0.717 0.736
Simplifying assumptions (parsimonious solution)
$M1
WOR MAN PRO EXT PUB.PRI
17 1 0 0 0 0
21 1 0 1 0 0
22 1 0 1 0 1
25 1 1 0 0 0
26 1 1 0 0 1
29 1 1 1 0 0
30 1 1 1 0 1
$M2
WOR MAN PRO EXT PUB.PRI
5 0 0 1 0 0
7 0 0 1 1 0
17 1 0 0 0 0
21 1 0 1 0 0
22 1 0 1 0 1
25 1 1 0 0 0
26 1 1 0 0 1
29 1 1 1 0 0
30 1 1 1 0 1
$M3
WOR MAN PRO EXT PUB.PRI
16 0 1 1 1 1
17 1 0 0 0 0
21 1 0 1 0 0
22 1 0 1 0 1
25 1 1 0 0 0
26 1 1 0 0 1
29 1 1 1 0 0
30 1 1 1 0 1
The intermediate solution
Finally I turn to the solution presented in the paper, the intermediate one. In the intermediate solution only logical remainders that are easy counterfactuals are included. The easy counterfactuals for the intermediate solution are defined via my theoretical expectations, where I expect all five conditions to have a positive effect on the outcome (as explained previously in the paper). Accordingly I use the code “dir.exp = c(1,1,1,1,1)” in SetMethods.
n OUT = 1/0/C: 27/34/0
Total : 61
From C1P1, C1P2, C1P3:
Number of multiple-covered cases: 16
M1: WOR*ext*PUB.PRI + WOR*PRO*EXT*pub.pri + (WOR*MAN*PRO*EXT) => COM
M2: WOR*ext*PUB.PRI + WOR*PRO*EXT*pub.pri + (WOR*MAN*PRO*PUB.PRI) => COM
-------------------
inclS PRI covS covU (M1) (M2)
----------------------------------------------------------------
1 WOR*ext*PUB.PRI 0.660 0.485 0.042 0.031 0.042 0.031
2 WOR*PRO*EXT*pub.pri 0.815 0.757 0.474 0.032 0.032 0.474
----------------------------------------------------------------
3 WOR*MAN*PRO*EXT 0.899 0.846 0.662 0.000 0.220
4 WOR*MAN*PRO*PUB.PRI 0.814 0.665 0.231 0.000 0.220
----------------------------------------------------------------
M1 0.802 0.717 0.736
M2 0.802 0.717 0.736
Easy counterfactuals for intermediate solution
WOR MAN PRO EXT PUB.PRI
22 1 0 1 0 1
26 1 1 0 0 1
30 1 1 1 0 1
Prime implicant chart – Intermediate solution
18 23 31 32
WOR*ext x - - -
WOR*MAN*PRO - - x x
WOR*PRO*pub.pri - x x -
man*PRO*pub.pri - x - -
MAN*PRO*EXT*PUB.PRI - - - x
Enhanced standard solutions (ESA)
First I produce a truth table and ESA solutions
Enhanced truth table
OUT: output value
n: number of cases in configuration
incl: sufficiency inclusion score
PRI: proportional reduction in inconsistency
WOR MAN PRO EXT PUB.PRI OUT n incl PRI
18 1 0 0 0 1 1 2 0.985 0.970
31 1 1 1 1 0 1 16 0.954 0.934
23 1 0 1 1 0 1 1 0.867 0.599
32 1 1 1 1 1 1 8 0.807 0.665
27 1 1 0 1 0 0 2 0.796 0.493
24 1 0 1 1 1 0 1 0.747 0.252
20 1 0 0 1 1 0 2 0.739 0.384
28 1 1 0 1 1 0 7 0.726 0.496
14 0 1 1 0 1 0 3 0.716 0.602
15 0 1 1 1 0 0 2 0.714 0.598
4 0 0 0 1 1 0 1 0.493 0.000
3 0 0 0 1 0 0 3 0.329 0.000
11 0 1 0 1 0 0 5 0.287 0.000
19 1 0 0 1 0 0 6 0.284 0.000
2 0 0 0 0 1 0 2 0.196 0.000
1 0 0 0 0 0 0 0 - -
5 0 0 1 0 0 0 0 - -
6 0 0 1 0 1 0 0 - -
7 0 0 1 1 0 ? 0 - -
8 0 0 1 1 1 ? 0 - -
9 0 1 0 0 0 0 0 - -
10 0 1 0 0 1 0 0 - -
12 0 1 0 1 1 ? 0 - -
13 0 1 1 0 0 0 0 - -
16 0 1 1 1 1 ? 0 - -
17 1 0 0 0 0 0 0 - -
21 1 0 1 0 0 0 0 - -
22 1 0 1 0 1 0 0 - -
25 1 1 0 0 0 0 0 - -
26 1 1 0 0 1 0 0 - -
29 1 1 1 0 0 0 0 - -
30 1 1 1 0 1 0 0 - -
Conservative enhanced solution
M1: WOR*MAN*PRO*EXT + WOR*PRO*EXT*~PUB.PRI -> COM
inclS PRI covS covU
---------------------------------------------------
1 WOR*MAN*PRO*EXT 0.899 0.846 0.662 0.220
2 WOR*PRO*EXT*~PUB.PRI 0.815 0.757 0.474 0.032
---------------------------------------------------
Parsimonious enhanced solution
M1: WOR*MAN*PRO*EXT + WOR*PRO*EXT*~PUB.PRI -> COM
M2: WOR*MAN*PRO*EXT + ~MAN*PRO*EXT*~PUB.PRI -> COM
M3: WOR*PRO*EXT*~PUB.PRI + MAN*PRO*EXT*PUB.PRI -> COM
--------------------------
inclS PRI covS covU (M1) (M2) (M3)
-------------------------------------------------------------------------
1 WOR*MAN*PRO*EXT 0.899 0.846 0.662 0.000 0.220 0.534
2 WOR*PRO*EXT*~PUB.PRI 0.815 0.757 0.474 0.022 0.032 0.474
3 ~MAN*PRO*EXT*~PUB.PRI 0.802 0.502 0.171 0.032 0.043
4 MAN*PRO*EXT*PUB.PRI 0.807 0.665 0.220 0.000 0.220
-------------------------------------------------------------------------
M1 0.813 0.733 0.694
M2 0.870 0.802 0.705
M3 0.813 0.733 0.694
Contradictory simplifying assumptions – Enhanced intermediate solution
The same logical remainder may in be included in the Boolean minimization for both the outcome and the negated outcome, this is in QCA called contradictory simplifying assumptions. I argue that there are no untenable LR in my design, since all conditions can theoretically and substantive be combined. I then test for CSA in R, but there are none for the intermediate that I emphasise (and present in the analysis).
Final Intermediate enhanced solution
The overall solution produced by the logical minimization (M1 in Table 2) has a consistency above 0.8, which typically is the cut for the overall solution, and the coverage is rather high
M1: WOR*MAN*PRO*EXT + WOR*PRO*EXT*~PUB.PRI -> COM
inclS PRI covS covU
---------------------------------------------------
1 WOR*MAN*PRO*EXT 0.899 0.846 0.662 0.220
2 WOR*PRO*EXT*~PUB.PRI 0.815 0.757 0.474 0.032
---------------------------------------------------
M1 0.813 0.733 0.694
Prime implicant chart – Enhanced intermediate solution
23 31 32
WOR*MAN*PRO*EXT - x x
WOR*PRO*EXT*~PUB.PRI x x -
2.2 Full analysis for Explaining Non-Compliance (com)
Testing for necessary conditions for the outcome (com)
inclN RoN covN
-------------------------------
1 WOR 0.615 0.373 0.403
2 MAN 0.644 0.529 0.490
3 PRO 0.485 0.581 0.423
4 EXT 0.864 0.197 0.472
5 PUB.PRI 0.443 0.730 0.503
-------------------------------
There are no conditions passing the 0.9 threshold of consistency, making the relevance measures (coverage and PRI) less relevant (Oana et al., 2021: 74)
XY-plots for necessary conditions (com)
XY-plots for necessary conditions (com, non-compliance)
Testing for sufficiency (com) (consistency 0.8)
XY-Plots for sufficiency
Solution terms (com)
Truth table
OUT: output value
n: number of cases in configuration
incl: sufficiency inclusion score
PRI: proportional reduction in inconsistency
WOR MAN PRO EXT PUB.PRI OUT n incl PRI
19 1 0 0 1 0 1 6 1.000 1.000
11 0 1 0 1 0 1 5 1.000 1.000
3 0 0 0 1 0 1 3 1.000 1.000
2 0 0 0 0 1 1 2 1.000 1.000
4 0 0 0 1 1 1 1 1.000 1.000
20 1 0 0 1 1 1 2 0.835 0.610
24 1 0 1 1 1 1 1 0.832 0.504
23 1 0 1 1 0 1 1 0.800 0.395
27 1 1 0 1 0 0 2 0.799 0.500
28 1 1 0 1 1 0 7 0.729 0.501
32 1 1 1 1 1 0 8 0.578 0.268
15 0 1 1 1 0 0 2 0.576 0.402
14 0 1 1 0 1 0 3 0.569 0.398
18 1 0 0 0 1 0 2 0.507 0.000
31 1 1 1 1 0 0 16 0.322 0.032
1 0 0 0 0 0 ? 0 - -
5 0 0 1 0 0 ? 0 - -
6 0 0 1 0 1 ? 0 - -
7 0 0 1 1 0 ? 0 - -
8 0 0 1 1 1 ? 0 - -
9 0 1 0 0 0 ? 0 - -
10 0 1 0 0 1 ? 0 - -
12 0 1 0 1 1 ? 0 - -
13 0 1 1 0 0 ? 0 - -
16 0 1 1 1 1 ? 0 - -
17 1 0 0 0 0 ? 0 - -
21 1 0 1 0 0 ? 0 - -
22 1 0 1 0 1 ? 0 - -
25 1 1 0 0 0 ? 0 - -
26 1 1 0 0 1 ? 0 - -
29 1 1 1 0 0 ? 0 - -
30 1 1 1 0 1 ? 0 - -
The conservative solution
n OUT = 1/0/C: 21/40/0
Total : 61
Number of multiple-covered cases: 0
M1: WOR*man*EXT + wor*man*pro*PUB.PRI + wor*pro*EXT*pub.pri => com
inclS PRI covS covU
--------------------------------------------------
1 WOR*man*EXT 0.851 0.717 0.388 0.388
2 wor*man*pro*PUB.PRI 1.000 1.000 0.080 0.080
3 wor*pro*EXT*pub.pri 0.910 0.890 0.227 0.227
--------------------------------------------------
M1 0.885 0.820 0.695
The parsimonious solution
n OUT = 1/0/C: 21/40/0
Total : 61
Number of multiple-covered cases: 4
M1: wor*pro + man*EXT => com
inclS PRI covS covU
--------------------------------------
1 wor*pro 0.907 0.882 0.329 0.169
2 man*EXT 0.873 0.776 0.548 0.388
--------------------------------------
M1 0.864 0.789 0.717
Simplifying assumptions (parsimonious solution)
$M1
WOR MAN PRO EXT PUB.PRI
1 0 0 0 0 0
7 0 0 1 1 0
8 0 0 1 1 1
9 0 1 0 0 0
10 0 1 0 0 1
12 0 1 0 1 1
The intermediate solution
From C1P1:
M1: ~MAN*EXT + ~WOR*~MAN*~PRO + ~WOR*~PRO*~PUB.PRI -> ~COM
inclS PRI covS covU
-------------------------------------------------
1 ~MAN*EXT 0.873 0.776 0.548 0.388
2 ~WOR*~MAN*~PRO 1.000 1.000 0.217 0.057
3 ~WOR*~PRO*~PUB.PRI 0.910 0.890 0.227 0.090
-------------------------------------------------
M1 0.872 0.801 0.695
Easy counterfactuals (intermediate solution)
WOR MAN PRO EXT PUB.PRI
1 0 0 0 0 0
7 0 0 1 1 0
8 0 0 1 1 1
9 0 1 0 0 0
Prime implicant chart – Intermediate solution
2 3 4 11 19 20 23 24
wor*man x x x - - - - -
wor*pro x x x x - - - -
man*PRO - - - - - - x x
man*EXT - x x - x x x x
man*pub.pri - x - - x - x -
wor*EXT*PUB.PRI - - x - - - - -
Enhanced solutions (com)
Enhanced conservative solution
n OUT = 1/0/C: 21/40/0
Total : 61
Number of multiple-covered cases: 0
M1: WOR*man*EXT + wor*man*pro*PUB.PRI + wor*pro*EXT*pub.pri => com
inclS PRI covS covU
--------------------------------------------------
1 WOR*man*EXT 0.851 0.717 0.388 0.388
2 wor*man*pro*PUB.PRI 1.000 1.000 0.080 0.080
3 wor*pro*EXT*pub.pri 0.910 0.890 0.227 0.227
--------------------------------------------------
M1 0.885 0.820 0.695
Enhanced parsimonious solution
M1: ~WOR*~PRO + ~MAN*EXT -> ~COM
inclS PRI covS covU
----------------------------------------
1 ~WOR*~PRO 0.907 0.882 0.329 0.169
2 ~MAN*EXT 0.873 0.776 0.548 0.388
----------------------------------------
M1 0.864 0.789 0.717
Contradictory simplifying assumptions – Enhanced intermediate solution
I then test for CSA (only for contradictory in R), and find the following:
[1] "1" "7" "8" "9" "10" "12"
New truth table after CSA
We see now that there three LR remainder rows less.
OUT: output value
n: number of cases in configuration
incl: sufficiency inclusion score
PRI: proportional reduction in inconsistency
WOR MAN PRO EXT PUB.PRI OUT n incl PRI
19 1 0 0 1 0 1 6 1.000 1.000
11 0 1 0 1 0 1 5 1.000 1.000
3 0 0 0 1 0 1 3 1.000 1.000
2 0 0 0 0 1 1 2 1.000 1.000
4 0 0 0 1 1 1 1 1.000 1.000
20 1 0 0 1 1 1 2 0.835 0.610
24 1 0 1 1 1 1 1 0.832 0.504
23 1 0 1 1 0 1 1 0.800 0.395
27 1 1 0 1 0 0 2 0.799 0.500
28 1 1 0 1 1 0 7 0.729 0.501
32 1 1 1 1 1 0 8 0.578 0.268
15 0 1 1 1 0 0 2 0.576 0.402
14 0 1 1 0 1 0 3 0.569 0.398
18 1 0 0 0 1 0 2 0.507 0.000
31 1 1 1 1 0 0 16 0.322 0.032
1 0 0 0 0 0 ? 0 - -
5 0 0 1 0 0 ? 0 - -
6 0 0 1 0 1 ? 0 - -
7 0 0 1 1 0 0 0 - -
8 0 0 1 1 1 0 0 - -
9 0 1 0 0 0 ? 0 - -
10 0 1 0 0 1 ? 0 - -
12 0 1 0 1 1 0 0 - -
13 0 1 1 0 0 ? 0 - -
16 0 1 1 1 1 ? 0 - -
17 1 0 0 0 0 ? 0 - -
21 1 0 1 0 0 ? 0 - -
22 1 0 1 0 1 ? 0 - -
25 1 1 0 0 0 ? 0 - -
26 1 1 0 0 1 ? 0 - -
29 1 1 1 0 0 ? 0 - -
30 1 1 1 0 1 ? 0 - -
cases
Enhanced intermediate solution (after CSA)
n OUT = 1/0/C: 21/40/0
Total : 61
From C1P1:
Number of multiple-covered cases: 0
M1: WOR*man*EXT + wor*man*pro*PUB.PRI + wor*pro*EXT*pub.pri => com
inclS PRI covS covU
--------------------------------------------------
1 WOR*man*EXT 0.851 0.717 0.388 0.388
2 wor*man*pro*PUB.PRI 1.000 1.000 0.080 0.080
3 wor*pro*EXT*pub.pri 0.910 0.890 0.227 0.227
--------------------------------------------------
M1 0.885 0.820 0.695
Prime implicant chart – Enhanced intermediate solution
2 3 4 11 19 20 23 24
WOR*~MAN*EXT - - - - x x x x
~MAN*~PRO*EXT - x x - x x - -
~WOR*~MAN*~PRO*PUB.PRI x - x - - - - -
~WOR*~PRO*EXT*~PUB.PRI - x - x - - - -
2.3 Standard robustness checks
Standard QCA robustness checks include changing the consistency threshold, re-calibration and potentially adding or removing cases (Schneider and Wagemann 2012; Oana and Schneider, 2021). I argue that the qualitative calibration secures a high validity of the calibration, but I tested for instance one case where the degree of worker participation was somewhat ambiguous; changing the calibration did not have a substantial effect on the findings. Further, it can be argued that the high number of cases and the qualitative data calibration makes it highly difficult to decide meaningfully, which cases to remove, and the value of the “drop-one sensitivity” test has also been called into question (Krogslund and Michel 2014). I therefore left out this type of robustness test, and mainly checked robustness by altering the consistency threshold instead. I tested my results with the standard test values of a 0.75 threshold and 0.9 threshold (see below). Schneider and Wagemann (2012) suggest that findings are robust if the consistency and coverage (in the original and robustness test) can be substantially interpreted in the same way, which they can.
As an additional robustness test in line with Ragin’s suggestion (2008) of a frequency threshold for the outcome, I conducted the analysis with a frequency threshold of two and three cases, which did not substantially alter the results, but left out solution term 2 and 3 for COM, since both of these have low unique coverage (see below). The results of the robustness checks for non-compliance was a bit more murky (see below), but mainly concerned the public/private dimension, which does not alter my overall findings (given the low consistency of the necessity of this condition).
Consistency levels for COM
Results with 0.9: (enhanced intermediate solution)
M1: WOR*MAN*PRO*EXT*~PUB.PRI -> COM
inclS PRI covS covU
-------------------------------------------------------
1 WOR*MAN*PRO*EXT*~PUB.PRI 0.954 0.934 0.442 -
-------------------------------------------------------
M1 0.954 0.934 0.442
Results with 0.75: (enhanced intermediate solution)
n OUT = 1/0/C: 29/32/0
Total : 61
From C1P1, C1P2, C1P3, C1P4:
Number of multiple-covered cases: 1
M1: WOR*MAN*PRO*EXT + WOR*MAN*EXT*pub.pri + WOR*PRO*EXT*pub.pri + WOR*man*pro*ext*PUB.PRI => COM
inclS PRI covS covU
------------------------------------------------------
1 WOR*MAN*PRO*EXT 0.899 0.846 0.662 0.220
2 WOR*MAN*EXT*pub.pri 0.899 0.859 0.473 0.032
3 WOR*PRO*EXT*pub.pri 0.815 0.757 0.474 0.032
4 WOR*man*pro*ext*PUB.PRI 0.985 0.970 0.042 0.042
------------------------------------------------------
M1 0.799 0.714 0.768
Consistency levels for com
Results with 0.9: (enhanced intermediate solution)
From C1P1:
M1: ~WOR*~MAN*~PRO*PUB.PRI + ~WOR*~PRO*EXT*~PUB.PRI +
~MAN*~PRO*EXT*~PUB.PRI -> ~COM
inclS PRI covS covU
-----------------------------------------------------
1 ~WOR*~MAN*~PRO*PUB.PRI 1.000 1.000 0.080 0.080
2 ~WOR*~PRO*EXT*~PUB.PRI 0.910 0.890 0.227 0.090
3 ~MAN*~PRO*EXT*~PUB.PRI 1.000 1.000 0.296 0.159
-----------------------------------------------------
M1 0.954 0.940 0.465
Results with 0.75: (enhanced intermediate solution)
From C1P1:
M1: WOR*~MAN*EXT + ~PRO*EXT*~PUB.PRI + ~WOR*~MAN*~PRO*PUB.PRI -> ~COM
inclS PRI covS covU
-----------------------------------------------------
1 WOR*~MAN*EXT 0.851 0.717 0.388 0.229
2 ~PRO*EXT*~PUB.PRI 0.882 0.845 0.420 0.261
3 ~WOR*~MAN*~PRO*PUB.PRI 1.000 1.000 0.080 0.080
-----------------------------------------------------
M1 0.854 0.773 0.729
Frequency Threshold (COM)
Results with a frequency threshold of 2
Enhanced intermediate solution
From C1P1:
M1: WOR*MAN*PRO*EXT -> COM
inclS PRI covS covU
----------------------------------------------
1 WOR*MAN*PRO*EXT 0.899 0.846 0.662 -
----------------------------------------------
M1 0.899 0.846 0.662
Results with a frequency threshold of 3 gives the same solution
Frequency Threshold (com)
Results with a frequency threshold of 2
From C1P1:
M1: ~WOR*~PRO*EXT*~PUB.PRI + WOR*~MAN*~PRO*EXT +
~WOR*~MAN*~PRO*~EXT*PUB.PRI -> ~COM
inclS PRI covS covU
----------------------------------------------------------
1 ~WOR*~PRO*EXT*~PUB.PRI 0.910 0.890 0.227 0.227
2 WOR*~MAN*~PRO*EXT 0.923 0.868 0.274 0.274
3 ~WOR*~MAN*~PRO*~EXT*PUB.PRI 1.000 1.000 0.057 0.057
----------------------------------------------------------
M1 0.925 0.893 0.557
Results with a frequency threshold of 3
From C1P1:
M1: ~WOR*~PRO*EXT*~PUB.PRI + ~MAN*~PRO*EXT*~PUB.PRI -> ~COM
inclS PRI covS covU
-----------------------------------------------------
1 ~WOR*~PRO*EXT*~PUB.PRI 0.910 0.890 0.227 0.090
2 ~MAN*~PRO*EXT*~PUB.PRI 1.000 1.000 0.296 0.159
-----------------------------------------------------
M1 0.945 0.930 0.386
2.4 Robustness protocol (Oana and Schneider, 2021)
Oana and Schneider (2021), argues that a consensus on robustness checks have emerged which mean that standard checks should include consistency threshold, frequency cut-offs, re-calibration and potentially adding or removing cases. All of these are dealt with in section 2.3 above. However, Oana and Schneider (2021) argue that we should further conduct three types of robustness checks, which can be said to be the frontier of robustness in QCA methodology (some of them overlap with the robustness tests conducted above, but still moves beyond).
The three tests suggested by Oana and Schneider are sensitivity ranges, fit-oriented tests and case-oriented. I’ll go through each one of these below, conducted on my data set in R. Nonetheless, it is also important to underline that the tests should align with the set-theoretic approach rather than “mimic robustness tests in regression analyses” (Schneider and Wagemann, 2012; cf. Greckhamer et al., 2018).
I start the robustness protocol by producing my own initial solution (IS) (I use the enhanced intermediate solution presented in the article), which is then test against the other solutions in the protocol.
Sensitivity ranges
When testing the sensitivity ranges Oana and Schneider (2021) proposes three calculations: calibration anchors, raw consistency threshold and frequency cutoff. However, as they state (footnote 6, p. 28); “The sensitivity ranges of the calibration anchors do not work for qualitative data (e.g., interview transcripts)”, hence I only calculate the other two sensitivity ranges (raw consistency threshold and frequency cutoff).
Raw consistency threshold
The consistency threshold shows a sensitivity range 0.80, which can also be seen in the enhanced truth table, as there is a case (no 23) that have a consistency value of exactly 0.80.
My Raw Consistency Threshold.: Lower bound 0.8 Threshold 0.8 Upper bound 0.8
Frequency cutoff
N.Cut: Lower bound 1 Threshold 1 Upper bound 1
The frequency cutoff range shows that my results will change by if I change the cut-off by one case. This is very much in line with my expectations as well as the robustness tests above.
Step 3
Next step in the robustness check is: “Produce Alternative Solutions, Taking Into Consideration the Sensitivity Range Analysis and Conceptually Plausible Changes in the Hard Test Range”
Here produce two solutions (since I don’t have a calibration sensivity range). First a test set (TS) TS1 with a consistency threshold of 0.75, and then a TS2 with a frequency cut-off of 2 (rather than 1)
These two TS joined into a TS-list, which are than compared to the “Robust core” (RC) below.
Parameter of fit for RC
Cons.Suf Cov.Suf PRI
Core fit 0.899 0.662 0.846
Robustness Fit-oriented tests
RF_cov RF_cons RF_SC_minTS RF_SC_maxTS
Robustness_Fit 0.954 0.904 0.862 0.93
As seen all the parameters for robustness fit (RFcov, RFcons, RFSCminTS, and RFSCmaxTS) are all less than one meaning a less than perfect overlap between IS and the RC nor IS and the minTS=maxTS. However the parameters are all close to one, indicating that there are no significant robustness issues identified here.
Case-oriented
Here I produce first the robustness plot below,
Robustness Case Parameters
RCRtyp RCRdev RCC_Rank
Robustness_Case_Ratio 0.913 0.75 4
According to Oana and Schneider (2021: 23) the RCRtyp parameter can be understood as per cent of the cases that are robust. The figure in my analysis is 91.3 per cent of the cases are robust. 75 % of the deviant cases (RCRdev) are robust.
$CaseTypes
Robust Typical Cases (IS*MIN_TS and Y > 0.5) :
Boolean Expression: EXT*MAN*PRO*WOR
Cases in the intersection/Total number of cases: 21 / 61 = 34.43 %
Cases in the intersection/Total number of cases Y > 0.5: 21 / 32 = 65.62 %
-------------------
Robust Deviant Cases (IS*MIN_TS and Y < 0.5) :
Boolean Expression: EXT*MAN*PRO*WOR
Cases in the intersection/Total number of cases: 3 / 61 = 4.92 %
Cases in the intersection/Total number of cases Y < 0.5: 3 / 29 = 10.34 %
-------------------
Shaky Typical Cases (IS*~MIN_TS and Y > 0.5) :
Boolean Expression: EXT*~MAN*PRO*~PUB.PRI*WOR
Cases in the intersection/Total number of cases: 1 / 61 = 1.64 %
Cases in the intersection/Total number of cases Y > 0.5: 1 / 32 = 3.12 %
-------------------
Shaky Deviant Cases(IS*~MIN_TS and Y < 0.5) :
Boolean Expression: EXT*~MAN*PRO*~PUB.PRI*WOR
Cases in the intersection/Total number of cases: 0 / 61 = 0 %
Cases in the intersection/Total number of cases Y < 0.5: 0 / 29 = 0 %
-------------------
Possible Typical Cases (~IS*MAX_TS and Y > 0.5) :
Boolean Expression: EXT*MAN*~PRO*~PUB.PRI*WOR
Cases in the intersection/Total number of cases: 1 / 61 = 1.64 %
Cases in the intersection/Total number of cases Y > 0.5: 1 / 32 = 3.12 %
-------------------
Possible Deviant Cases (~IS*MAX_TS and Y < 0.5) :
Boolean Expression: EXT*MAN*~PRO*~PUB.PRI*WOR
Cases in the intersection/Total number of cases: 1 / 61 = 1.64 %
Cases in the intersection/Total number of cases Y < 0.5: 1 / 29 = 3.45 %
-------------------
Extreme Deviant Coverage Cases (~IS*~MAX_TS and Y > 0.5) :
Boolean Expression: ~EXT + ~WOR + ~MAN*~PRO + ~MAN*PUB.PRI + ~PRO*PUB.PRI
Cases in the intersection/Total number of cases: 9 / 61 = 14.75 %
Cases in the intersection/Total number of cases Y > 0.5: 9 / 32 = 28.12 %
-------------------
Irrelevant Cases (~IS*~MAX_TS and Y < 0.5) :
-------------------
Boolean Expression: ~EXT + ~WOR + ~MAN*~PRO + ~MAN*PUB.PRI + ~PRO*PUB.PRI
Cases in the intersection/Total number of cases: 25 / 61 = 40.98 %
Cases in the intersection/Total number of cases Y < 0.5: 25 / 29 = 86.21 %
Interpreting the robustness.
The fit-oriented parameters were all quite close to one, and for the case – oriented parameters these are also indicating a high robustness degree, with only one case being a “shaky case”. Hence I conclude that the robustness protocol does not indicate any substantial
2.5 Skewness
It is obvious from my data set and in line with my qualitative expectation and the initial analysis of the ‘raw’ data that the condition EXT (whether the company experience external pressure or not) is skewed. My expectation is that this condition will only be important for companies experiencing a high degree of external pressure (the condition is crips). A descriptive skewness check shows that 54 of the 61 cases (88.5 %) have full membership (since full membership is the absence of external pressure). If too many cases have a high or low degree of membership in a single condition this may affect the validity of the results (Schneider and Wagemann, 2012232-248; Thomann and Maggetti, 2020: 372). A rule of thumb is that the membership degree should not be > 20 %, which my condition is. However, it does seem like the impact of the skewness of this condition is of less relevance for my analysis. According to Schneider and Wagemann (2012: 232) skewness issues relate to two aspects; trivialness of necessary conditions and simultaneous subset relations. Addressing the issue of trivialness first, I argue that I have substantive and theoretical reasons to include the condition despite the trivialness (yet only if does not alter the overall results), based on the case knowledge. The presence of external pressure does in some of my case overrule the other conditions (see within case analysis in the article). Hence I expect the condition to be trivial for the occurrence of the outcome.
Then turning to the simultaneous subset relations Thomann and Maggetti (2020: 373) states that the proportional reduction in inconsistency measure (PRI) can help detect these (when substantive interpretability is emphasised). The PRI for the occurrence (COM) as well as non-occurrence are all high (see table 3 and 4 in the article) suggesting that the skewness problem may not be problematic for the overall results. Moreover applying the Enhanced standard analysis (ESA, as above and in the analysis see Schneider and Wagemann, 2012) preludes the simultaneous subset relations.
However, to further assess the degree to which the skewness of the condition EXT is a problem for my analysis I run the analysis without the condition to see how it affects my results.
Analysis without EXT (COM)
To test the implication of the skewness of the condition EXT I ran the analysis without the condition. This did not alter the overall results in a substantial way – the solution terms are largely the same as can be seen below, in particular for compliance, hence meeting the recommendations of Schneider and Wagemann, 2012) that the interpretations should not be significantly altered. Some of the fit and threshold changed, but not greatly. However, two of the solution terms for non-compliance did change, but only in the configurations, less so when assessed qualitatively. But some of the consistency values changed, but most of the overall results were not dramatically changed for non-compliance either.
There were no necessary conditions when conducting the analysis without EXT
Truth table (without EXT)
OUT: output value
n: number of cases in configuration
incl: sufficiency inclusion score
PRI: proportional reduction in inconsistency
WOR MAN PRO PUB.PRI OUT n incl PRI
15 1 1 1 0 1 16 0.954 0.934
11 1 0 1 0 1 1 0.867 0.599
16 1 1 1 1 1 8 0.814 0.665
10 1 0 0 1 1 4 0.800 0.546
13 1 1 0 0 0 2 0.796 0.493
12 1 0 1 1 0 1 0.783 0.252
14 1 1 0 1 0 7 0.738 0.496
8 0 1 1 1 0 3 0.716 0.602
7 0 1 1 0 0 2 0.714 0.598
1 0 0 0 0 0 3 0.329 0.000
5 0 1 0 0 0 5 0.287 0.000
9 1 0 0 0 0 6 0.284 0.000
2 0 0 0 1 0 3 0.281 0.000
Parsimonious enhanced solution (without EXT)
n OUT = 1/0/C: 29/32/0
Total : 61
Number of multiple-covered cases: 16
M1: WOR*MAN*PRO + WOR*man*pro*PUB.PRI + (WOR*PRO*pub.pri) => COM
M2: WOR*MAN*PRO + WOR*man*pro*PUB.PRI + (man*PRO*pub.pri) => COM
-------------------
inclS PRI covS covU (M1) (M2)
----------------------------------------------------------------
1 WOR*MAN*PRO 0.901 0.846 0.673 0.136 0.136 0.449
2 WOR*man*pro*PUB.PRI 0.800 0.546 0.137 0.043 0.043 0.043
----------------------------------------------------------------
3 WOR*PRO*pub.pri 0.815 0.757 0.474 0.022 0.032
4 man*PRO*pub.pri 0.802 0.502 0.171 0.032 0.043
----------------------------------------------------------------
M1 0.803 0.719 0.747
M2 0.854 0.780 0.758
Conservative enhanced solution (without EXT)
n OUT = 1/0/C: 29/32/0
Total : 61
Number of multiple-covered cases: 16
M1: WOR*MAN*PRO + WOR*PRO*pub.pri + WOR*man*pro*PUB.PRI => COM
inclS PRI covS covU
--------------------------------------------------
1 WOR*MAN*PRO 0.901 0.846 0.673 0.136
2 WOR*PRO*pub.pri 0.815 0.757 0.474 0.032
3 WOR*man*pro*PUB.PRI 0.800 0.546 0.137 0.043
--------------------------------------------------
M1 0.803 0.719 0.747
Intermediate enhanced solution (without EXT)
n OUT = 1/0/C: 29/32/0
Total : 61
From C1P1, C1P2:
Number of multiple-covered cases: 16
M1: WOR*MAN*PRO + WOR*PRO*pub.pri + WOR*man*pro*PUB.PRI => COM
inclS PRI covS covU
--------------------------------------------------
1 WOR*MAN*PRO 0.901 0.846 0.673 0.136
2 WOR*PRO*pub.pri 0.815 0.757 0.474 0.032
3 WOR*man*pro*PUB.PRI 0.800 0.546 0.137 0.043
--------------------------------------------------
M1 0.803 0.719 0.747
com-analysis without EXT
Parsimonious enhanced solution (without EXT) (com)
n OUT = 1/0/C: 19/42/0
Total : 61
Number of multiple-covered cases: 4
M1: wor*pro + man*PRO + man*pub.pri => com
inclS PRI covS covU
------------------------------------------
1 wor*pro 0.907 0.882 0.329 0.169
2 man*PRO 0.832 0.537 0.340 0.135
3 man*pub.pri 0.894 0.826 0.388 0.114
------------------------------------------
M1 0.875 0.796 0.715
Conservative enhanced solution (without EXT) (com)
n OUT = 1/0/C: 19/42/0
Total : 61
Number of multiple-covered cases: 4
M1: wor*man*pro + wor*pro*pub.pri + WOR*man*PRO + (WOR*man*pub.pri) => com
M2: wor*man*pro + wor*pro*pub.pri + WOR*man*PRO + (man*pro*pub.pri) => com
-------------------
inclS PRI covS covU (M1) (M2)
------------------------------------------------------------
1 wor*man*pro 1.000 1.000 0.217 0.080 0.080 0.080
2 wor*pro*pub.pri 0.910 0.890 0.227 0.090 0.090 0.090
3 WOR*man*PRO 0.827 0.444 0.272 0.135 0.135 0.216
------------------------------------------------------------
4 WOR*man*pub.pri 0.880 0.785 0.251 0.011 0.114
5 man*pro*pub.pri 1.000 1.000 0.296 0.000 0.103
------------------------------------------------------------
M1 0.897 0.830 0.692
M2 0.896 0.826 0.681
Intermediate enhanced solution after CSA (without EXT) (com)
n OUT = 1/0/C: 19/42/0
Total : 61
From C1P1, C1P2, C2P1, C2P2:
Number of multiple-covered cases: 10
M1: wor*man*pro + wor*pro*pub.pri + WOR*man*PRO + (WOR*man*pub.pri) => com
M2: wor*man*pro + wor*pro*pub.pri + WOR*man*PRO + (man*pro*pub.pri) => com
-------------------
inclS PRI covS covU (M1) (M2)
------------------------------------------------------------
1 wor*man*pro 1.000 1.000 0.217 0.080 0.080 0.080
2 wor*pro*pub.pri 0.910 0.890 0.227 0.090 0.090 0.090
3 WOR*man*PRO 0.827 0.444 0.272 0.135 0.135 0.216
------------------------------------------------------------
4 WOR*man*pub.pri 0.880 0.785 0.251 0.011 0.114
5 man*pro*pub.pri 1.000 1.000 0.296 0.000 0.103
------------------------------------------------------------
M1 0.897 0.830 0.692
M2 0.896 0.826 0.681
List of references used for the QCA-analysis and Technical appendix:
Baumgartner, M., & Thiem, A. (2020) Often Trusted but Never (Properly) Tested: Evaluating Qualitative Comparative Analysis. Sociological Methods & Research, 49, 279-311.
Dusa, A. (2019a). QCA with R. A Comprehensive Resource. Springer International Publishing.
Dusa, A. (2019b). Critical Tension: Sufficiency and Parsimony in QCA. Sociological Methods & Research, 51(2), 541-565.
De Block, D., & Vis, B. (2019). Addressing the challenges related to transforming qualitative into quantitative data in qualitative comparative analysis. Journal of Mixed Methods Research, 13(4), 503-535.
Greckhamer, T., Furnari, S, Fiss, P. C., et al. (2018). Studying configurations with qualitative comparative analysis: Best practices in strategy and organization research. Strategic Organization, 16(4), 482-495.
Kahwati, L.C., & Kane, H. L. (2018). Qualitative Comparative Analysis in Mixed Methods Research and Evaluation. SAGE Publications.
Krogslund, C., & Michel, K. (2014). A Larger-N, Fewer Variables Problem? The Counterintuitive Sensitivity of QCA. Newsletter of the American Political Science Association Organized Section for Qualitative and Multi-Method Research, 12, 25-33.
Oana, I.-E., Schneider, C. Q., & Thomann, E. (2021) Qualitative Comparative Analysis Using R. Cambridge University Press.
Oana, I.-E., & Schneider, C. Q, (2021). A Robustness Test Protocol for Applied QCA: Theory and R Software Application. Sociological Methods & Research. SAGE Publications Inc.
Ragin, C. C. (2000). Fuzzy-set Social Science. University of Chicago Press.
Ragin, C. C. (2008). Redesigning Social Inquiry: Fuzzy Sets and Beyond. University of Chicago Press.
Schneider CQ and Wagemann C (2012) Set-Theoretic Methods for the Social Sciences: A Guide to Qualitative Comparative Analysis. Cambridge: Cambridge University Press.
Schneider, C. Q. (2016). Real Differences and Overlooked Similarities: Set-Methods in Comparative Perspective. Comparative Political Studies, 49, 781-792.
Thomann, E., & Maggetti. M. (2020). Designing Research with Qualitative Comparative Analysis (QCA): Approaches, Challenges, and Tools. Sociological Methods & Research, 49(2), 356-386.