Simulation Methods

In order to evaluate the small-sample performance of the uncertain species model, we conducted a small simulation study generating capture history data using known input parameter values. Multistate data were simulated using parameters that were loosely based on estimates from the actual analysis of model, S (species*state) (species*state) p (.) (age+state) (age) (state):

Number of sample periods = 4

N (initial population size of both species for each age, in each state) = 200

B (new recruits of both species to each age, in each state at each sample occasion after occasion 1) = 20

 (apparent survival rate for sp = A, state = 1) = 0.80

 (apparent survival rate for sp = A, state = 2) = 0.35

 (survival rate for sp = B, state = 1) = 0.80

 (survival rate for sp = B, state = 2) = 0.65

 (movement rate for sp = A, ungrazed to grazed habitat) = 0.20

 (movement rate for sp = A, grazed to ungrazed) = 0.0

 (movement rate for sp = B, ungrazed to grazed) = 0.02

 (movement rate for sp = B, grazed to ungrazed) = 0.20

 (capture probability) = 0.50

 (prop. of pop. = species A, age = young, state = 1) = 0.40

 (prop. of pop. = species A, age = adult, state = 1) = 0.15

 (prop. of pop. = species A, age = young, state = 2) = 0.95

 (prop. of pop. = species A, age = adult, state = 2) = 0.85

(prob. [species assigned correctly], age = young) = 0.60

 (prob. [species assigned correctly], age = adult) = 0.90

 (prob. [animal survives capture and is released], state = 1) = 0.80

 (prob. [animal survives capture and is released], state = 2) = 0.90

Particular parameters were then varied in order to draw inferences about two specific questions of interest. First, apparent survival rates of the two species in ungrazed habitat (where sample size was largest) were estimated to be very similar (near 0.8). In order to address the influence of differences between species, we varied apparent survival of meadow vole (denoted species B) in ungrazed habitat from 0.8 to 0.6, 0.4, and 0.2, reflecting larger differences between the two species in this single parameter. In addition, we explored the consequences of increased classification uncertainty by considering ^(a) = 0.6, in addition to the estimated 0.9. For each scenario that we evaluated, 500 simulated data sets were generated. Each capture history in each data set was generated as a multinomial random variable with cell probabilities defined by underlying probabilities of apparent survival, capture, movement, correct classification, and release, and the true species mixture. The generating model was then fit to each data set using program MSSRV_UNK_SPECIES, and estimates of the parameters and their standard errors were computed and then summarized in various ways. For some of the simulations, the estimation procedure failed to converge, and results were discarded. In the tables below we report the number of simulations (out of 500) for which convergence was reached (denoted M).

In addition to estimation under the generating model, the data were treated as known-species data based on the species assignments and then analyzed using a standard multistate model with all parameters except capture probability estimated as separate for the 2 species. This analysis was conducted in MARK (White and Burnham 1999) and was intended to reflect an approach that did not properly incorporate classification uncertainty but was otherwise reasonable. Both programs (MARK and MSSRV_UNK_SPECIES) were run on each simulated dataset. Mean values of the estimates and standard errors were calculated along with the bias of the estimators (bias = ), where  is the estimator being evaluated and n is the number of simulations), the root-mean-squared-error (RMSE), 'TrueSE', and 95% confidence interval coverage of each parameter for each analysis. The RMSE and 'TrueSE' were computed as:

RMSE =  , and TrueSE = , where is the mean of the n estimates and θ is the true parameter used to generate the data. We computed the 95% CI coverage as the proportion of simulated results in which the true parameter value was within the interval defined by the estimated parameter value plus or minus 1.96 times its estimated standard error.

------------------------------------------------------------------

TABLE A1. Simulation results for uncertain species analysis (“US”) vs. naïve analysis (“NVE”): Correct classification probabilities are 0.6 for adults and young of both species.

Parameter	True value	Bias (of point estimate)		True SE		Bias (of estimated SE)		RMSE		95% CI coverage
		US	Nve	US	Nve	US	Nve	US	Nve	US	Nve
S = 0.6:
	0.80	-0.0061	-0.2869	0.1063	0.0661	0.0481	-0.0025	0.1065	0.2948	0.9926	0.0124
	0.35	0.0010	0.0293	0.0554	0.0511	0.0190	-0.0012	0.0555	0.0589	0.9827	0.9084
	0.20	0.0063	0.2005	0.0626	0.0618	0.0328	-0.0044	0.0629	0.2100	0.9901	0.0619
	0.65	-0.0364	-0.2390	0.2223	0.0579	0.1257	-0.0002	0.2252	0.2462	0.9653	0.0173
	0.20	-0.0016	-0.0365	0.0672	0.0542	0.0591	0.0014	0.0672	0.0654	1.0000	0.8589
	0.02	0.0225	0.1120	0.0786	0.0538	0.2569	0.0010	0.0817	0.1244	0.9703	0.4629
	0.20	0.0225	-0.1464	0.1927	0.0381	0.1015	-0.0044	0.1940	0.1514	0.9059	0.1163
S = 0.4:
	0.80	-0.0119	-0.2256	0.1086	0.0555	0.0531	0.0033	0.1093	0.2326	0.9855	0.0337
	0.35	-0.0024	0.0132	0.0517	0.0472	0.0169	0.0001	0.0518	0.0490	0.9880	0.9470
	0.40	-0.0012	0.1006	0.0647	0.0526	0.0290	0.0012	0.0647	0.1137	0.9904	0.5398
	0.65	-0.0416	-0.2469	0.2417	0.0561	0.0965	-0.0012	0.2453	0.2534	0.9325	0.0145
	0.20	-0.0043	-0.0741	0.0695	0.0424	0.0687	0.0014	0.0696	0.0855	0.9976	0.5663
	0.02	0.0057	0.0725	0.0419	0.0389	0.1046	-0.0009	0.0422	0.0824	0.9518	0.5807
	0.20	0.0408	-0.1438	0.2100	0.0375	0.0909	-0.0038	0.2140	0.1488	0.9036	0.1157
S = 0.2:
	0.80	-0.0045	-0.1234	0.1269	0.0520	0.0373	0.0016	0.1270	0.1341	0.9809	0.3771
	0.35	0.0021	0.0154	0.0513	0.0460	0.0139	-0.0003	0.0513	0.0485	0.9833	0.9523
	0.60	-0.0036	0.0380	0.0613	0.0533	0.0158	-0.0039	0.0614	0.0655	0.9785	0.8640
	0.65	-0.0292	-0.2447	0.2284	0.0551	0.1139	-0.0020	0.2303	0.2511	0.9499	0.0167
	0.20	-0.0047	-0.1011	0.0812	0.0338	0.0612	-0.0002	0.0813	0.1068	0.9952	0.1909
	0.02	0.0047	0.0495	0.0295	0.0261	0.0491	0.0009	0.0299	0.0560	0.9642	0.6158
	0.20	0.0459	-0.1427	0.1958	0.0344	0.1115	-0.0007	0.2011	0.1470	0.9403	0.1074
S = 0.0:
	0.80	-0.0041	-0.0019	0.1248	0.0462	0.0310	0.0012	0.1249	0.0463	0.9906	0.9437
	0.35	0.0004	0.0139	0.0468	0.0451	0.0166	-0.0003	0.0468	0.0472	0.9812	0.9413
	0.80	-0.0012	0.0003	0.0521	0.0432	0.0082	-0.0001	0.0521	0.0432	0.9789	0.9554
	0.65	-0.0283	-0.2446	0.2336	0.0512	0.1133	0.0005	0.2354	0.2502	0.9554	0.0023
	0.20	-0.0017	-0.1215	0.0830	0.0260	0.0604	0.0001	0.0830	0.1244	1.0000	0.0235
	0.02	0.0021	0.0329	0.0223	0.0200	0.0244	-0.0008	0.0224	0.0386	0.9812	0.6596
	0.20	0.0530	-0.1415	0.2068	0.0345	0.0990	-0.0017	0.2135	0.1458	0.9319	0.0798

Survival for meadow vole in ungrazed habitat ( ) simulated as [0.2, 0.4, 0.6, 0.8]. M = 404, 415, 419, and 426 for analyses corresponding to ΔS of 0.6, 0.4, 0.2, and 0.0. All other parameters based upon input values detailed in “Simulation methods” section.

------------------------------------------------------------------

TABLE A2. Simulation results for uncertain species analysis (“US”) vs. naïve analysis (“NVE”): Correct classification probabilities are 0.9 for adults and 0.6 for young of both species.

Parameter	True value	Bias (of point estimate)		True SE		Bias (of estimated SE)		RMSE		95% CI coverage
		US	Nve	US	Nve	US	Nve	US	Nve	US	Nve
S = 0.6:
	0.80	-0.0060	-0.1792	0.0918	0.0692	0.0352	-0.0005	0.0920	0.1923	0.9833	0.2547
	0.35	0.0004	0.0143	0.0430	0.0433	0.0184	0.0024	0.0430	0.0456	0.9896	0.9666
	0.20	0.0014	0.1254	0.0515	0.0534	0.0258	-0.0025	0.0516	0.1364	0.9937	0.2860
	0.65	-0.0038	-0.1916	0.1659	0.0660	0.0423	0.0009	0.1659	0.2029	0.9812	0.1900
	0.20	0.0008	-0.0187	0.0544	0.0582	0.0449	-0.0021	0.0544	0.0611	1.0000	0.8747
	0.02	0.0074	0.0832	0.0478	0.0529	0.2226	-0.0024	0.0484	0.0986	0.9708	0.7307
	0.20	0.0195	-0.1062	0.1605	0.0542	0.0848	-0.0034	0.1616	0.1193	0.9645	0.4593
S = 0.4:
	0.80	-0.0043	-0.1467	0.0955	0.0627	0.0292	0.0017	0.0956	0.1596	0.9873	0.3546
	0.35	-0.0020	0.0046	0.0421	0.0443	0.0153	-0.0008	0.0422	0.0445	0.9873	0.9257
	0.40	0.0020	0.0633	0.0558	0.0508	0.0178	-0.0012	0.0558	0.0812	0.9936	0.7558
	0.65	-0.0233	-0.2061	0.1704	0.0642	0.0352	0.0001	0.1720	0.2160	0.9724	0.1210
	0.20	0.0032	-0.0373	0.0610	0.0495	0.0404	0.0011	0.0611	0.0620	0.9979	0.8301
	0.02	-0.0004	0.0448	0.0251	0.0320	0.0777	-0.0016	0.0251	0.0551	0.9660	0.7856
	0.20	0.0253	-0.1053	0.1863	0.0571	0.0530	-0.0076	0.1880	0.1198	0.9363	0.4607
S = 0.2:
	0.80	0.0007	-0.0810	0.0893	0.0593	0.0268	0.0004	0.0893	0.1004	0.9939	0.7184
	0.35	-0.0027	0.0011	0.0404	0.0419	0.0129	0.0001	0.0405	0.0419	0.9878	0.9510
	0.60	-0.0011	0.0236	0.0514	0.0457	0.0097	0.0002	0.0514	0.0515	0.9755	0.9184
	0.65	-0.0073	-0.2031	0.1671	0.0656	0.0278	-0.0033	0.1673	0.2136	0.9776	0.1204
	0.20	0.0004	-0.0625	0.0644	0.0431	0.0372	0.0000	0.0644	0.0759	0.9959	0.6571
	0.02	-0.0010	0.0263	0.0188	0.0211	0.0335	-0.0008	0.0188	0.0337	0.9755	0.8469
	0.20	0.0289	-0.1021	0.1585	0.0527	0.0864	-0.0031	0.1611	0.1150	0.9755	0.4286
S = 0.0:
	0.80	-0.0018	0.0015	0.0922	0.0552	0.0158	-0.0008	0.0922	0.0552	0.9939	0.9426
	0.35	-0.0023	0.0030	0.0414	0.0421	0.0094	-0.0012	0.0415	0.0422	0.9795	0.9324
	0.80	0.0002	-0.0001	0.0434	0.0407	0.0055	-0.0006	0.0434	0.0407	0.9775	0.9570
	0.65	-0.0057	-0.2040	0.1614	0.0613	0.0239	-0.0004	0.1615	0.2132	0.9754	0.0820
	0.20	0.0099	-0.0772	0.0687	0.0384	0.0360	-0.0009	0.0694	0.0863	0.9959	0.4467
	0.02	0.0002	0.0184	0.0139	0.0155	0.0168	-0.0007	0.0139	0.0241	0.9857	0.8648
	0.20	0.0291	-0.0988	0.1527	0.0508	0.0747	-0.0018	0.1555	0.1111	0.9734	0.4713

Survival for meadow vole in ungrazed habitat ( ) simulated as [0.2, 0.4, 0.6, 0.8]. M = 481, 471, 490, and 488 for analyses corresponding to ΔS of 0.6, 0.4, 0.2, and 0.0. All other parameters based upon input values detailed in “Simulation methods” section.

------------------------------------------------------------------

TABLE A3. Simulation results for uncertain species analysis (“US”) vs. naïve analysis (“NVE”): Varying number of individuals of each age in each stratum (N).

Parameter	True value	N	Bias (of point estimate)		True SE		Bias (of estimated SE)		RMSE		95% CI coverage
			US	Nve	US	Nve	US	Nve	US	Nve	US	Nve
	0.80	100	-0.0371	-0.2897	0.1596	0.0894	0.0953	0.0001	0.1639	0.3038	0.9868	0.1404
		200	-0.0061	-0.2869	0.1063	0.0661	0.0481	-0.0025	0.1065	0.2948	0.9926	0.0124
		500	-0.0022	-0.2852	0.0684	0.0402	0.0159	-0.0002	0.0684	0.2883	0.9879	0.0000
		1000	0.0016	-0.2838	0.0463	0.0282	0.0080	0.0002	0.0463	0.2855	0.9720	0.0000
	0.35	100	-0.0134	0.0162	0.0771	0.0692	0.0480	0.0007	0.0783	0.0711	0.9868	0.9430
		200	0.0010	0.0293	0.0554	0.0511	0.0190	-0.0012	0.0555	0.0589	0.9827	0.9084
		500	0.0001	0.0259	0.0310	0.0300	0.0100	0.0016	0.0310	0.0396	0.9859	0.8952
		1000	0.0026	0.0299	0.0219	0.0216	0.0055	0.0009	0.0221	0.0369	0.9880	0.7580
	0.20	100	0.0471	0.2096	0.0768	0.0882	0.0809	-0.0075	0.0901	0.2278	0.9956	0.2544
		200	0.0063	0.2005	0.0626	0.0618	0.0328	-0.0044	0.0629	0.2100	0.9901	0.0619
		500	0.0016	0.1992	0.0386	0.0364	0.0129	-0.0002	0.0387	0.2027	0.9819	0.0000
		1000	0.0001	0.2025	0.0263	0.0275	0.0070	-0.0018	0.0263	0.2045	0.9960	0.0000
	0.65	100	-0.0076	-0.2295	0.2471	0.0813	0.1699	0.0011	0.2472	0.2439	0.9561	0.1842
		200	-0.0364	-0.2390	0.2223	0.0579	0.1257	-0.0002	0.2252	0.2462	0.9653	0.0173
		500	0.0061	-0.2358	0.1632	0.0351	0.0482	0.0014	0.1633	0.2386	0.9839	0.0000
		1000	-0.0054	-0.2358	0.1161	0.0253	0.0141	0.0006	0.1162	0.2373	0.9760	0.0000
	0.02	100	-0.0137	-0.0418	0.1075	0.0810	0.1332	-0.0065	0.1084	0.0912	0.9956	0.8026
		200	-0.0016	-0.0365	0.0672	0.0542	0.0591	0.0014	0.0672	0.0654	1.0000	0.8589
		500	-0.0011	-0.0425	0.0384	0.0352	0.0223	-0.0004	0.0384	0.0552	0.9940	0.7399
		1000	0.0014	-0.0408	0.0257	0.0257	0.0111	-0.0010	0.0257	0.0483	0.9920	0.5900
	0.02	100	0.0513	0.1088	0.1198	0.0787	0.3790	-0.0061	0.1303	0.1344	0.9868	0.7719
		200	0.0225	0.1120	0.0786	0.0538	0.2569	0.0010	0.0817	0.1244	0.9703	0.4629
		500	0.0002	0.1066	0.0403	0.0350	0.1134	-0.0006	0.0403	0.1123	0.9637	0.0625
		1000	-0.0057	0.1070	0.0254	0.0254	0.0574	-0.0009	0.0261	0.1101	0.8880	0.0000
	0.20	100	-0.0229	-0.1450	0.1865	0.0612	0.2522	-0.0213	0.1879	0.1577	0.9518	0.2763
		200	0.0225	-0.1464	0.1927	0.0381	0.1015	-0.0044	0.1940	0.1514	0.9059	0.1163
		500	0.0239	-0.1483	0.1198	0.0239	0.0414	-0.0015	0.1222	0.1504	0.9839	0.0101
		1000	0.0063	-0.1491	0.0697	0.0157	0.0178	0.0004	0.0700	0.1501	0.9760	0.0000

Correct classification probabilities 0.6 for adults and young of both species, and apparent survival for species B in ungrazed habitat ( ) held constant at 0.2. M = 228, 404, 496, and 500 for analyses corresponding to N of 100, 200, 500, and 1000. All other parameters based upon input values detailed in “Simulation methods” section except B, which was 10% of N.

------------------------------------------------------------------

TABLE A4. Simulation results for uncertain species analysis (“US”) vs. naïve analysis (“NVE”): Estimation of ΔS (= - ) for three values of true ΔS. 95% CI coverage is the proportion of 95% confidence intervals for that contained the true value of ΔS.

			95% CI coverage
True ΔS	US	NVE	US	NVE
0.6	0.586	0.116	0.985	0.000
0.2	0.196	0.035	0.988	0.303
0	-0.008	-0.003	0.990	0.962

Correct classification probabilities 0.6 for adults of both species. All other parameters based upon input values detailed in “Simulation methods” section.

------------------------------------------------------------------

FIG. A1. Root mean squared error (RMSE) of estimated apparent survival rates ( , ) for uncertain species (US) and naïve (Nve) analyses as a function of the true difference in apparent survival rates between the two species ( - ) for true = 0.8 and correct adult classification rates ((a)) of 0.6 and 0.9.

------------------------------------------------------------------

FIG. A2. 95% confidence interval coverage of true parameter values for apparent survival of species A ( ) for uncertain species (US) and naïve (Nve) analyses as a function of the true difference in apparent survival rates between the two species ( - ) for true = 0.8 and correct adult classification rates ((a)) of 0.6 and 0.9.

------------------------------------------------------------------

LITERATURE CITED

White, G. C., and K. P. Burnham. 1999. Program MARK: Survival estimation from populations of marked animals. Bird Study 46(suppl):120–138.