Chapter 6. Impacts of Permanent Housing Subsidy (SUB)
6.3 Impacts on Housing Stability in the Permanent
(SUB) Versus Usual Care (UC) Comparison
As discussed in Chapter 3, proponents of SUB view the crisis of housing affordability as the root cause of homelessness among families. These observers believe that, because fam-ilies who experience homelessness are very poor, they are likely to require long-term rental subsidies to become stably housed. The SUB-versus-UC comparison in the current study provides a direct test of this claim by measuring the effects of making the SUB intervention easily available to families compared with a situation in which permanent hous - ing subsidies are relatively difficult to access in the near term.
What do estimates of the effects of SUB on housing stability tell us? Exhibit 6-4 shows the experimentally based evidence of measured effects on homelessness, housing independence, residential moves, and housing quality. All of the rows of the exhibit (and other impact exhibits in the balance of this
report) have the same format. The first three columns of the exhibit provide information about the SUB families—the number of families with data on a particular outcome and the mean value and standard deviation of the outcome.
The next three columns provide the corresponding informa-tion for the UC families included in this particular pairwise comparison.73 The seventh column is the difference between the mean value (or proportion) of the SUB families and the mean value (or proportion) of the UC families, referred to as the impact of SUB relative to UC.74 Asterisks to the right of this column denote the statistical significance of the impact estimate, with more asterisks indicating higher levels of statistical significance. The eighth column of the exhibit contains the standard error of the impact estimates, which is used to test for statistical significance and can be used to construct a confidence interval around the impact estimate.
The last column shows the standardized effect size of the impact, calculated by dividing the impact estimate by the standard deviation of the outcome for all families assigned to UC.75 The standardized effect size is thus a measure of
73 The UC families in this comparison are those who could have been randomized to SUB. The mean values of outcomes for all UC families are shown in Chapter 5.
74 As explained in Chapter 4, the mean values and the impact estimate are regression adjusted for baseline covariates.
75 The standard deviations for the entire UC group are shown in Chapter 5.
40
Number of months of SUB receipt 0
Percent of SUB families who ever used SUBa
Exhibit 6-4. SUB Versus UC: Impacts on Housing Stability
Outcome SUB UC ITT Impact Effect
sizea N Mean (SD) N Mean (SD) Impact (SE) Homelessness or doubled up during the followup period
At least 1 night homelessb or doubled up in past 6 months or in shelter
in past 12 months (%) [Confirmatory]c 529 21.6 (41.2) 415 49.6 (50.1) – 28.0*** (3.1) – 0.49
At least 1 night homelessb or doubled up in past 6 months (%) 529 16.0 (37.3) 415 40.9 (49.0) – 24.9*** (3.0) – 0.45 At least 1 night homelessb in past 6 months (%) 529 10.5 (31.5) 414 26.4 (43.7) – 15.9*** (2.6) – 0.33 At least 1 night doubled up in past 6 months (%) 530 12.2 (33.3) 415 30.6 (45.6) – 18.4*** (2.7) – 0.35 Any stay in emergency shelter in months 7 to 18 after RA (%) 530 14.8 (34.9) 415 27.8 (45.9) – 12.9*** (2.6) – 0.25 Number of days homelessb or doubled up in past 6 months 528 20.4 (52.2) 413 51.5 (74.6) – 31.2*** (4.4) – 0.37 Number of days homelessb in past 6 months 527 10.5 (36.9) 410 24.0 (51.3) – 13.5*** (3.1) – 0.24 Number of days doubled up in past 6 months 529 12.4 (39.0) 415 33.9 (63.5) – 21.5*** (3.6) – 0.29
Housing independence
Living in own house or apartment at followup (%) 530 73.0 (44.5) 415 57.9 (49.7) 15.1*** (3.0) 0.27 Living in own house or apartment with no housing assistance (%) 530 9.7 (29.3) 415 32.8 (46.2) – 23.0*** (2.8) – 0.43 Living in own house or apartment with housing assistance (%) 530 63.2 (48.2) 415 25.1 (43.7) 38.1*** (3.0) 0.78
Number of places lived
Number of places lived in past 6 months 528 1.4 (1.0) 415 1.8 (1.2) – 0.4*** (0.1) – 0.26
Housing quality
Persons per room 526 1.2 (0.8) 398 1.6 (1.2) – 0.4*** (0.1) – 0.28
Housing quality is poor or fair (%) 527 25.4 (43.4) 411 34.1 (47.6) – 8.7*** (3.1) – 0.16
SUB = permanent housing subsidy. UC = usual care.
ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.
*/**/*** Impact estimate is significantly different from 0 at the .10, .05, and .01 levels, respectively, using a two-tailed t-test.
a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.
b The definition of homeless in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.
c After adjustment for multiple comparisons, the impact on the confirmatory outcome is statistically significant at the .01 level for the SUB-versus-UC comparison.
Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 5 and Appendix B for outcome definitions.
Sources: Family Options Study 18-month followup survey; Program Usage Data
impact relative to natural variability in the outcome. Such standardized effect sizes are a conventional way to compare impact magnitudes across outcomes and domains with dif-ferent scales. For example, one may compare the standardized effect sizes for housing stability outcomes in Exhibit 6-4 with those for other outcomes in other domains shown in this chapter. Standardized effect sizes may also allow for the size of effects found in this study to be compared with the size of effects in other studies.
Exhibit 6-4 shows that the SUB intervention reduced the experience of subsequent stays in shelter or places not meant for human habitation during the 20-month followup period by a large amount. The first row of the exhibit shows evidence for the confirmatory outcome of the study: (1) at least 1 night in shelter or a place not meant for human habitation or dou-bled up in the past 6 months (from the followup survey), or (2) at least 1 night in emergency shelter in the past 12 months
(from the study’s Program Usage Data). Of the families in the UC sample, 50 percent experienced one of these two situations.
For the SUB group, that proportion declined to 22 percent, representing a reduction in homelessness of 28 percentage points and hence eliminating more than one-half of the home - lessness captured by this measure. This impact is highly sta-tistically significant (even after the adjustment for multiple comparisons).76
The following discussion addresses estimated impacts for three outcomes constructed solely from survey data: (1) at least 1 night homeless or doubled up in the past 6 months, (2) at least 1 night homeless in the past 6 months, and (3) at least 1 night doubled up in the past 6 months. The impact estimates in these three rows of the exhibit show that, compared with UC, the SUB intervention caused sub - stantial, statistically significant reductions in all three of these survey-based measures of homelessness.77
76 The study estimates impacts on this confirmatory outcome for each of the six paired comparisons and four pooled comparisons. Seven of these estimates have been prespecified as “confirmatory tests.” A multiple comparison procedure is performed to compute adjusted p-values for these tests to reduce the possibility of chance findings of statistical significance. The details of this procedure are provided in Appendix C.
77 All impacts in this table with the exception of the first row are considered exploratory and are not adjusted for the presence of multiple comparisons. Likewise, all impacts in other study domains are also considered exploratory.
Short-Term Impacts of Housing and Services Interventions for Homeless Families 68 Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)
The fifth row of Exhibit 6-4 shows the impact on any stay in emergency shelter in months 7 to 18 after random assignment (measured largely from administrative data).78 About 28 per - cent of UC families spent at least 1 night in emergency shelter during the yearlong period beginning 6 months after random assignment. Only 15 percent of SUB families spent at least 1 night in emergency shelter during this time, a reduction of 13 percentage points. Thus, the shelter usage rate was cut nearly in half by the availability of a housing subsidy.
Exhibit 6-5 provides a more detailed characterization of the effect of SUB relative to UC on emergency shelter stays. It shows month-by-month impacts on emergency shelter stays.
As discussed in Chapter 5, the Program Usage Data are miss - ing the initial shelter stay for about 20 percent of families.
We have no reason to believe, however, that missing data rates are associated with random assignment group (that is,
they are equally likely to be missing for the SUB group as for the group assigned to UC). The data can therefore be used to calculate estimates of impacts without concern for bias.
As can be seen, a gap between the shelter use of SUB and UC families begins to emerge in the third month after study entry and reaches 7 percentage points by the fifth month, with 18 percent of SUB families having at least 1 night in shelter in the fifth month after random assignment compared with a UC rate of 25 percent.79 A difference emerging in the third month is consistent with what one might expect, given that those assigned to SUB needed to go through the process of using the program (having their incomes verified, finding and leasing a unit) and may have remained in shelter during that process. This gap of 6 to 9 percentage points remains through the 18th month. From the 10th month onward, the share of SUB families in shelter is much less than one-half the proportion of UC families, a notable reduction.
Exhibit 6-5. SUB Versus UC: Percent of Families With at Least 1 Night Stay in Emergency Shelter During Month, by Month After RA
SUB = permanent housing subsidy. UC = usual care.
RA = random assignment.
Notes: Percentages are weighted for survey nonresponse to represent all UC families in the study. Missing data on emergency shelter stays biases the percentages somewhat downward. The baseline stay in emergency shelter does not appear in the data for 18.7 percent of UC respondent families. The missing data rate for subsequent stays in emergency shelter is unknown.
Source: Family Options Study Program Usage Data
78 Outcomes regarding shelter stays are based on study Program Usage Data (which are described in Chapter 4 and Appendix A).
79 Although most of the families using emergency shelter in the fifth month after random assignment had not yet departed from shelter, a few had departed and returned.
The proportion of all study families in shelter who had departed and returned increases with time since random assignment. In the 13th month after random assignment, the proportion of SUB families in shelter who have departed and returned rises above one-half. It does so for the UC families in this comparison 1 month before.
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
0 2 4 6 8 10 12 14 16 18 20
Percent of families
Month after RA UC
SUB
The last three homelessness outcomes examined (see again Exhibit 6-4) measure the number of days in the past 6 months that a family was homeless or doubled up. Assignment to SUB reduced time spent homeless or doubled up by an aver age of 4 weeks in the past 6 months relative to UC. This difference is entirely accounted for by the difference in pro - portions of families who experienced homelessness or doubling up in the past 6 months (shown in the second row of the exhibit). The 16 percent of SUB families and 41 percent of UC families who experienced any homelessness or doubling up were in these conditions for a combined 18 weeks during the past 6 months.80
The housing independence outcomes in the next panel of Exhibit 6-4 measure whether a family lived in its own house or apartment at followup, either with or without housing assistance. The SUB intervention increased the proportion of families living in their own dwelling place from 58 to 73 percent relative to UC. This difference is the net result of two opposing effects. As would be expected, the proportion of SUB families living in their own places without housing assistance (10 percent) is much lower than the correspond-ing proportion for UC (33 percent). By contrast, and more than offsetting that decline, the proportion of SUB families living in their own places with housing assistance (63 percent) is much higher than the corresponding proportion for UC (25 percent).81
The stability offered by the SUB intervention also reduced the average number of places lived in during the past 6 months from 1.8 to 1.4.82 Because this outcome has a lower bound of 1, the UC mean of 1.8 compared with 1.4 for the SUB group means that the SUB intervention cut the number of moves during the final 6 months of the followup period in half.
The last two rows in Exhibit 6-4 show how the SUB inter-vention affects the nature of the housing occupied by study families at the point of followup survey by considering the number of persons per room (a measure of crowding) and residence in poor quality housing. Persons per room is a standard proxy for overcrowding and therefore for housing quality. The SUB intervention reduced the number of persons per room from 1.6 to 1.2. The SUB intervention similarly reduced the proportion of families living in units of poor or fair quality from 34 to 25 percent.
As SUB appears to be highly effective in preventing sub-sequent stays in shelter or places not meant for human habitation and being doubled up, the reader may wonder whether the incidence of these situations within the SUB group is entirely among those families who never used SUB.
Exhibit 6-6 shows a comparison of SUB families who never took up their assigned intervention with SUB families who did use SUB. The right column of the top panel shows that a small proportion (about 7 to 11 percent) of SUB families who used SUB nevertheless experienced homelessness or being doubled up during the period between 7 and 18 months after random assignment, presumably after the period of SUB use had ended.83 Whereas SUB is usually available to families only after a sometimes lengthy wait, it was available in this study on a priority basis to those families assigned to SUB. This priority access enabled SUB families to exit shelter faster than UC families. As shown in Exhibit 6-7, SUB fami-lies on average had shorter initial stays in emergency shelter than UC families (3.2 months for SUB compared with 3.7 months for UC).
In sum, the SUB intervention had a strong, positive effect on housing stability as compared with UC for every measure considered.
80 Dividing the average number of days spent homeless or doubled up in the past 6 months for SUB families by the percentage who experienced either state (20.4 days/0.160 = 127.5 days) reveals that those who did experience either state spent 128 days on average either homeless or doubled up in the past 6 months. Performing the same calcu-lation for UC families (51.5 days/0.409 = 125.9 days) reveals that UC families who experienced either state spent nearly the same amount of time (126 days) on average either homeless or doubled up in the past 6 months.
81 Although the survey response indicates that 63 percent of the SUB families were living in their own house or apartment with housing assistance at the time of the survey, the Program Usage Data show that the proportion of families using SUB, public housing, or project-based vouchers in the survey month is 77 percent. This discrepancy between response to the survey item and the Program Usage Data (largely based on HUD administrative records for these program types) suggests some measurement error in one or both of these data sources.
82 Although this outcome is not technically the same as the number of moves plus one (because it is possible for a family to move out of a place (for example, a housing unit shared with friends or relatives) and then move back into the same unit during the 6-month period), its interpretation is essentially the same.
83 Although the difference in homelessness outcomes between those who did and did not use SUB is consistent with a strong, negative impact on homelessness, these dif-ferences cannot be causally attributed to the use of SUB. Because the use of SUB was not randomly assigned, comparisons are fundamentally subject to selection bias (that is, people who did not take up SUB might have been more likely to become homeless in the absence of SUB).
Short-Term Impacts of Housing and Services Interventions for Homeless Families 70 Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)
Exhibit 6-6. SUB Versus UC: Percent of Families With at Least 1 Night Stay in Emergency Shelter During Month, by Month After RA
Outcome
Families Assigned to SUB Who Did Not Use
SUB
Families Assigned to SUB Who
Used SUB
N = 84 N = 446
Homeless or doubled up during the followup period
At least 1 night homeless or doubled up in past 6 months or in shelter in past 12 months) (%) 51.2 16.0†
At least 1 night homeless or doubled up in past 6 months (%) 44.0 11.5†
At least 1 night homeless in past 6 months (%) 33.3 7.0†
At least 1 night doubled up in past 6 months (%) 36.9 8.1†
Any stay in emergency shelter in months 7 to 18 after RA (%) 36.9 9.9†
Number of days homeless or doubled up in past 6 months 65.2 11.9†
Number of days homeless in past 6 months 37.1 6.1†
Number of days doubled up in past 6 months 40.6 6.6†
Housing independence
Living in own house or apartment at followup (%) 56.0 76.0†
Living in own house or apartment with no housing assistance (%) 42.9 3.1†
Living in own house or apartment with housing assistance (%) 13.1 72.9†
Number of places lived
Number of places lived in past 6 months 2.0 1.3†
Housing quality
Persons per room 1.6 1.1†
Housing quality is poor or fair (%) 28.6 24.4
Length of stay in shelter
Length (in months) of baseline stay in emergency sheltera 4.3 2.9†
SUB = permanent housing subsidy.
RA = random assignment.
† Difference in means is statistically significant at the .10 level.
a The length of baseline stay in emergency shelter outcome includes one-half of the month of random assignment and is topcoded at 18 months. The 21 percent of families assigned to SUB whose baseline shelter stay does not appear in the Program Usage Data are not included in the analysis.
Notes: Means are unweighted. See Chapter 5 and Appendix B for outcome definitions.
Sources: Family Options Study baseline survey; 18-month followup survey; Program Usage Data
Exhibit 6-7. SUB Versus UC: Impact on Length of Baseline Stay in Emergency Shelter
Outcome
SUB UC ITT Impact
Effect sizea
N Mean
(SD) N Mean
(SD) Impact
(SE)
Length (in months) of baseline stay in emergency shelterb 417 3.2 342 3.7 – 0.6** – 0.12
(3.1) (4.6) (0.2)
SUB = permanent housing subsidy. UC = usual care.
ITT = intention-to-treat. SD = standard deviation. SE = standard error.
*/**/*** Impact estimate is significantly different from 0 at the .10, .05, and .01 levels, respectively, using a two-tailed t-test.
a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.
b The length of baseline stay in emergency shelter outcome includes one-half of the month of random assignment and is topcoded at 18 months. The 20 percent of families in this comparison whose baseline shelter stay does not appear in the Program Usage Data are not included in the analysis.
Notes: Impact estimate and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Appendix B for outcome definition.
Sources: Family Options Study 18-month followup survey; Program Usage Data