Introduction
Like other homelessness assistance programs, HPRP grantees were required to collect data and enter it into HUD’s Homeless Management Information System (HMIS). This chapter explores HMIS and other data collection. The chapter answers the following research question (from Exhibit 1.1, highlighted in the diagram below):
• What types of data did communities collect for HPRP?
The chapter starts with national highlights from the HPS survey to paint in broad strokes the picture of HPRP data entry nationally. It then provides detailed findings on the processes and procedures HPRP
grantees/subgrantees created to allow program staff to enter client-level data; how HPRP programs have used HMIS data so far; and what challenges grantees and subgrantees encountered with this program requirement.
National Highlights—HMIS Data Collected for HPRP
42• A little more than two-thirds of grantees (65 percent) and subgrantees (68 percent) entered information from the screening process into a database.
• An overwhelming majority of subgrantees (84 percent) entered information from their assessments into a database.
• About one-third of grantees and subgrantees entered data collected during the screening and assessment process on those whom they screened out.
• About half of grantees (52 percent) and subgrantees (47 percent) used program data to track household outcomes after HPRP program exit.
• About 71 percent of grantees and 64 percent of subgrantees collected data to understand how much HPRP prevention programs cost.
42 Weighted Homelessness Study survey results, October through December 2011; HPS, Grantees and Subgrantees.
What types of data did communities collect for HPRP?
How did they use the data they collected, what did they find?
HUD-required data elements for HPRP Screening/assessment information
Case plan/progress Other information required locally Immediate and/or longer-term impacts of HPRP assistance
HPRP expenditures
Data uses: Program design, monitoring, modifications, outcomes tracking, future plans
Exhibit 7.1 Research Questions
Community Level Decisions, Design,
Structures Program Level Implementation, Program Activities
Research Questions
The Data That HPRP Programs Collected
As a condition of receiving HPRP funding, grantees and subgrantees had to report to HUD on persons served with HPRP assistance and the use of HPRP funds in the CoC’s HMIS43 or a comparable database (for victim service providers and some legal services providers). HMIS is an electronic data collection system that stores information on persons who use homeless services provided in a community. Data collected in HMIS include client demographics, health and disability information, housing status prior to program entry, income and benefits, services received during program participation, and similar information at program exit, including housing status. HUD required specific HMIS data elements for HPRP.
Additionally, grantees submitted Quarterly Performance Reports (QPRs) and an Annual Performance Report (APR) to HUD each grant year. These reports show aggregate client-level data on persons and households served by the program during the reporting and grant period. HUD instructed grantees to use HMIS or a comparable database to generate the data for these reports. While many HPRP grantees and subgrantees contributed data to their communities’ HMIS systems prior to HPRP, for some programs this data collection and data entry requirement was new.
The requirements listed above encouraged grantees and subgrantees to focus on monitoring data quality and data completeness, and also think about ways to use their program- and client-level data to manage their programs and monitor outcomes. In addition to reporting HMIS data to HUD, grantees that entered the relevant data can use HMIS data to track shelter entry rates, compare eligible and non-eligible program applicants, and monitor housing outcomes of program participants. Some grantees used HMIS or other mechanisms to monitor program outcomes, although HUD did not require such post-program tracking.
HUD required HPRP grantees and subgrantees to collect the following elements on households that receive assistance from HPRP funds and enter them into HMIS:
• Current name • Length of stay in previous place
• Social Security Number • Residence prior to program entry
• Date of birth • Housing status
• Race • Income and sources
• Ethnicity • Non-cash benefits
• Gender • Program exit date
• Veteran status • Destination at program exit
• Disabling condition • HPRP financial assistance provided
• ZIP Code of last permanent address
• Program entry date • HPRP housing relocation and stabilization services provided
The majority of grantees and subgrantees in communities visited also used HMIS to record and store supplemental information on households’ housing and financial situations (see Exhibit 7.1).
43 According to HUD’s HPRP Data Collection and Reporting Guidance, “A Homeless Management Information System (HMIS) is a client-level data collection and management system implemented at the community level that allows for better coordination among agencies providing services to clients. It is not a national reporting system and it is not designed to be a financial reporting/accounting system. Agencies providing HPRP assistance must enter client-level data into their community’s HMIS so the community can provide aggregate data to grantees.”
Supplemental information could include housing histories and assessments; stability plans; assessment information, scores, and ratings; case plans and goals; eviction notices and leases; household outcomes;
and reassessment screenings. For example, in Pima County/City of Tucson, Arizona, subgrantee staff uploaded lease agreements and eviction notices to HMIS. By having these documents on the HMIS system, staff at legal aid could review lease agreements and eviction notices for inconsistencies and errors before issuing payments to landlords for an HPRP household’s rental arrears and rent assistance.
Exhibit 7.1: How the Communities Visited for This Evaluation Use HMIS Communities Visited
(states at bottom) Supplemental Information Entered Into
HMIS How the Site Uses HMIS Data
Albuquerque, NM • Data on public benefits that clients receive
• Number of children in the household
• Employment status
• To track housing stability
• To determine average length of time required for a client to find employment
Arlington County, VA • Housing stability at follow-up (3, 6,
and 12 months after program exit) • To examine client characteristics
• To monitor shelter reentry
• To develop a vulnerability index from client risk factors
Dayton/Montgomery
County, OH • Assessment
• Case management notes • To examine patterns of homelessness, shelter reentry, previous episodes of homelessness, and primary homelessness risk factors
Fall River, MA • Only collected HUD-required data
elements • To verify grantee expenditure reports
Jefferson County, AL • Case management plans • To examine client outcomes Kalamazoo, MI • Program cost data
• Data on residence at exit, work status, changes in income
• To examine client outcomes
• To present to potential funders for the umbrella eviction diversion program
Lancaster City and
County, PA • Homeless risk factors from screening
• Landlord payments
• Housing stability plans
• To monitor shelter reentry
• To improve program performance, including reducing shelter use and helping clients achieve permanent affordable housing
Miami-Dade County,
FL • Data on client reassessments and
legal expenses • To monitor shelter reentry Pasco County, FL • Only collected HUD-required data
elements • To monitor shelter reentry
Philadelphia, PA • All intake and screening data
• Full assessment data
• Client exit data
• To improve program targeting
• To assess why HPRP applicants become homeless or face the threat of homelessness
• To examine services received by clients
• To monitor financial data and client outcomes Pima County/City of
Tucson, AZ • Case management notes
• Housing plans
• Eviction notices
• Lease agreements
• For local evaluation efforts
• To examine client characteristics and client outcomes
• To monitor shelter reentry Santa Clara County,
CA • Prescreening information
• Modified Arizona Self-Sufficiency Matrix scores
• Case management and housing stability plans
• Information on shelter entry
• To examine client outcomes and client characteristics
• To monitor program and client progress
• To look at housing stability
Indiana • Housing assessments • For data quality report cards
• To examine client outcomes, including destination at exit, change of income from program entry to program exit, and length of stay
Exhibit 7.1: How the Communities Visited for This Evaluation Use HMIS Maine • Screening data for enrollees
• Case management services
• Information on housing search, outreach, legal assistance, and credit reports
• Housing stability plans
• For data quality reports
• For data sharing and tracking across state jurisdictions
• To monitor shelter reentry
Massachusetts • Nothing beyond HUD-required data
elements • One local program monitored shelter reentry
North Carolina • Varying degrees of screening and assessment data, depending on subgrantee
• To monitor housing barrier levels served by programs
• To examine the population that programs serve
• To report on the program’s progress
• To examine client outcomes and housing stability rates at program exit
Rhode Island • Modified Arizona Self-Sufficiency
Matrix scores • To assist in the design of the Emergency Solutions Grant (ESG) program
• To examine client characteristics, length of stay, program exit data, client income levels, and residence prior to program entry
• To monitor shelter reentry Source: HPS site visits
Santa Clara County, California, HPRP service providers entered Self-Sufficiency Matrix (SSM) scores from their assessment into HMIS for households participating in the program, in addition to HUD’s required information. Staff tracked and monitored household outcomes via the SSM and compared SSM scores at program exit to those at program entry to determine whether the program was meeting its goals.
Program staff developed several HPRP goals, including that 85 percent of households receiving
assistance would remain stably housed and 75 percent of households that complete the program would improve their SSM scores. Chapter 8 examines in greater depth Santa Clara’s efforts to use data to assess program outcomes.
In Philadelphia, Pennsylvania, HPRP program staff did intakes directly into the citywide HMIS before making an eligibility decision. This meant that the grantee had intake information on those denied assistance as well as on those who enrolled, giving it the ability to compare households that requested assistance but did not receive it to households that did receive assistance. Philadelphia was also able to track all HPRP applicants through HMIS to see if they entered shelter, and to compare shelter entry among those who received assistance to shelter use among those that HPRP rejected to see if HPRP made a difference to rates of shelter entry. Philadelphia’s efforts to evaluate program impacts are discussed in greater detail in Chapter 8.
Entering Data on Homelessness Prevention
HPRP grantees and subgrantees used varied procedures to enter client- and program-level data into HMIS systems, taking into account agency and staff capacity, data quality and data entry timeliness, and the cost for user licenses and system maintenance.
While HPRP grantees were responsible for reporting HMIS data to HUD, grantee staff did not typically enter the data themselves. In most HMIS implementations, subgrantee or service provider staff entered
and maintained client-level data in the HMIS system because they worked directly with households and had the easiest access and most up-to-date information on clients. Service provider staff included case managers, intake workers, client specialists, and any other staff who provided services to households.
This data entry model also allowed service provider staff to enter or upload households’ service referrals, case plans, and program goals to HMIS as they were created.
In 15 of the 17 HPRP homelessness prevention communities that the study team visited, service provider staff entered client-level data into HMIS. However, some HPRP programs preferred not to use this data entry approach because of the cost and burden it places on service provider staff. For example, in Miami/Dade County, Florida, the lead agency for the countywide HPRP homelessness prevention program decided to centralize data collection and reporting to avoid buying HMIS user licenses and providing training to all service providers. Additionally, by creating a centralized data entry and reporting team, the program reduced the risk of data entry errors and duplication across providers.
In Kalamazoo, Michigan, Housing Resources Inc. (HRI), the HPRP service provider, performed all HMIS data entry for the Eviction Diversion/HPRP Homelessness Prevention program. Department of Human Services (DHS) caseworkers sent completed paper-based household assessments to HRI for data entry.
This approach allowed DHS caseworkers to focus on working with households to resolve their housing crises while assuring consistent data entry through HRI staff.
Some HPRP grantees and subgrantees also used their HMIS system as a way to communicate among service agencies. For example, in Pima County/City of Tucson, Arizona, staff at each subgrantee entered client-level data, case plans, leases, eviction notices, and eligibility determinations into HMIS, where other subgrantees could see them. Thus HMIS served as a common tool for data sharing and program collaboration across subgrantees. Maine designed the HMIS module for its statewide HPRP program to allow data sharing throughout the state, permitting program staff to track households across jurisdictions.
In Jefferson County, Alabama, household intake, screening, assessment, and eligibility determination were conducted by staff from both the grantee and subgrantees, with client-level data flowing among the parties. In this case, HMIS provided a data exchange mechanism for the grantee and subgrantees.
When Data Entry for Homelessness Prevention Occurred
HPRP grantees and subgrantees entered household information into HMIS at various points during the screening and eligibility determination process. The Recovery Act statute required the use of HMIS, and HUD clarified that this meant grantees and subgrantees were required to enter client-level data on all households served with HPRP funds. Required client-level data included client demographics, information on prior living situation, program entry date, income and non-cash benefits, date and destination of program exit, and HPRP financial assistance and housing relocation and stabilization services provided.
Other information used for screening and intake could be entered but was not required. Grantees could decide when a household’s information would first be entered into HMIS as well as whether information was entered only for households admitted to the program or also for those screened out.
HPS survey results for HPRP grantees, reported in Exhibit 7.2, indicate that 65 percent of grantees entered information from screening on households screened in, but only 33 percent entered information from
screening on those screened out. HPS survey results for subgrantees and direct service providers showed that these agencies were more likely to enter assessment data than grantees, as they were more likely to work directly with clients and need the assessment data for case planning and follow-through. More than 80 percent of subgrantees and service providers entered information from assessments for households served. This is a fairly high percentage given that subgrantees and direct service providers were not required to enter information on assessments for households being served into a data system. Approximately 9 percent of subgrantees and 7 percent of direct service providers indicated that they did not enter information on assessments for households being served. Additionally, 6 percent of subgrantees and 7 percent of direct service providers stated that they were not sure if they enter information on assessments for households being served.
Only about one-third of HPRP agencies in any category entered data on households screened out of the program. While this information would be useful for programs to have, HUD did not require it and most HPRP communities chose not to enter it.
Some grantees entered all household information into HMIS, including initial intake and screening, the eligibility decision, and everything that happened up to program exit. For example, in Philadelphia, Pennsylvania, the grantee, Office of Supportive Housing (OSH), instructed staff to enter all household information collected from initial screening to full assessment to program closeout into HMIS. As a result, OSH has been able to compare the characteristics of applicants who requested homelessness prevention assistance but did not receive it with those applicants who did receive the assistance.
Moreover, OSH can examine the rate of shelter entry for applicants who did receive homelessness prevention assistance to that of applicants who did not receive the assistance, controlling for individual characteristics. Information about the characteristics of people who do and do not receive services, in combination with data on homelessness, can be used to develop models to target services, as discussed in Chapter 1 (see also Shinn et al. 2013) and in Chapter 10.
Despite the evaluation advantages of having data on HPRP applicants who did not receive help from HPRP, most HPRP grantees only entered household information into HMIS once households were accepted for and enrolled in HPRP services. Indiana’s HPRP program, for instance, entered clients into HMIS after they were screened and determined eligible for HPRP homelessness prevention assistance.
Subgrantees in Indiana used an electronic, Web-based assessment tool to screen households for HPRP eligibility. However, the screening tool and Indiana’s HMIS system do not interface with each other, so screening data on households screened out of HPRP were not entered into HMIS.
Exhibit 7.2: HPRP Screening and Assessment Data
Data Entered Into HMIS Grantees Subgrantees Direct Service
Providers Information from screening (on households screened in) 65% 68% 69%
Information from assessments (on households being
served) 67% 84% 85%
Information from screening (on households screened out) 33% 36% 37%
Information from assessments (on households not being
served) 31% 31% 33%
Source: Analysis of HPS survey data
Finally, some grantees left the decision as to when households are entered into HMIS up to subgrantees. In North Carolina, subgrantees of the state’s HPRP program entered information on households at various points after the prescreening process. Some subgrantees entered information into HMIS on households that passed the initial prescreen, even though those households might not ultimately receive help from HPRP. By contrast, other North Carolina subgrantees waited until final eligibility determinations had been made to enter data, and entered only information on eligible households into HMIS.
How HPRP Programs Used HMIS
HPRP grantees and subgrantees used HMIS data in a variety of ways. All HPRP grantees used HMIS data to populate the required Quarterly Performance Reports (QPRs) and Annual Performance Reports (APRs) submitted to HUD during the HPRP grant period. Additionally, some grantees used HMIS data to monitor subgrantees, make program adjustments, and track reentry or repeat requests for assistance and household outcomes.
Key informants interviewed during visits to Indiana, Lancaster City and County, Pennsylvania, Rhode Island, and North Carolina said that they used HMIS data to monitor their HPRP subgrantees, specifically
examining data quality, data completeness, and data entry timeliness. In Indiana, the state’s HPRP grantee generated data quality report cards for each subgrantee to assess HMIS data quality on collected data elements. The grantee found this encouraged subgrantees to enter data completely and correctly.
HPS survey results from grantees and subgrantees indicate that they used program data to track household outcomes after program exit, as shown in Exhibit 7.3. However, 23 percent of grantees were not sure if they were doing this.
Approximately 70 percent of surveyed grantees reported that they collected data to understand how much their HPRP homelessness prevention programs cost (Exhibit 7.4). Some 12 percent of grantees reported that they did not use their data to understand program cost, and 15 percent of grantees were not sure. Some grantees who reported “yes” to this survey question may have used their data to evaluate HPRP homelessness prevention program costs relative to other homelessness interventions in their community. However, given the general wording of the survey question, the research team suspects that many grantees reported “yes” because they simply used their HPRP program cost data to document how much money they spent on homelessness prevention services over the HPRP grant term,
Approximately 70 percent of surveyed grantees reported that they collected data to understand how much their HPRP homelessness prevention programs cost (Exhibit 7.4). Some 12 percent of grantees reported that they did not use their data to understand program cost, and 15 percent of grantees were not sure. Some grantees who reported “yes” to this survey question may have used their data to evaluate HPRP homelessness prevention program costs relative to other homelessness interventions in their community. However, given the general wording of the survey question, the research team suspects that many grantees reported “yes” because they simply used their HPRP program cost data to document how much money they spent on homelessness prevention services over the HPRP grant term,