Data Disaggregation

 

Data Disaggregation

Read our latest policy recommendations on AI/AN data disaggregation at the state level that were developed in collaboration with other civil rights organizations (2021). View here.

Read our literature review on AI/AN data disaggregation (2016). View here. 

Making the Case for Data Disaggregation to Advance a Culture of Health - a collection of reports and information funded by the Robert Wood Johnson Foundation (2015-2020). View here. 

The Asterisk Nation

While American Indians and Alaska Natives are an integral and unique part of US society, we continue to be invisible to most other Americans due to an absence of data, accurate media images, and historical and contemporary awareness about Native peoples in schools, healthcare facilities, professions, military service, and daily life.  

This invisibility is perpetuated by federal and state agencies and policies that leave American Indians and Alaska Natives out of data collection efforts, data reporting and analysis, and/or public media campaigns. American Indians and Alaska Natives may be described as the “Asterisk Nation” because an asterisk, instead of data point, is often used in data displays when reporting racial and ethnic data due to various data collection and reporting issues, such as small sample size, large margins of errors, or other issues related to the validity and statistical significance of data on American Indians and Alaska Natives.

There is a critical need for accurate, meaningful, and timely data collection in American Indian/Alaska Native (AI/AN) communities. Federal agencies are charged with collecting data in AI/AN communities, as well as from the general US population, in order to determine budget requests; support and strengthen budget justifications; allocate resources; provide services; conduct strategic planning; and comply with statutory and regulatory reporting processes.  Accurate data collection and community-based planning captures true needs, and thus can drive larger programmatic investments resulting in a cost-effective use of tribal, federal, and private resources. Without quality data, policymakers and community planners cannot set policy goals, monitor implementation, measure impact, or plan for demographic shifts in an effective way. The absence of American Indian and Alaska Native peoples in data and policy domains reflects the lack of a national public discourse on the status of our nation’s First Peoples.

In addition, the research and data on the importance of Native cultures and languages is taking place at a local, and often isolated, level such that national Indian policy research rarely accounts for or highlights the value of Native cultures and languages. A coordinated and comprehensive approach is needed for measuring and reporting how Native cultures and languages matter, especially in light of the rapidly changing demographics in the US and in Indian Country.

Because policymakers use national datasets (both government and private) to shape billions of dollars in funding allocations and develop policy interventions to serve American Indian and Alaska Native communities, it is vital for federal agencies and private entities to collect adequate data in AI/AN communities. Unfortunately, the data describing Native communities is often insufficient, unreliable or completely absent. The lack of data affects policymaking at federal, tribal, and state levels.

Key Considerations in Data Quality:

  1. What data related to American Indians and Alaska Natives are being collected by federal agencies?
  2. What is the quality of these data and measures (e.g., sample size, age of report)?
  3. What is the method of data collection (e.g., individual self-report on a survey, organizational records) and has tribal approval been granted?
  4. What is the ‘definition of Indian’ used?
  5. Are the comparisons used appropriate (e.g., Native to non-Native; Native to Native; regional; international comparisons)
  6. What measures are important to American Indians and Alaska Native leaders?

Priority Issues

Measurement of Small Populations
Emerging work on the issue of measurement of small populations is being developed with regional Census Information Centers, university researchers, and federal agency partners. This work aims to address the persistent issue of non-inclusion of small and rural populations like American Indian and Alaska Native peoples in national, longitudinal studies due to sampling challenges and high costs. To share insight on this issue, please contact research@ncai.org.

Undercounting of American Indian and Alaska Native Youth & Other Populations
Related to the issue of measurement in small populations noted above, there is a particular issue with undercounting of AI/AN people in major federal efforts such as the American Community Survey. These issues of undercount disproportionately affect those living on or near reservations and AI/AN youth, who make up a large proportion of AI/AN people nationally and in certain states. Analyses by Deweaver (linked below) suggest that these undercounts may be due in part to the smaller reach in the sampling approach used by the American Community Survey as compared to the broader sampling used by the Decennial Census. Other analyses by UCLA Researchers Ong & Ong suggests there may also be weighting issues in the sampling approach. These undercounts have serious impacts as American Community Survey data is used as part of the distribution of over $400 billion in federal and state funds each year, some of which the federal government has a trust responsibility to provide to tribal nations.

Trends in Size of AI/AN Alone Youth Population by Type of Land Area – 1990 to 2010 (Deweaver, 2013)

US Census Bureau Analysis of the American Community Survey and Puerto Rico Community Survey Coverage(January 16, 2013)

Comments on Census Bureau Evaluation Report on ACS Coverage Measured by Comparison with the 2010 Decennial Census (Deweaver, 2013)

American Community Survey Data on the American Indian/Alaska Native Population: A Look behind the Numbers (Deweaver, 2013)

The American Community Survey: Serious Implications for Indian Country (Deweaver, 2010)

International efforts in countries with significant populations of Indigenous peoples are emerging to strengthen methodologies related to improving estimates of Indigenous populations in national and other important data sets:

Improving Estimates of Indigenous Under-Identification in Key Data Sets


Measuring Joblessness

The Secretary of the Interior, in consultation with the Secretary of Labor, is statutorily required to publish, not less than biennially, a report that includes gender-specific information on the population eligible for services provided to Indian people by the Secretary of the Department of the Interior (DOI). The report is required to include, at a minimum, information (i) at the national level by state; (ii) at the Bureau of Indian Affairs Service area; and (iii) at the tribal level for the:

  • Total service population;
  • Service population under age 16 and over 64;
  • Population available for work, including those not considered to be actively seeking work;
  • Employed population, including those employed with annual earnings below the poverty line; and
  • Numbers employed in private sector positions and in public sector positions.

Enacted as Section 17 of Public Law 102-477 in October of 1992, as amended (codified at 25 U.S.C. 3416), the American Indian labor market report (“Report”) was mandated by the Indian Employment, Training and Related Services Demonstration Act of 1992 (“Act”). The Act allows Indian tribes to integrate federally-funded employment, training and related services programs provided by the Departments of the Interior, Labor, Education, and Health and Human Services. The last Labor Market Report issued by DOI was provided to Congress in 2007 for the year 2005.

Data from this Report are used to develop economic policy approaches to address the unique demographic and labor force contexts in tribal contexts that other Department of Labor (DOL) and US Census Bureau measures do not capture. Specifically, the Report’s measure of “joblessness” – or “the population available for work, including those not considered to be actively seeking work” – is not currently captured by other federal data collection efforts and is the most cited aspect of the Report. These data are used to inform the Congress’ policymaking, serving as the subject of a Senate Committee on Indian Affairs hearing during the 112th Congress and regularly used in hearings by the House and Senate. They are also used locally for planning and program purposes to identify appropriate economic development approaches and gauge particular community needs and resources. Another critical use of these data is to determine levels of federal funding for tribes under the Workforce Investment Act, the Indian Housing Block Grant program, and the BIA Tribal Transportation program.

Any significant changes to data collection and the continued non-reporting of data impacts the ability of tribal governments to adequately provide for their citizens, and affects the federal government from carrying out its trust responsibility in essential social and economic areas. While DOI has traditionally relied on tribes to provide data for this report, tribes should not bear sole or primary responsibility for providing quality data with little to no resources, training, or other support from DOI to do so. As stated in statute, this Report is the responsibility of the Department of the Interior. It is also an essential mechanism for monitoring the quality of services that DOI is responsible to provide to American Indian and Alaska Native people. DOI's 2012 request for comments on the AIPLF Report includes questions about the possibility of using Census data on unemployment rather than tribal data on joblessness in order to improve data quality and consistency, especially given the Office of Management and Budget's data quality standards. At times, it appears that tribes are being held responsible for a lack of federal agency coordination around the issue of data quality and the measurement of small populations. Specifically, there needs to be greater coordination between DOI, DOL, Census, and OMB to address the widespread problems that plague data collection for Indian Country.

[The above section was excerpted from the official Comments NCAI filed in 2012 that can be viewed here.]

Other information from the Bureau of Indian Affairs at the US Department of the Interior on the American Indian Population and Labor Force Report can be found at: http://www.bia.gov/cs/groups/public/documents/text/idc-024548.pdf.
 

US Department of Education (USDOE) Non-Reporting of Key Education Data on American Indian and Alaska Native Students.

When the USDOE released its 2007 Final Guidance on Maintaining, Collecting and Reporting Racial and Ethnic Data,the impacts across Indian Country were felt almost immediately. While the USDOE will continue to collect data on American Indian and Alaska Native (AI/AN) students whether or not they report a racial/ethnic status that is in combination with other racial/ethnic groups (e.g., Hispanic/Latino, White), the USDOE will only report AI/AN specific data for students who indicate they are not Hispanic/Latino ethnically and select only American Indian and Alaska Native as their race. American Indian/Alaska Native students who indicate that they are also Hispanic/Latino ethnically will only be reported in the Hispanic/Latino category. Regardless of whether they indicate Hispanic/Latino ethnicity, American Indian/Alaska Native students selecting an additional racial category will only be reported as multiracial. The effect is major and detrimental at local, state, and national levels as AI/AN communities have historically relied on USDOE data as a quality source of information for planning and development efforts (see image below from the 2012 NCES STATS-DC Presentation prepared by NCAI and NIEA):

The USDOE has data that it can disaggregate for AI/AN alone, AI/AN in combination with other ethnicities and races, and AI/AN alone and in combination as the Census does, but it has opted not to do so. Stemming from the Executive Order of May 9, 2013, Making Open and Machine Readable the New Default for Government Information, President Obama established an Open Data Policy through Memorandum that:

“establishes a framework to help institutionalize the principles of effective information management at each stage of the information's life cycle to promote interoperability and openness…Specifically, this Memorandum requires agencies to collect or create information in a way that supports downstream information processing and dissemination activities.”

The USDOE Guidance does not support downstream information processing or dissemination activities in American Indian and Alaska Native contexts.

2012 NCES STATS-DC PowerPoint Presentation

Gaps in Existing Data

Another major data quality issue has to do with gaps in existing data, which can relate to non-existence of data on key indicators or the inconsistent reporting over time that leads to gaps in data. For example, the American Indians and Crime Series published by the Bureau of Justice Statistics at the US Department of Justice provided essential data between 1999 and 2004 but has not been published since. Also, an analysis by Richard Todd, the Vice President of the Minneapolis Federal Reserve Bank, identified a range of gaps in economic data with relevance to tribal and reservation communities. Todd notes that there is a particular need for longitudinal data on tribal government economic development, reservation business activity, and data on individual reservation residents and households. A 2007 report by Westat details the gaps in data related to American Indian and Alaska Native health data and offers strategies for improvement.

Lack of Coordination and Data Linkage across Sectors
Emerging research on Native youth suggests that we need data that can be compared across sectors like education, health, labor, and justice, for example. Knowing the high school dropout rate for AI/AN students may tell a particular story in a region, but having information about where students who leave school go (e.g., workforce, military, justice system) could assist community leaders in planning robust and innovative initiatives to support these students’ development over time. There needs to be improved efforts to coordinate federal and state data across agencies and sectors. Other countries like Australia have invested in developing guidelines for linking data sets that include information on Aboriginal and Torres Strait Islander peoples to improve data quality:

Guidelines for Data Linkage Activities Relating to Aboriginal and Torres Strait Islander People


Definition of Indian
While NCAI does not advocate for any one particular definition in federal policy, the NCAI Policy Research Center has begun to compile information on the various definitions of Indian in use by federal agencies to inform policy development.

While members of federally recognized tribes have a particular status here in the US, this issue of Indigenous identification is one that plagues other nations as well. Australia has developed some guidelines to steward the identification of Aboriginal and Torres Strait Island peoples in health data sets:

Improving Indigenous Identification in Health Data Sets

Content moved here from Data Resources Webpage on 6/17/2021:

Data Disaggregation

Over the past several years, the NCAI Policy Research Center has sought to share with tribal leaders, planners, program administrators, and policymakers various approaches to interpreting and employing federal data on American Indian and Alaska Native youth and families. In disaggregating data to a state level, we feel that we can advance a deeper understanding of about the realities facing Native communities, as well as the systemic levers that might be used to generate better outcomes, greater resources. Our purpose and methodology include the following dimensions:

Advancing a New Narrative. Discussions of equity are often overtaken by a focus on disparity, which can render invisible community strengths and obscure potential solutions. We aim to advance a new narrative by presenting existing data as part of four lenses to promote new understandings of Native youth and families that extend beyond a poverty lens; by promoting Native-to-Native comparisons alongside disparity data; and by disaggregating national data to the state level to take place into account. By doing so, we hope to provide tools to data users and consumers to see new patterns and possibilities when it comes to American Indian and Alaska Native youth and families.

Promoting Increased Use of AIAN Data. American Indian and Alaska Native youth are often invisible in federal data sets, due in part to the small size of the overall population of Native peoples in the US. However, the extent to which they are represented in data sets reflects our commitment to and ability to understand and address their unique needs. And where we find fairly good data at a national level, there is an ever-present need to access data at state, county, and tribal levels to develop relevant policy and program supports. Tribal and community planners need better data to harness local resources to areas of greatest need and to hold state and federal governments accountable to meet the needs of our nation’s first peoples. Our hope is that these profiles will represent a new way to tell the story of Native youth and families, will increase community users’ awareness and use of existing data, and will promote comparisons and analyses that lead to better outcomes for Native youth and families.

Sources of Data. For this analysis, we sourced publically available federal data that was both disaggregated for American Indians and Alaska Natives and by state. Additional state data was sourced selectively where consistent federal data was unavailable, but used sparingly as indicators are not always the same across states. The analysis builds on earlier work presented in NCAI’s Regional Data Profiles. When multiple years of data are available, the most recent data is presented. When referring to AIAN data, this includes data for American Indian and Alaska Native alone, except where indicated that it is in combination with other races. 

Exploring the Data through a Thematic View and State View. There are two frames through which we explore the data – a thematic view that looks across four states to understand economic, education, health, and justice system outcomes for Native youth and families and a state view that looks within a state to understand how a particular state enables or constrains Native prosperity.

Using Comparisons to Gauge Strength & Disparity. It is often quite difficult to identify even a common percentage for American Indian and Alaska Native youth at a state level because of issues with data collection, quality, and disaggregation. Yet, if we can access these data, we must then ask how we know whether a percentage is too high, too low, or what we would expect. To do so, we must identify useful points of comparison.

The four thematic areas we have selected  as analytic lenses are:  Access to Opportunity; Child Health & Wellness; Family Economic Capability; and System Involvement. Over the next several months, we plan to release several products that highlight this effort surrounding data disaggregation. Below, please find one-pager synopses of Native youth system involvement for the four states of Alaska, Montana, New Mexico, and Oklahoma:

Profile of Alaska: System Involvement
Profile of Montana: System Involvement
Profile of New Mexico: System Involvement
Profile of Oklahoma: System Involvement

Frequently Used Data (FUDs)

The NCAI Policy Research Center is working with our NCAI Legislative Policy team to identify their most common data needs and produce data synthesis that will drive NCAI’s policy work. As part of this work, we are compiling a list of the major databases used to host data on American Indian and Alaska Native peoples by the federal government. Our current work is summarized in the Demographics Profile we issue annually, and in some instances has been tailored for particular issue areas as was done in this Snapshot of Indian Education. This work is also being informed by work by other organizations to develop indicators and indices to monitor the status of Native and other groups. See, for example:

Data Displays & Mapping

While many data reports typically come in narrative form, increasingly we see the value and reach of infographics that use data to tell a visual story. Sometimes, these infographics involve mapping information onto land maps to show regional impacts or trends. The NCAI Policy Research Center staff is works to develop visual and map-based images for use in all of our products and online dissemination to present data in various ways. Some of the resources we draw on include: