education-data-source-edfacts

EDFacts Data Source Reference

EDFacts — federal K-12 outcome data from State Education Agencies, covering state assessment proficiency rates, ACGR graduation rates, and ESSA accountability indicators at school and district level (assessments 2009-2020, graduation rates 2010-2019). Use when analyzing within-state achievement trends, subgroup proficiency gaps, or adjusted cohort graduation rates. Complements CCD (school characteristics) with outcome data. State assessment scores CANNOT be compared across states; use NAEP for cross-state comparisons.

EDFacts is the U.S. Department of Education's centralized data collection system for pre-K through grade 12 education data from State Education Agencies (SEAs). It provides state assessment proficiency rates, graduation rates, and accountability indicators — the authoritative federal source for state-level K-12 outcome data.

> CRITICAL: Value Encoding

>

> The Urban Institute Education Data Portal converts NCES string codes (e.g., ALL, CWD, LEP) to integer codes. Always verify actual data values before filtering — do not rely on documentation labels alone.

>

> | Context | Subgroup "All" | English Learner | Sex "Male" |

> |---------|----------------|-----------------|------------|

> | Portal integer | 99 | 1 | 1 |

> | NCES string | ALL | LEP | M |

>

> See ./references/variable-definitions.md for complete encoding tables.

What is EDFacts?

• Collector: U.S. Department of Education, via State Education Agencies (SEAs)
• Coverage: All public schools and districts in 50 states + DC
• Content: State assessment proficiency rates, ACGR graduation rates, participation rates, accountability indicators
• Frequency: Annual collection
• Available years: Assessments 2009-10 to present; Graduation rates 2010-11 to present
• Primary identifiers: ncessch (school ID, Int64), leaid (district ID, Int64), fips (state FIPS code, Int64)
• Key limitation: State assessment scores CANNOT be compared across states (different tests, different cut scores)
• Available through: Education Data Portal mirrors

Reference File Structure

| File | Purpose | When to Read |

|------|---------|--------------|

| accountability-context.md | ESSA, NCLB history, accountability systems | Understanding policy context |

| assessment-data.md | Proficiency levels, test scores, limitations | Working with assessment data |

| graduation-rates.md | ACGR methodology, cohort definitions | Analyzing graduation data |

| variable-definitions.md | Key variables, suppression codes, special values | Interpreting specific variables |

| data-quality.md | Known issues, state variations, COVID impacts | Data cleaning, limitations |

| subgroup-reporting.md | Special populations, disaggregation | Analyzing by student groups |

Decision Trees

What type of analysis?

What EDFacts data do you need?
├─ Assessment/proficiency data
│   ├─ Within-state trends → Valid analysis
│   ├─ Cross-state comparison → INVALID - use NAEP instead
│   └─ Subgroup gaps → See ./references/subgroup-reporting.md
├─ Graduation rates (ACGR)
│   ├─ Understand methodology → See ./references/graduation-rates.md
│   ├─ Extended rates (5-year, 6-year) → See ./references/graduation-rates.md
│   └─ Subgroup rates → See ./references/subgroup-reporting.md
├─ Understanding variables
│   ├─ Missing/suppressed values → See ./references/variable-definitions.md
│   ├─ Range vs. exact values → See ./references/variable-definitions.md
│   └─ Subgroup codes → See ./references/subgroup-reporting.md
└─ Data quality concerns
    ├─ COVID-19 impacts (2019-20) → See ./references/data-quality.md
    ├─ State reporting changes → See ./references/data-quality.md
    └─ Suppression rates → See ./references/data-quality.md

Is my comparison valid?

What are you comparing?
├─ Same state, different years
│   ├─ Same assessment system? → Valid
│   └─ Different tests? → Break in time series
├─ Schools within same state → Valid
├─ Districts within same state → Valid
├─ Subgroups within same school → Valid (check suppression)
├─ Different states
│   ├─ Proficiency rates → INVALID
│   ├─ Graduation rates (ACGR) → More comparable
│   └─ Use NAEP instead → Valid
└─ National ranking by proficiency → INVALID

Quick Reference: EDFacts Data Elements

Assessment Data

| Data Element | Description | Available Years |

|--------------|-------------|-----------------|

| Proficiency rates | % meeting state standards in reading/math | 2009-10 to present |

| Participation rates | % of students assessed | 2012-13 to present |

| Achievement levels | Below Basic, Basic, Proficient, Advanced | Varies by state |

| Grade levels | Grades 3-8, high school (varies) | 2009-10 to present |

Graduation Data

| Data Element | Description | Available Years |

|--------------|-------------|-----------------|

| 4-year ACGR | Adjusted Cohort Graduation Rate | 2010-11 to present |

| 5-year ACGR | Extended graduation rate | 2011-12 to present |

| 6-year ACGR | Further extended rate | 2012-13 to present |

| Diploma types | Regular diploma only in ACGR | All years |

Key Identifiers

> Portal Data Types: All identifiers are Int64 in the Portal parquet files. The NCES source format (zero-padded strings) is shown for reference only. When joining with other Portal datasets, join on the integer columns directly.

| ID | Portal Type | NCES Source Format | Level | Example (Int64) |

|----|-------------|-------------------|-------|-----------------|

| ncessch | Int64 | 12-char zero-padded | School | 10000500870 |

| ncessch_num | Int64 | Same as ncessch | School | 10000500870 |

| leaid | Int64 | 7-char zero-padded | District/LEA | 100005 |

| leaid_num | Int64 | Same as leaid | District/LEA | 100005 |

| fips | Int64 | 2-digit | State | 1 (Alabama) |

Data Levels

| Level | Identifier | Dataset Path Pattern |

|-------|------------|---------------------|

| School | ncessch (Int64) | edfacts/schools_edfacts_* |

| District/LEA | leaid (Int64) | edfacts/districts_edfacts_* |

| State | fips (Int64) | Aggregate from lower levels |

Subgroups Reported

> Note: Not all subgroup columns are present in every dataset. Grad rates data does NOT have sex, migrant, or military_connected columns.

| Subgroup | NCES Code | Portal Integer | Column | Available In |

|----------|-----------|----------------|--------|--------------|

| All students | ALL | 99 | race, sex, lep, disability | Assessments, Grad Rates |

| Economically disadvantaged | ECODIS | 1 | econ_disadvantaged | Assessments, Grad Rates |

| Students with disabilities | CWD | 1 | disability | Assessments, Grad Rates |

| English learners | LEP | 1 | lep | Assessments, Grad Rates |

| Homeless | HOM | 1 | homeless | Assessments, Grad Rates |

| Foster care | FCS | 1 | foster_care | Assessments, Grad Rates |

| Migrant | MIG | 1 | migrant | Assessments only |

| Military connected | MIL | 1 | military_connected | Assessments only |

| Race/ethnicity | Multiple | 1-7, 99 | race | Assessments, Grad Rates |

| Sex | M/F | 1, 2, 99 | sex | Assessments only |

EDFacts Filter Column Pattern:

• Special population columns (lep, disability, homeless, etc.) use 1 = subgroup, 99 = total
• Race column uses integer codes (1=White, 2=Black, etc.)
• Sex column uses 1 = Male, 2 = Female, 99 = Total (assessments only)

Grade Codes (grade_edfacts)

| Code | Grade Level |

|------|-------------|

| 3-8 | Grades 3-8 (individual) |

| 9 | Grades 9-12 combined |

| 99 | Total (all grades) |

Race Codes

> Empirically verified from 2018 school assessment data. Only these values appear in the race column:

| Code | Category |

|------|----------|

| 1 | White |

| 2 | Black |

| 3 | Hispanic |

| 4 | Asian |

| 5 | American Indian/Alaska Native |

| 7 | Two or More Races |

| 99 | Total |

> Note: Code 6 (Native Hawaiian/Pacific Islander) is NOT observed in the data. Codes 8 (Nonresident alien), 9 (Unknown), 20 (Other), -1, -2, -3 are also not observed in the race column. These codes may exist in other Portal sources but are absent from EDFacts.

Sex Codes

| Code | Category |

|------|----------|

| 1 | Male |

| 2 | Female |

| 9 | Unknown |

| 99 | Total |

Disability Codes

> Empirically verified from 2018 school assessment and 2019 grad rate data. Only 1 and 99 are observed in the disability column. The expanded codes (0-4) documented in other Portal sources are NOT present in EDFacts datasets.

| Code | Category |

|------|----------|

| 1 | Students with disabilities (IDEA-eligible) |

| 99 | Total (all students) |

LEP Codes

| Code | Category |

|------|----------|

| 1 | Students who are limited English proficient |

| 99 | All students (total) |

Special Population Columns

For homeless, migrant, econ_disadvantaged, foster_care, military_connected:

| Code | Category |

|------|----------|

| 1 | Yes (in subgroup) |

| 99 | Total (all students) |

Missing Data Codes

| Code | Meaning | When Used |

|------|---------|-----------|

| -1 | Missing/not applicable | Data not reported |

| -2 | Not reported | Item doesn't apply to this entity |

| -3 | Suppressed for privacy | Data suppressed for small N-size |

| -9 | Rounds to zero | Value rounds to zero |

| Range values | Exact value suppressed | Range provided instead of exact value |

| _midpt suffix | Calculated midpoint of suppressed range | Use for analysis when exact values are suppressed |

Always use _midpt variables for analysis when exact values are suppressed.

Data Access

All EDFacts data is fetched via the Education Data Portal mirror system. There is no API access.

Key references:

• mirrors.yaml -- Mirror definitions, URL templates, read strategies
• datasets-reference.md -- Canonical dataset paths (one path works for all mirrors)
• fetch-patterns.md -- fetch_from_mirrors() and fetch_yearly_from_mirrors() patterns

> Truth Hierarchy: When interpreting variable values, apply this priority:

> 1. Actual data file (what you observe in the parquet/CSV) — this IS the truth

> 2. Live codebook (.xls in mirror) — authoritative documentation, may lag

> 3. This skill documentation — convenient summary, may drift from codebook

>

> If this documentation contradicts the codebook, trust the codebook. If the codebook contradicts observed data, trust the data and investigate.

Key Datasets

| Dataset | Path | Type | Columns |

|---------|------|------|---------|

| School Assessments | edfacts/schools_edfacts_assessments_{year} | Yearly (2009-2018, 2020) | 26 cols |

| School Grad Rates | edfacts/schools_edfacts_grad_rates_{year} | Yearly (2010-2019) | 18 cols |

| District Assessments | edfacts/districts_edfacts_assessments_{year} | Yearly (2009-2018, 2020) | 23 cols |

| District Grad Rates | edfacts/districts_edfacts_grad_rates_{year} | Yearly (2010-2019) | 15 cols |

> Note: 2019 assessment data is NOT available (at any level) due to COVID testing waivers.

Codebooks

Codebook .xls files are available for both assessment and graduation rate datasets. Use get_codebook_url() from fetch-patterns.md:

# Assessment codebooks:
url = get_codebook_url("edfacts/codebook_schools_edfacts_assessments")
url = get_codebook_url("edfacts/codebook_districts_edfacts_assessments")

# Graduation rate codebooks:
url = get_codebook_url("edfacts/codebook_schools_edfacts_graduation")
url = get_codebook_url("edfacts/codebook_districts_edfacts_graduation")

> Codebook naming note: Graduation rate codebooks use _graduation (not _grad_rates), while the data files use _grad_rates. This follows the same pattern as other Portal sources where codebook names differ from data file names. See datasets-reference.md for the authoritative path mapping.

Dataset Column Differences

Assessment and graduation rate datasets have different column sets:

| Column | Assessments | Grad Rates |

|--------|-------------|------------|

| sex | Yes (1, 2, 99) | No |

| migrant | Yes (1, 99) | No |

| military_connected | Yes (1, 99) | No |

| grade_edfacts | Yes (3-9, 99) | No |

| read_test_ / math_test_ | Yes | No |

| grad_rate_* | No | Yes |

| cohort_num | No | Yes |

| school_name / lea_name | Yes | Yes |

Filtering

# Grade filtering: grade_edfacts uses integer codes
df = df.filter(pl.col("grade_edfacts") == 4)  # Grade 4
df = df.filter(pl.col("grade_edfacts") == 99)  # All grades combined

# Subgroup filtering: special population columns use 1/99 pattern
df_total = df.filter(pl.col("sex") == 99)  # All students (total)
df_econ = df.filter(pl.col("econ_disadvantaged") == 1)  # Economically disadvantaged only

# Race filtering: integer codes
df_black = df.filter(pl.col("race") == 2)  # Black students

Common Pitfalls

| Pitfall | Issue | Solution |

|---------|-------|----------|

| Ranking states by proficiency | Different tests, different cut scores make comparisons meaningless | Use NAEP for cross-state comparisons |

| Comparing 2019-20 to other years | COVID testing waivers created data gaps | Note data gap, exclude year |

| Ignoring suppression | Results biased toward larger schools/subgroups | Document suppression rates, use _midpt variables |

| Assuming proficiency = same thing | State definitions of "proficient" vary widely | Clarify each state's definition |

| Pre/post ESSA comparison | Different accountability systems (NCLB vs ESSA) | Note policy change at 2015 boundary |

| Using string codes for filtering | Portal uses integer encoding, not NCES strings | Always check actual data values; see encoding tables above |

Key Policy Context

| Law | Years | Key Features |

|-----|-------|--------------|

| NCLB | 2002-2015 | AYP, 100% proficiency goal, HQT |

| ESSA | 2015-present | State flexibility, multiple indicators |

• AYP (Adequate Yearly Progress): NCLB requirement eliminated by ESSA
• ESSA Accountability: States design own systems with federal guardrails
• N-size: Minimum students required for reporting (varies by state, typically 10-30)

CRITICAL WARNING: Cross-State Comparisons

State assessment proficiency rates CANNOT be compared across states.

| Factor | Why It Varies |

|--------|---------------|

| Assessment content | Each state creates its own tests |

| Proficiency cut scores | Each state sets own thresholds |

| Standards alignment | State academic standards differ |

| Test difficulty | Not calibrated nationally |

A student "proficient" in one state may score "below basic" in another state taking a harder test with higher cut scores. Rankings of states by proficiency rates are meaningless.

Use NAEP (National Assessment of Educational Progress) for valid cross-state comparisons.

Valid vs. Invalid Analysis Examples

Valid Analysis:

# Within-state trend analysis
state_df = df.filter(pl.col("fips") == 6)  # California only
trend = state_df.group_by("year").agg(
    pl.col("read_test_pct_prof_midpt").mean()
)
# Valid: Same state, same test system

INVALID Analysis:

# DO NOT DO THIS - Cross-state comparison
# This comparison is MEANINGLESS
state_comparison = df.group_by("fips").agg(
    pl.col("read_test_pct_prof_midpt").mean()
).sort("read_test_pct_prof_midpt", descending=True)
# INVALID: Different tests, different standards

Related Data Sources

| Source | Relationship | When to Use |

|--------|--------------|-------------|

| education-data-source-ccd | CCD provides school/district demographics | Combining outcome data with school characteristics |

| education-data-source-crdc | CRDC has discipline, AP, school climate data | Analyzing school equity alongside achievement |

| education-data-source-saipe | SAIPE provides district poverty estimates | Linking poverty to achievement |

| education-data-source-meps | MEPS provides school poverty estimates | School-level poverty and assessment analysis |

| education-data-explorer | Parent discovery skill | Finding available endpoints |

| education-data-query | Data fetching | Downloading via mirrors |

Topic Index

| Topic | Reference File |

|-------|---------------|

| NCLB to ESSA transition | ./references/accountability-context.md |

| State accountability systems | ./references/accountability-context.md |

| Federal reporting requirements | ./references/accountability-context.md |

| Proficiency levels | ./references/assessment-data.md |

| Why states can't be compared | ./references/assessment-data.md |

| NAEP comparison | ./references/assessment-data.md |

| Assessment system changes | ./references/assessment-data.md |

| ACGR calculation | ./references/graduation-rates.md |

| Cohort adjustments | ./references/graduation-rates.md |

| Extended graduation rates | ./references/graduation-rates.md |

| Diploma types | ./references/graduation-rates.md |

| Suppression codes | ./references/variable-definitions.md |

| Missing data values | ./references/variable-definitions.md |

| Range/midpoint variables | ./references/variable-definitions.md |

| Participation rates | ./references/variable-definitions.md |

| COVID-19 data gaps | ./references/data-quality.md |

| State reporting variations | ./references/data-quality.md |

| Known data issues | ./references/data-quality.md |

| Time series breaks | ./references/data-quality.md |

| Students with disabilities | ./references/subgroup-reporting.md |

| English learners | ./references/subgroup-reporting.md |

| Economically disadvantaged | ./references/subgroup-reporting.md |

| Race/ethnicity reporting | ./references/subgroup-reporting.md |

| Homeless/foster/migrant | ./references/subgroup-reporting.md |

| N-size requirements | ./references/subgroup-reporting.md |