Demographic Economic Data Extraction (DEDE) API Tools
... create custom demographic-economic files from large Web-based datasets
... use structured, menu-based API tools to extract data you need
... for both developers & users with no coding experience
February 2020. Use the Windows-based Demographic Economic Data Extraction (DEDE) software to access and build your own geostatistical datasets using API tools. This document provides information on how to install and use DEDE. The focus is on accessing and using data from the American Community Survey 2013 through ACS 2018 for wide-ranging geography down to the block group level.
Use data developed using DEDE for applications such as analyzing affordable housing or income inequality at the neighborhood or subneighborhood geographic levels. Businesses can use the data to analyze demographic-economic characteristics of existing and prospective potentially new markets and service areas. Create time-series or longitudinal datasets to examine trends and patterns. Benefit from the speed of data development and reduced staff time to develop datasets with essential program management and decision-making data.
01 DEDE Overview
02 DEDE Start-up View & User Interface
03 DEDE First Time Use
04 Subject Matter/Items File; Selecting Subject Matter
041 Viewing Subject Matter Items Available
05 Geography & Geographic Vintages
06 Interactive Table; Illustrative Demographic Economic Data Extract
07 Mapping & Geospatial Analysis of DEDE Output
08 Working with Census/ACS Table/Item Data
09 Working with Census/ACS Block Group & Related Small Area Data
DEDE supports a wide range of subject matter data for a wide range of geographic levels and time periods. We release updated versions of DEDE2020 to include annually released updates to ACS, Census 2010, Census 2020, other Federal statistical programs and the ProximityOne Situation & Outlook.
Getting Started -- installation
Install the small footprint DEDE2020 software on any Windows computer. Run the DEDE2020 installer:
Run DEDE2020 installer
-- requires your ProximityOne User Group userid (join here; no fee).
-- operation requires Internet and an API key.
-- get your API key code here (different/separate from User Group userid).
Start-up .. go to top
At completion of installation, launch the DEDE application. Initially you will see a message that says "missing API key?. Insert your API key (an approximate 30 character code received from the above step). With DEDE running, click Settings>Start-up Setting; the settings form appears; paste your API key in the upper API key edit box, click the Save and Apply buttons and close the settings form.
Start-up View .. go to top
Main Menu Toolbar
Control Panel .. at left ..
Files Panel .. at top right
Grid Panel .. lower right .. majority of main window
First time Use .. go to top
The following view shows user interface after processing is complete when the Start button is clicked with all default settings. Progress is shown in statusbar where the state+county FIPS code (GEOID) is displayed. Grid populated at end of processing. Processing time about 1 minute.
Starting the process .. the user selects:
- the Statistical Progrm
- the Geographic Level
- sets the File Query if desired (if empty all U.S. processed)
- then clicks Start button to start processing .. wait for "Done" message.
What happens during processing :
- the file specified in the "Output File Path Name" edit box is written .. always a dbf/dbase file.
.. this file may be merged directly with a BG shapefile using the CV XE GIS database merge feature.
- if the Create CSV file is checked, a CSV file is generated based on the "Output File Path Name" edit box
.. dbf extension changed to csv.
- if the Open CSV in Grid is checked, the data are shown in the grid.
Using the geography Query
- the geography Query sets a geographic (state and/or county) limitation/filter.
- counties process will be limited by this setting.
- the default setting stab='DE' says to limit processing to counties is Delaware.
.. Delaware is used as the state has 3 counties.
- use the "View State County" button (in Control Panel) to preview a list of names and codes meeting the Query.
- the grid shows one line/row for each area processed (block groups in this case).
- the first column shows the geocode (SS-CCC-TTTTTT=G for block group).
- additional colums show values of subject matter items as specified in the "Items File Path/Name" edit box.
.. see about structure below.
- in the above view,, the pointer is located on the cell that has value 3354 (3,354).
.. this is cell[1,1] .. the value 3354 is shown in cell[1,2]
- "block group 100010401001 has a total population (B01001001E) ACS 2018 estimate of 3354."
Items File .. go to top
The "Items File Path/Name" edit box contains the name of the items file that will be processed.
Use an existing items file provided with the installer or create your own.
The items file is a text file that must contain a row for each item to be processed.
The number of rows/items limit per file is 50 items.
The leftmost part of the item text line must be a valid Census API item descriptor followed by a blank character.
.. item codes may be displayed by clicking the "View ACS Metadata" button in Control Panel.
.. item code B01001_001 (total population) must be entered in the Items File as B01001_001E.
.. the E (estimate) character is mandatory.
Additional descriptive text is recommended but not required.
Content of the default file:
About the base file items ...
This selection of items covers many basic subject matter items of interest.
- the first several items pertain to race/origin.
- population 65 years and over might relate to naturally occurring retirement communities
- population in households enables also examining population in group quarters (B01001_001E - B11002_001E)
- Civilian Labor Force and Employed enable examining unemployment and the unemployment rate.
- Population 25 years+, H.S. graduate (Male, age 25+), H.S. graduate (Female, age 25+)
enable computing percent H.S. graduate.
Viewing Subject Matter Items Available .. go to top
To view all ACS subject matter items available, click the "View ACS Metadata" button in Control Panel. The following view shows results when the "ACS Metadata Query" is set to "BG='1'". This query will show all items available for block groups.
- if the query is empty, all ACS table/items will be displayed.
- use the query "BG='1' .and. TABLE='1'" to view table headers for tables available for BGs.
- use the query like "TABLEID='B25075'" to view the entire table elements for Table B25075.
.. B25075 is a Housing Value table -- number of units by housing value interval (27 items)
- do not use the quotation character (") in above examples, only tick(') character.
- embedded dot (.) character around logical operators required.
Grid2 is used to display subject matter metadata and geographic reference data. Grid2 is displayed "on top of" Grid1 which displays the statistical data.
- Grid1 and Grid2 may be used interactively; switching between each other.
Geography & Geographic Vintages .. go to top
DEDE provides access to subject matter data for wide-ranging geography. While many of the types of geographic areas remain the same over the time periods supported by DEDE, others are subject to change. As an example, for CBSAs, ACS 2018 data are tabulated for the August 15, 2017 OMB/Census vintage while the ACS 2017 data are tabulated for the July 15, 2015 OMB/Census vintage. For the ACS annual data, use the following links to view the scope of areas published and vintages.
- ACS 2018
- ACS 2017
- ACS 2016
- ACS 2015
- ACS 2014
- ACS 2013
ACS 2018 California & Texas Block Group Demographic Characteristics interactive table .. go to top
Click column header to sort; again to sort other direction.
Click ShowAll button between queries to refresh. Usage notes below table.
See related Interactive Tables Main Page
The above table shows the DEDE2020 Base Items File subject matter for California and Texas block groups.
- underlying file created by DEDE when "base items file" is used with Query: stab='CA' .or. stab='TX'
- there are 39,023 BGs in these two states. -- scroll to bottom of table to determine this number.
- sorting on Total Population, note that the largest BG is in state-county 48157
-- Fort Bend, County, TX with 55,407 population.
- sorting in descending order on Median Household Income,
-- see that there are 194 BGs in these two states having a top coded value of $250,001.
- a cell value of -1 indicates estimate not available.
Mapping & Geospatial Analysis of DEDE Output .. go to top
Files created using DEDE are structured for immediate use in mapping and geospatial analysis applications. The following view shows patterns of diversity by block group in the Houston, TX area ... percent Hispanic population (of any race) by block group. Click graphic for larger view -- expand browser to full window for best quality view. Larger view shows block groups labeled with the ACS 2018 Hispanic population.
- view created using ProximityOne CV XE GIS software and related GIS project.
- see related ACS-based thematic maps for this and other areas.
To develop the above view (and create wide-ranging variations), the "CA-TX" base file generated using DEDE was merged with the Texas by block group shapefile.
1 - download the Texas by block group shapefile using the GeoGateway feature of the CV XE GIS software.
2 - Use the CV XE GIS Database>dBMerge feature to merge the DEDE dbf/dbase file with the
Texas block group shapefile based on the common geoid key.
3 - add the shapefile to a GIS project and create layer setting to display the above pattern map or a variation.
Working with Census/ACS Table/Item Data .. go to top
DEDE can be used to extract decennial census and ACS data stored in large databases to build local datasets using API tools. DEDE enables access to these demographic-economic data typically structured as tables containing one or more items. Stated differently, the data are published as "tables" comprised of one of more cells or "items."
The ACS 2018 table B25003 is shown below. The title of this table is Tenure. Table B25003 contains three items, tabulated for the universe of occupied housing units. The representation of the subject matter as shown below is referred to as a table shell. The individual table items are: Total (total occupied units), Owner occupied (units), and Renter occupied (units). Among all table/items ACS 2018 "summary detailed tables" published, the item codes (B25003_001, B25003_002, B25003_003) are handles or variable names used with the descriptive attribute.
While tables and table/item structures are very similar for each annual ACS data release, there are some variations from year to year. Items included in the default Items File (see above) (C:\dede\api_items_acs_base.txt) are available in each annual ACS program 2013 through 2018/the most recent year available.
Working with Census/ACS Block Group & Related Small Area Data .. go to top
These notes are focused on the block group (BG -- summary level 150) for which demographic-economic data are tabulated/published for each annual set of ACS 5 year estimates. BGs are the smallest geographic level for ACS estimates are tabulated. Data for BGs, tracts, ZIP code areas and state legislative districts are tabulated only in the ACS 5-year estimates. ACS 2018 5-year estimates cover annual respondent data for the calendar years 2014 through 2018. In some respects the estimates are centric to mid-2016. See related information about ACS 1-year vs 5-year tabulation levels in the ACS 2018 5-year data section.
Block groups are subsets of census tracts, BGs cover the U.S. wall-to-wall. There are 5,015 items tabulated/published by BG for each annual ACS release; a total of 1,091,966,100 data cells for each release year. (5,015 cells for each of 217,739 BGs). BGs are one of many geographic levels for which ACS data are tabulated. All of these data, except for two items, are estimates subject to errors of estimation and other errors. Generally, the smaller the geographic area, and the smaller the number of respondents, the larger the relative error. This said, our experience in using these data is that they are quite good.
Some items, such as total population, total housing units, households are iterated throughout the 5,015 subject matter items, even though they have the same value for any given BG, resulting in the number of unique data items by BG more around 3,000. In summary, DEDE2020 enables the user to access ~3,000 items for each of 217,739 BGs for each year 2013 through 2018 (expanding to 2019 in December 2020) for a total of approximately 4.9 billion items.
The DEDE level 1 software is available for personal use by members of the ProximityOne User Group. There is no warranty of any type associated with any aspect of the use of DEDE. The user is solely responsible for any use made of DEDE. DEDE may not be redistributed.
ProximityOne User Group ... goto top
Join the ProximityOne User Group to keep up-to-date with new developments relating to metros and component geography decision-making information resources. Receive updates and access to tools and resources available only to members. Use this form to join the User Group.
Support Using these Resources
Learn more about accessing and using demographic-economic data and related analytical tools. Join us in a Data Analytics Lab session. There is no fee for these one-hour Web sessions. Each informal session is focused on a specific topic. The open structure also provides for Q&A and discussion of application issues of interest to participants.
ProximityOne develops geodemographic-economic data and analytical tools and helps organizations knit together and use diverse data in a decision-making and analytical framework. We develop custom demographic/economic estimates and projections, develop geographic and geocoded address files, and assist with impact and geospatial analyses. Wide-ranging organizations use our tools (software, data, methodologies) to analyze their own data integrated with other data. Follow ProximityOne on Twitter at www.twitter.com/proximityone. Contact us (888-364-7656) with questions about data covered in this section or to discuss custom estimates, projections or analyses for your areas of interest.