Demographic Economic Data Extraction (DEDE) API Tools
... create custom demographic-economic files from large Web-based datasets
... use structured, menu-based API tools to extract data you need
... for both developers & users with no coding experience
Use the Demographic Economic Data Extraction (DEDE) software to extract demographic-economic data from various statistical programs and datasets. Use DEDE to extract data from statistical programs including:
Census 2000 & Census 2010
American Community Survey 2010 through ACS and ACS
BEA Economic Time Series
BLS Economic Time Series
Situation & Outlook
Many other statistical programs - see operations details for complete list.
DEDE, and extended data access features, are now available as a part of the CV XE GIS APIGateway.
DEDE includes the ability to setup custom data extraction (geographic code lists and subject matter code lists) that can be re-used and completely controlled by the user. Downloaded data may be saved as XLS, CSV, or dBase (.dbf) files that can be converted to other formats. Use the data with your preferred statistical analysis software. The dBase structure lends itself to easy merger of multi-sourced subject matter data for the same geography (e.g., Census 2010 and ACS 2012). The dBase structure also enables easy integration of subject matter data into corresponding shapefiles.
DEDE makes use of Application Programming Interface (API) operations that enable downloading data directly from servers. As a result, the data extraction application makes little use of locally stored data. An Internet connection is required for most applications.
DEDE is a batch-oriented application that makes use of a nominal user interface. The follow topics review DEDE operations focused on block group level data extraction.
Click graphic for larger view. See pointer in grid at cell showing total population (item B01001_001E from ACS 2012 5 year estimates) of 1,510 for Census 2010 block group 51510200812. See subject matter and GEOID codes in lists at left. These lists were populated by reading the geocodes file and items file shown above the grid. Export data in the grid using File>SaveAs operation.
Illustration Application Area -- Alexandria, VA
Alexandria, VA bold boundary, Census 2010 block groups black boundary and Census 2000 BGs red boundary. Labels show Census 2010 BGs.
Click graphic for larger view. View developed using CV XE GIS. Integrate DEDE extracted data into block group shapefiles, as shown above, to create thematic/pattern maps.
To run a block group data extraction proceed as follows:
Create/select the Geocode List file
Create/select the Items List file
Select the source statistical program and dataset
Processing results are displayed in grid
Optionally click on a column header to sort the grid on that column; click again to sort in other direction
Optionally use File>SaveAs to save data in grid to file
- optionally toggle File>Settings>ExportShowXLS to automatically open if saved as XLS.
Geocode List File
The Geocode List file is a text file that contains a list of codes for block groups to be extracted. The start-up list includes a few sample block group codes. For Census 2010 and ACS datasets, the block group codes are Census 2010 vintage codes. For Census 2000 datasets, the block group codes are Census 2000 vintage codes. Open a geocodes list file using File>Open Geocodes File.
The block group code must be a numeric code structured like SSCCCTTTTTTG where SS is the FIPS state code, CCC is the FIPS county code, TTTTTT is the census tract code and G is the block group code.
Sample block group code files are provided with the installer.
tl_2010_51510_bg00.txt -- all Census 2000 block groups in Alexandria, VA
tl_2010_51510_bg10.txt -- all Census 2010 block groups in Alexandria, VA
If using a county-wide set of block groups, the easiest way to develop the block group code list is to pull the GEOID column from the corresponding block group shapefile. Resources:
Census 2010 state by block group codes from TIGER 2013 state by block group shapefiles.
Census 2010 county by block group codes from TIGER 2010 Census 2010 county by block group shapefiles.
Census 2000 state by block group codes from TIGER 2010 Census 2000 county by block group shapefiles.
More about TIGER 2013.
Subject Matter Items and Items List File
The Items List file is a text file that contains a list of codes for subject matter items to be extracted. The start-up list includes a few sample ACS subject matter items (items_acs.txt). Open an items list file using File>Open Items File.
The item code must correspond exactly to the "name" of the item assigned by Census or other authority for use with API calls. Sample item code files are included with the installer.
items_2000_sf3.txt -- selected Census 2000 SF3 items
items_2010_sf1.txt -- selected Census 2010 SF1 items
items_acs.txt -- selected ACS summary file items (works with ACS 2010, 2011, 2012 and later as available)
Subject matter field names must correspond to:
Census 2000 SF3.-- PDF -- Table Matrix Section 7; "data dictionary reference name"
Census 2010 SF1 -- DOC -- Table Matrix; "data dictionary reference name"
ACS 5 year Summary File data items similar, almost the same, from year to year but some differences exist
ACS 2010 SF -- PDF -- BG Availability shown in Appendix E
Use ACS 2010 Tables -- XLS -- to select subject matter field name
ACS 2011 SF -- PDF -- BG Availability shown in Appendix E
Use ACS 2011 Tables -- XLS -- to select subject matter field name
ACS 2012 SF -- PDF -- BG Availability shown in Appendix E
Use ACS 2012 Tables -- XLS -- to select subject matter field name
In each sample file, a brief description of the subject matter follows the subject matter item name. The descriptions are optional and not otherwise used. A space delimits the subject matter item name from the optional subject matter item description. There are no spaces in the subject matter item name.
Subject Matter Look-up & Reference Resources
Statistical Program Geographies & Items -- http://api.census.gov/data.html
ACS 2013 1-Year DP Items (example for this program) -- http://api.census.gov/data/2013/acs1/profile/variables
... as a JSON object: http://api.census.gov/data/2013/acs1/profile/variables.json
... for a single item: http://api.census.gov/data/2013/acs1/profile/variables/DP02_0001E
ACS 2012 5-Year SF Items (example for this program) -- http://api.census.gov/data/2012/acs5/variables
... as a JSON object: http://api.census.gov/data/2012/acs5/variables.json
... for a single item: http://api.census.gov/data/2012/acs5/variables/B19001_001E
Dataset Selection/Source Statistical Program
Use the DEDE Dataset dropdown to select the Dataset. Geocode list items and subject matter list items must be in accord with the selected Dataset. That is, Census 2000 geocodes cannot (in general) be used with Census 2010 subject matter items. Or, Census 2010 SF1 subject matter items cannot be used with ACS Datasets. Statistical program datasets supported include:
Census 2000 Summary File 3
Census 2010 Summary File 1
ACS 2010 5-year estimates
ACS 2011 5-year estimates
ACS 2012 5-year estimates
ACS 2013 5-year estimates
Use File>SaveAs to save the grid context to file.
Processing & Operations
1. Geocodes not found (an error in the geocodes list file).
If a geocode is not located, the data record is written with the specified geocode but all subject matter field values are written as -1.
2. Geocodes & Item Codes Manual Entry.
Optionally use the mouse to empty the Geocodes List and/or Item Codes List and key in codes manually. Use this feature to look-up any one or a few item value(s) for one or a few block group(s).
3. How to determine block group geocodes of interest? What is my block group code?
Option 1. For all block groups in county(s) or state(s), use TIGER files. See the above discussion under Geocode List File. Option 2. Use the APIGeocoder (all versions including no fee version). Use the "CodeOne" button to code one address; the geocode is displayed in the grid or code a group of addresses.
Extended Data Resources Supported by DEDE
These additional data resources may also be accessed using DEDE API operations:
1. Access subject matter/datasets:
Situation & Outlook multi-geography; current estimates & projections
Current Employment & Wages establishments, employment & wages; by NAICS; county up; quarterly time series
Regional Economic Information System gdp & personal income by component; by NAICS; county up; time series
2. Access geography supported in Level 1 plus other geographies (varies by statistical program/datasets):
ProximityOne User Group
Join the ProximityOne User Group to keep up-to-date with new developments relating to metros and component geography decision-making information resources. Receive updates and access to tools and resources available only to members. Use this form to join the User Group.
Support Using these Resources
Learn more about accessing and using demographic-economic data and related analytical tools. Join us in a Data Analytics Lab session. There is no fee for these one-hour Web sessions. Each informal session is focused on a specific topic. The open structure also provides for Q&A and discussion of application issues of interest to participants.
ProximityOne develops geographic-demographic-economic data and analytical tools and helps organizations knit together and use diverse data in a decision-making and analytical framework. We develop custom demographic/economic estimates and projections, develop geographic and geocoded address files, and assist with impact and geospatial analyses. Wide-ranging organizations use our tools (software, data, methodologies) to analyze their own data integrated with other data. Follow ProximityOne on Twitter at www.twitter.com/proximityone. Contact ProximityOne (888-364-7656) with questions about data covered in this section or to discuss custom estimates, projections or analyses for your areas of interest.