CV XE GIS Home Page





  ProximityOne Main

  ACS 2018

  New Topics & Updates

  Situation & Outlook

  Related Sections
  - Applications Gallery
  - States
  - School Districts
  - Metros
  - Congressional
  - Counties
  - Census Tracts
  - Block Groups
  - Census Blocks
  - ZIP Codes
  - Urban Areas

  Estimates-Projections
  - County Interactive
  - 5-year Projections
  - Quarterly Projections
  - 2030 Projections
  - 2060 Projections

  CV XE GIS
    Visual Analysis

  Interactive Tables

  National Scope
  Demographic Change
  2000-2010
  - States
  - Metros
  - Congressional Dist
  - School Districts
  - Counties
  - Cities/Places
  - Census Tracts


 
Decision-Making Information
  ProximityOne
  information resources & solutions
  (888) DMI-SOLN
  (888) 364-7656





Put data to work more effectively.
Certificate in Data Analytics



%Hispanic Population by Block Group
Houston, TX area -- see details below




Data Analytics Blog
Power of Combining Maps with Data

Support & Technical Assistance
help using these resources
 

 

Demographic Economic Data Extraction (DEDE) API Tools
  ... create custom demographic-economic files from large Web-based datasets
  ... use structured, menu-based API tools to extract data you need
  ... for both developers & users with no coding experience

February 2020. Use the Windows-based Demographic Economic Data Extraction (DEDE) software to access and build your own geostatistical datasets using API tools. This document provides information on how to install and use DEDE. The focus is on accessing and using data from the American Community Survey 2013 through ACS 2018 for wide-ranging geography down to the block group level.

Use data developed using DEDE for applications such as analyzing affordable housing or income inequality at the neighborhood or subneighborhood geographic levels. Businesses can use the data to analyze demographic-economic characteristics of existing and prospective potentially new markets and service areas. Create time-series or longitudinal datasets to examine trends and patterns. Benefit from the speed of data development and reduced staff time to develop datasets with essential program management and decision-making data.

Topics
  • 01 DEDE Overview
  • 02 DEDE Start-up View & User Interface
  • 03 DEDE First Time Use
  • 04 Subject Matter/Items File; Selecting Subject Matter
        041 Viewing Subject Matter Items Available
  • 05 Geography & Geographic Vintages
  • 06 Interactive Table; Illustrative Demographic Economic Data Extract
  • 07 Mapping & Geospatial Analysis of DEDE Output
  • 08 Working with Census/ACS Table/Item Data
  • 09 Working with Census/ACS Block Group & Related Small Area Data
  • 10 Terms of Use

DEDE supports a wide range of subject matter data for a wide range of geographic levels and time periods. We release updated versions of DEDE2020 to include annually released updates to ACS, Census 2010, Census 2020, other Federal statistical programs and the ProximityOne Situation & Outlook.

DEDE Overview
Use the Demographic Economic Data Extraction (DEDE) software to extract demographic-economic data from web-based databases using Application Programming Interface (API) technologies. DEDE is not affiliated or sponsored by the Census Bureau. It is a proprietary resource of ProximityOne and subject to these terms of use .. see below.

Getting Started -- installation
Install the small footprint DEDE2020 software on any Windows computer. Run the DEDE2020 installer:
  Run DEDE2020 installer
  -- requires your ProximityOne User Group userid (join here; no fee).
  -- operation requires Internet and an API key.
  -- get your API key code here (different/separate from User Group userid).

Start-up .. go to top
At completion of installation, launch the DEDE application. Initially you will see a message that says "missing API key?. Insert your API key (an approximate 30 character code received from the above step). With DEDE running, click Settings>Start-up Setting; the settings form appears; paste your API key in the upper API key edit box, click the Save and Apply buttons and close the settings form.

Start-up View .. go to top

User Interface:
Main Menu Toolbar
Settings .. use Settings>Start-up INI to change DEDE2020 start-up values.
Help .. use Help>Reference to view this page
  .. use Help>About for information about DEDE2020.
Control Panel .. at left ..
Statistical Program .. use to select statistical program.
Situation & Outlook
Census 2010
Census 2020
ACS 2018_5 -- 5 year estimates
ACS 2018_1 -- 1-year estimates
ACS 2017_5
ACS 2017_1
ACS 2016_5
ACS 2016_1
ACS 2015_5
ACS 2015_1
ACS 2014_5
ACS 2014_1
ACS 2013_5
ACS 2013_1

Geographic Level .. use to select geographic/summary level of geography to be processed.
100 Census Block
150 Block Group
140 Census Tract
050 County
160 City/Place
970 School District
500 116th Congressional District
300 CBSA (MSA/MISA)
320 Combined Statistical Area
040 State
795 PUMA
860 ZIP Code Area
400 Urban Area
010 United States

Start button .. starts process to generate API extracted data, saved to the local computer.
Stop button .. not recommended but can be used to immediately stop processing.
Create CSV File .. if checked, creates a CSV file.
Open in Grid .. if checked, shows data in grid.

ACS Metadata Query .. specifies the SQL-like query to be used to display ACS metadata in Grid.
  BG='1' will result in only BG-relevant content to be displayed.
View ACS Metadata button .. click this button to show ACS metadata in grid.
Close button .. click this button to close ACS metadata grid.
Files Panel .. at top right
Items File Path/Name .. The "Items File Path/Name" edit box contains the name of the items file that will be processed.
  .. see details below. .. default: stab='AZ' .. process all areas in all counties in AZ .. Arizona USPS abbreviation
.. example: geoid='04013' .. process all areas in Maricopa County, AZ .. 04013 is state+county FIPS code
.. example: stab='AZ' .or. stab='CA' .. process all areas in Arizona and California.
.. leave Query empty to process all U.S. areas
Output File Path Name .. name of dbase/dbf file that will be generated when start button pressed.
  .. file automatically named based on inout file name.
  .. optionally manually edit the name. The file name extension must be dbf.

Grid Panel .. lower right .. majority of main window
..
..

First time Use .. go to top
The following view shows user interface after processing is complete when the Start button is clicked with all default settings. Progress is shown in statusbar where the state+county FIPS code (GEOID) is displayed. Grid populated at end of processing. Processing time about 1 minute.


Starting the process .. the user selects:
- the Statistical Progrm
- the Geographic Level - the Items File (see below) to be processed
- sets the File Query if desired (if empty all U.S. processed)
- then clicks Start button to start processing .. wait for "Done" message.
What happens during processing : - an API query is performed accessing data for each area set to be processed.
- the file specified in the "Output File Path Name" edit box is written .. always a dbf/dbase file.
  .. this file may be merged directly with a BG shapefile using the CV XE GIS database merge feature.
- if the Create CSV file is checked, a CSV file is generated based on the "Output File Path Name" edit box
  .. dbf extension changed to csv.
- if the Open CSV in Grid is checked, the data are shown in the grid.

Using the geography Query
- the geography Query sets a geographic (state and/or county) limitation/filter.
- counties process will be limited by this setting.
- the default setting stab='DE' says to limit processing to counties is Delaware.
  .. Delaware is used as the state has 3 counties.
- use the "View State County" button (in Control Panel) to preview a list of names and codes meeting the Query.

Grid content
- the grid shows one line/row for each area processed (block groups in this case).
- the first column shows the geocode (SS-CCC-TTTTTT=G for block group).
- additional colums show values of subject matter items as specified in the "Items File Path/Name" edit box.
  .. see about structure below.
- in the above view,, the pointer is located on the cell that has value 3354 (3,354).
  .. this is cell[1,1] .. the value 3354 is shown in cell[1,2]
- "block group 100010401001 has a total population (B01001001E) ACS 2018 estimate of 3354."

Items File .. go to top
The "Items File Path/Name" edit box contains the name of the items file that will be processed.
Use an existing items file provided with the installer or create your own.
The items file is a text file that must contain a row for each item to be processed.
The number of rows/items limit per file is 50 items.
The leftmost part of the item text line must be a valid Census API item descriptor followed by a blank character.
  .. item codes may be displayed by clicking the "View ACS Metadata" button in Control Panel.
  .. item code B01001_001 (total population) must be entered in the Items File as B01001_001E.
  .. the E (estimate) character is mandatory.
Additional descriptive text is recommended but not required.
Content of the default file:
B01001_001E Total Population
B01001_002E Male
B01001_026E Female
B02001_002E White alone
B02001_003E Black or African American alone
B02001_004E American Indian and Alaska Native alone
B02001_005E Asian alone
B02001_006E Native Hawaiian and Other Pacific Islander alone
B02001_007E Some other race alone
B02001_008E Two or more races
B03002_012E Hispanic (any race)
B09020_001E Population 65 years and over
B11002_001E Population in Households
B11001_001E Total Households
B11001_002E Family Households
B28007_002E Civilian Labor Force
B28007_003E Employed (in Civilian Labor Force)
B15002_001E Population 25 years and over
B15002_011E High school graduate (Male, age 25+)
B15002_028E High school graduate (Female, age 25+)
B25002_001E Total housing units
B25003_002E Owner occupied units
B25003_003E Renter occupied units
B25002_003E Vacant units
B19013_001E Median Household Income
B19113_001E Median Family Income
B25077_001E Median housing value
B25064_001E Median gross rent

About the base file items ...
This selection of items covers many basic subject matter items of interest.
- the first several items pertain to race/origin.
- population 65 years and over might relate to naturally occurring retirement communities
- population in households enables also examining population in group quarters (B01001_001E - B11002_001E)
- Civilian Labor Force and Employed enable examining unemployment and the unemployment rate.
- Population 25 years+, H.S. graduate (Male, age 25+), H.S. graduate (Female, age 25+)
    enable computing percent H.S. graduate.

Viewing Subject Matter Items Available .. go to top
To view all ACS subject matter items available, click the "View ACS Metadata" button in Control Panel. The following view shows results when the "ACS Metadata Query" is set to "BG='1'". This query will show all items available for block groups.

- if the query is empty, all ACS table/items will be displayed.
- use the query "BG='1' .and. TABLE='1'" to view table headers for tables available for BGs.
- use the query like "TABLEID='B25075'" to view the entire table elements for Table B25075.
    .. B25075 is a Housing Value table -- number of units by housing value interval (27 items)
- do not use the quotation character (") in above examples, only tick(') character.
- embedded dot (.) character around logical operators required.

Grid2 is used to display subject matter metadata and geographic reference data. Grid2 is displayed "on top of" Grid1 which displays the statistical data.
- Grid1 and Grid2 may be used interactively; switching between each other.

Geography & Geographic Vintages .. go to top
DEDE provides access to subject matter data for wide-ranging geography. While many of the types of geographic areas remain the same over the time periods supported by DEDE, others are subject to change. As an example, for CBSAs, ACS 2018 data are tabulated for the August 15, 2017 OMB/Census vintage while the ACS 2017 data are tabulated for the July 15, 2015 OMB/Census vintage. For the ACS annual data, use the following links to view the scope of areas published and vintages.
- ACS 2018
- ACS 2017
- ACS 2016
- ACS 2015
- ACS 2014
- ACS 2013

ACS 2018 California & Texas Block Group Demographic Characteristics interactive table .. go to top
  Click column header to sort; again to sort other direction.
  Click ShowAll button between queries to refresh. Usage notes below table.
  See related Interactive Tables Main Page


The above table shows the DEDE2020 Base Items File subject matter for California and Texas block groups.
- underlying file created by DEDE when "base items file" is used with Query: stab='CA' .or. stab='TX'
- there are 39,023 BGs in these two states. -- scroll to bottom of table to determine this number.
- sorting on Total Population, note that the largest BG is in state-county 48157
    -- Fort Bend, County, TX with 55,407 population.
- sorting in descending order on Median Household Income,
    -- see that there are 194 BGs in these two states having a top coded value of $250,001.
- a cell value of -1 indicates estimate not available.

Mapping & Geospatial Analysis of DEDE Output .. go to top
Files created using DEDE are structured for immediate use in mapping and geospatial analysis applications. The following view shows patterns of diversity by block group in the Houston, TX area ... percent Hispanic population (of any race) by block group. Click graphic for larger view -- expand browser to full window for best quality view. Larger view shows block groups labeled with the ACS 2018 Hispanic population.


- view created using ProximityOne CV XE GIS software and related GIS project.
- see related ACS-based thematic maps for this and other areas.

To develop the above view (and create wide-ranging variations), the "CA-TX" base file generated using DEDE was merged with the Texas by block group shapefile.
1 - download the Texas by block group shapefile using the GeoGateway feature of the CV XE GIS software.
2 - Use the CV XE GIS Database>dBMerge feature to merge the DEDE dbf/dbase file with the
      Texas block group shapefile based on the common geoid key.
3 - add the shapefile to a GIS project and create layer setting to display the above pattern map or a variation.

Working with Census/ACS Table/Item Data .. go to top
DEDE can be used to extract decennial census and ACS data stored in large databases to build local datasets using API tools. DEDE enables access to these demographic-economic data typically structured as tables containing one or more items. Stated differently, the data are published as "tables" comprised of one of more cells or "items."

The ACS 2018 table B25003 is shown below. The title of this table is Tenure. Table B25003 contains three items, tabulated for the universe of occupied housing units. The representation of the subject matter as shown below is referred to as a table shell. The individual table items are: Total (total occupied units), Owner occupied (units), and Renter occupied (units). Among all table/items ACS 2018 "summary detailed tables" published, the item codes (B25003_001, B25003_002, B25003_003) are handles or variable names used with the descriptive attribute.

 

While tables and table/item structures are very similar for each annual ACS data release, there are some variations from year to year. Items included in the default Items File (see above) (C:\dede\api_items_acs_base.txt) are available in each annual ACS program 2013 through 2018/the most recent year available.

Working with Census/ACS Block Group & Related Small Area Data .. go to top
These notes are focused on the block group (BG -- summary level 150) for which demographic-economic data are tabulated/published for each annual set of ACS 5 year estimates. BGs are the smallest geographic level for ACS estimates are tabulated. Data for BGs, tracts, ZIP code areas and state legislative districts are tabulated only in the ACS 5-year estimates. ACS 2018 5-year estimates cover annual respondent data for the calendar years 2014 through 2018. In some respects the estimates are centric to mid-2016. See related information about ACS 1-year vs 5-year tabulation levels in the ACS 2018 5-year data section.

Block groups are subsets of census tracts, BGs cover the U.S. wall-to-wall. There are 5,015 items tabulated/published by BG for each annual ACS release; a total of 1,091,966,100 data cells for each release year. (5,015 cells for each of 217,739 BGs). BGs are one of many geographic levels for which ACS data are tabulated. All of these data, except for two items, are estimates subject to errors of estimation and other errors. Generally, the smaller the geographic area, and the smaller the number of respondents, the larger the relative error. This said, our experience in using these data is that they are quite good.

Some items, such as total population, total housing units, households are iterated throughout the 5,015 subject matter items, even though they have the same value for any given BG, resulting in the number of unique data items by BG more around 3,000. In summary, DEDE2020 enables the user to access ~3,000 items for each of 217,739 BGs for each year 2013 through 2018 (expanding to 2019 in December 2020) for a total of approximately 4.9 billion items.

Terms of Use .. go to top
The DEDE level 1 software is available for personal use by members of the ProximityOne User Group. There is no warranty of any type associated with any aspect of the use of DEDE. The user is solely responsible for any use made of DEDE. DEDE may not be redistributed.

ProximityOne User Group ... goto top
Join the ProximityOne User Group to keep up-to-date with new developments relating to metros and component geography decision-making information resources. Receive updates and access to tools and resources available only to members. Use this form to join the User Group.

Support Using these Resources
Learn more about accessing and using demographic-economic data and related analytical tools. Join us in a Data Analytics Lab session. There is no fee for these one-hour Web sessions. Each informal session is focused on a specific topic. The open structure also provides for Q&A and discussion of application issues of interest to participants.

Additional Information
ProximityOne develops geodemographic-economic data and analytical tools and helps organizations knit together and use diverse data in a decision-making and analytical framework. We develop custom demographic/economic estimates and projections, develop geographic and geocoded address files, and assist with impact and geospatial analyses. Wide-ranging organizations use our tools (software, data, methodologies) to analyze their own data integrated with other data. Follow ProximityOne on Twitter at www.twitter.com/proximityone. Contact us (888-364-7656) with questions about data covered in this section or to discuss custom estimates, projections or analyses for your areas of interest.


Copyright © . ProximityOne. All Rights Reserved.
Sitemap | Contact Us | News