Saturday, September 15, 2012

How life really has gotten better for researchers

The first non-thesis project I worked on as an assistant professor used the 1 in 1000 Public Use Microdata Sample of the US Census.  My recollection is that the sample had about 63,000 observations; I had to spin a tape to read data into a big iron VMS machine using SAS, and it took about 10 hours to do so.  The year was 1990.

Yesterday, I read five year ACS household data into my Macbook Pro using Stata.  It is broken into four files with about 1.5 million observations each.  Each file took about 2 minutes to read. This is really, really nice.