Ancestors

Written by Scott Williams 🐧 on 2025-02-05 at 00:58

We've been collecting and mirroring what we can find of public data scrapes of data that has recently gone missing from federal sites or is likely to in the near future. The repos here include public data from CDC, NIH, and NOAA. Be warned that some of these repos are quite large!

https://git.lsit.ucsb.edu/publicdata

[#]datascience #cdc #nih #noaa

=> More informations about this toot | More toots from vwbusguy@mastodon.online

Written by Scott Williams 🐧 on 2025-02-05 at 04:38

Stay tuned. There's more on the way!

=> More informations about this toot | More toots from vwbusguy@mastodon.online

Toot

Written by Scott Williams 🐧 on 2025-02-05 at 23:29

Now including data from #DeptEd as well. Added more CDC data. More on the way!

https://git.lsit.ucsb.edu/publicdata/DeptEd

=> More informations about this toot | More toots from vwbusguy@mastodon.online

Descendants

Written by Scott Williams 🐧 on 2025-02-05 at 23:45

Here's the additional #CDC data that wasn't included in the original dump, with some of these reports going back decades, as well as reports on LGBTQ and HIV/AIDS.

https://git.lsit.ucsb.edu/publicdata/CDC-Data-2025/src/branch/main/other_reports

=> More informations about this toot | More toots from vwbusguy@mastodon.online

Written by Scott Williams 🐧 on 2025-02-06 at 05:42

And now, #Climate data archive from #ClimateGov

https://git.lsit.ucsb.edu/publicdata/climate-gov-data

[#]Climate #datascience #archive

=> More informations about this toot | More toots from vwbusguy@mastodon.online

Written by Scott Williams 🐧 on 2025-02-07 at 01:55

Added more DeptEd and CDC data today and added globalchange.gov data.

https://git.lsit.ucsb.edu/publicdata/globalchange-gov

=> More informations about this toot | More toots from vwbusguy@mastodon.online

Written by Scott Williams 🐧 on 2025-02-07 at 02:00

I do appreciate those who having pointed out some data that I've missed or otherwise haven't archived yet. Please do let me know if you see such things. Unfortunately, some data has use restrictions and I'm only hosting public data here. If it's not public domain or clearly marked creative commons, etc, then I can't host it here.

=> More informations about this toot | More toots from vwbusguy@mastodon.online

Written by Sam Van Horne, Ph.D. on 2025-02-05 at 23:58

@vwbusguy @adamhsparks Does this have public data files for the NCES sample surveys? Like Baccalaureate and Beyond?

=> More informations about this toot | More toots from DataAngler@vis.social

Written by Scott Williams 🐧 on 2025-02-06 at 00:09

@DataAngler @adamhsparks There's a bunch of stuff in a different web layout that I haven't gotten yet. The two sections I got were the most straightforward to find the data. Some of the other sections of the site used asp link generators for the downloads that I haven't tried to fetch yet.

=> More informations about this toot | More toots from vwbusguy@mastodon.online

Written by Scott Williams 🐧 on 2025-02-06 at 03:08

@DataAngler @adamhsparks I just added surveys and EDGE program data. Does this have what you're looking for?

=> More informations about this toot | More toots from vwbusguy@mastodon.online

Written by Sam Van Horne, Ph.D. on 2025-02-06 at 18:51

@vwbusguy @adamhsparks I actually seem to think that those public-use datafiles are no longer available. Maybe NCES changed to only letting people interact with sample surveys through their online tools. (Only academic researchers can apply to use the restricted-use datafiles.) The list of sample surveys is here: https://nces.ed.gov/datalab

=> More informations about this toot | More toots from DataAngler@vis.social

Written by Scott Williams 🐧 on 2025-02-06 at 19:14

@DataAngler @adamhsparks Some of these appear to be "restricted use", which means I'm not sure if I can legally host it here, but that doesn't mean you can't legally download it yourself.

https://nces.ed.gov/surveys/b&b/

=> More informations about this toot | More toots from vwbusguy@mastodon.online

Written by Scott Williams 🐧 on 2025-02-06 at 19:16

@DataAngler @adamhsparks It looks like B&B might already be offline?

https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2024483

=> More informations about this toot | More toots from vwbusguy@mastodon.online

Written by Scott Williams 🐧 on 2025-02-06 at 19:18

@DataAngler @adamhsparks But yeah, looks like anything in DataLab isn't going to be something I can host here, given that it's not public domain, creative commons, etc.

https://nces.ed.gov/datalab/membership/login

=> More informations about this toot | More toots from vwbusguy@mastodon.online

Written by Sam Van Horne, Ph.D. on 2025-02-06 at 19:29

@vwbusguy @adamhsparks Some surveys do have public-use datafiles, like the Early Childhood Studies Program.

https://nces.ed.gov/ecls/dataproducts.asp

=> More informations about this toot | More toots from DataAngler@vis.social

Written by Scott Williams 🐧 on 2025-02-06 at 21:44

@DataAngler @adamhsparks Good catch! I'll add the public ECLS data shortly.

=> More informations about this toot | More toots from vwbusguy@mastodon.online

Written by Scott Williams 🐧 on 2025-02-06 at 22:46

@DataAngler @adamhsparks I added ECLS and the public Longitudinal data (under surveys).

=> More informations about this toot | More toots from vwbusguy@mastodon.online