Link to Open Austin Dataset Catalog

[mateoclarke]

I hadn't seen this before b/c it hadn't been working until a recent update...

This is a nice complementary tool that @chip-rosenthal created that we could link to:
http://data.open-austin.org/data-catalogs/views/by-department.html

[luqmaan]

Maybe we can use this data to include a column for number of datasets or freshness.

[mateoclarke]

@luqmaan: I like that idea!

As we think about how to flesh out the original card layout, it would be nice to include data like what we have here.

[courtney-rosenthal]

Look at http://data.open-austin.org/data-catalogs/summary-data_austintexas_gov-LATEST.json to get a feel for the sort of information that's available.

Be warned that the category assignment is a bit iffy, and there would need to be some mapping between categories to depts in the scorecard. A BIG POSITIVE to including this in the scorecard is that it would create large positive incentive for the depts/city to provide accurate information here -- and anything that improves the quality of the metadata is very helpful.

If there is some sort of nightly process that can be done to produce summary data in a form you want, let me know and I can add that to the nightly pull-data-portal-catalog process.

[haileypate]

This is pretty cool! @chip-rosenthal, I have the resource IDs mapped to the department values for this report, will upload this week.

The department metadata thing is pretty wonky. We will need a standard, but it's important to think about what questions we want to be able to answer when we query by or click on "Department". Would love to hear from OpenAustin... what could/should department mean?

I've been collecting notes on how department names/codes/acronyms are used across the City... and I'm thinking the budget might be a strong candidate as the foundation to which we map dept values... as best we can.

[haileypate]

this table also brings up some interesting topics for discussion. from what we can tell (socrata analytics are quite limited in this case) -- resource owners are all over the place when it comes to pushing new data to the portal. some append rows, some replace rows, some have completely changed schemas while preserving the original resource id. i could use some help studying this behavior... i have some data and some starting questions to answer, would love to collaborate for sure.

[spatialaustin]

that's an excellent tool, @chip-rosenthal!

this makes me wonder what the next level of progress, beyond the sprint, might look like and be measured against. Like @luqmaan 's line of thinking some things that come to mind are quantity of datasets, frequency of updates, and some kind of assessment of richness or robustness. I guess this really ties into the city's longer term goals for Open Data....are those published anywhere, @haileypate?

[courtney-rosenthal]

@spatialaustin I was thinking a good next step would be to start loading metadata into a database and doing a periodic report to identify: new/updated datasets, regularly updated data feeds (a la 311), updated data feeds that have stopped updating (currently Animal Center Intakes is in this state).

[haileypate]

All, at the City, we are currently preparing ideas to pitch to CoA Web Services leadership for the next episode of the Open Data Initiative. @chip-rosenthal, I agree, it's time to dive a bit deeper into the analytics. I've got a schema drafted that would help me answer some key questions... and I'll pitch it on the project ideas repo in a few. Stuart at Socrata expressed some interest today in collaborating with CoA to scale whatever solution we come up with. Perhaps our solution design could inform their approach... I would love to see some new metadata assets made available as published Socrata resources and accessible by all.

@spatialaustin and all, I think City leadership would be open to hearing ideas on short term and long term goals alike. What thoughts do you have? Any other forums to which we could call out for ideas? It's important that when folks pitch, they explain how their ideas would be valuable and feasible.

[haileypate]

I owe this repo a mapping table of resources to departments for the report. A few things came up, but I haven't forgotten @chip-rosenthal =)