Are you also working on pushing standards for data sources, such as a state-level standard? Ideally federal standards?
Maintaining thousands of scrapers for different formats seems like a nightmare, and it won't take long for departments to learn they can slightly tweak the format of their reporting to cause extra work for you.
On the plus side, working with all this data probably makes you all very qualified to advise on developing standards.
Maintaining thousands of scrapers for different formats seems like a nightmare, and it won't take long for departments to learn they can slightly tweak the format of their reporting to cause extra work for you.
On the plus side, working with all this data probably makes you all very qualified to advise on developing standards.