When your bosses or your voters ask you how you’re doing in your job, what do you tell them? Increasingly cities and city leaders have open data and big data to quantify successes. If you’ve fixed more potholes, responded to more city requests and even plowed the streets more efficiently – there’s probably a pretty graphic dashboard you can drop in a PowerPoint to show off how well you’re performing your duties.

But what about the larger issues that supersede the day-to-day quality of life in your community? Equity is the new resilience, which was the new sustainability – at least if you look at conference themes and agendas these days. Creating places that work for everyone is a long-standing, systemic problem. Inequality isn’t hard to measure over time. But it’s challenging to measure in real time, and unless something changes in the 2016 federal budget, that’s about to get worse.

Open data and public data can help cities become better places to live in a number of ways. Having access to this data can lead to more efficient government delivering more efficient services. Pot holes and street lights can get fixed faster, with greater accountability. Crime rates can be tracked and police resources better deployed. Watchdogs can keep an eye on which streets are getting plowed and which aren’t. Data can help find restaurants that need inspections and building growing marijuana illegally. Ideally all of this should lead to lower costs, less waste and higher levels of day-to-day livability in the cities that are taking advantage of these new technologies.

There are two broad issues worth discussing in the context of informed decision-making. First, is a lack of standards. These are new technologies and new ideas. Each city is finding its own way. To this point, there’s very little consistency among what cities measures and how they release it. That makes benchmarking and comparing extremely difficult.

Very little of the data comes tied to demographic profiles, so those have to be overlaid from other sources to answer questions like “are poorer neighborhoods getting their potholes filled as quickly as more well-off neighborhoods?”

For most larger issues, like equity, we need to leave the Internet of Things and turn to well-established and trusted sources of governmental statistical data such as the U.S. Census Bureau.

Here’s where your job is already difficult: Other than the Gini coefficient, there is no be-all-end-all stat to measure equity. You’re left developing your own frankenstats like “the difference in the white unemployment rate and the black unemployment rate.” That kind of analysis requires parsing the population into smaller and smaller groups. By the nature of statisics as geographies and population groups get smaller, margins of error get bigger leading to huge pitfalls as you try to develop meaningful analysis. You can augment this at the local level – do your own research and surveys – but you still need the Census to benchmark.

Here’s where it’s going to get worse, and I apologize in advance for a slight bit of impending nerdiness. In the announced budget for 2016, the Census has proposed eliminating what’s called the 3-year American Community Survey data release. The 3-year data is an annual release based on a rolling 36-month average of measures like employment-by-race-and-ethnicity for cities with populations greater than 20,000. For a mid-sized city, this is really helpful data.

I’ve had some discussions with a frustrated mayor trying to wrangle meaning out of this data about how his administration can measure its success. He’s been looking at the 2013 3-year release and the preceding three-year release to gauge the situation before and after taking office. Staffers in your administration are likely doing the same thing. However if the 3-year set goes away, you’re left comparing the 5-year period (rolling 60 months) of 2009-2013 to the previous 5-year period. At the moment there isn’t even a previous 5-year set because this product isn’t that old yet.

We’ll have two sets to look at soon, but even then you will be looking at two data sets which include periods of massive upheaval in the economy. Each set will include some years of recession, but neither is fully in or out of a recession. It’s hard to draw conclusions about the unemployment rate of Black citizens in your city when it might easily have swung by 15 percentage points during that span. The margins of error decrease but the chance for inherent ambiguity increase.

“Inherent ambiguity” isn’t a phrase that sits well with bosses or voters. Throwing up your hands in a debate or a performance review and saying, “We’ll know if we’ve succeeded in a few years, maybe.” isn’t really going to fly either.

We need better data to drive better decisions. Now, however, we might have to speak up and demand it a little more loudly. Your jobs, your programs and your city’s continued economic strength might depend on it.

Matt Carmichael is the editor of Livability.com. He’s a seasoned journalist who specializes in population trends and urban issues. Previously he was with Crain Communications where he served in a number of roles including the Senior director of research and data at Crain’s Chicago Business and Director of data strategy at Advertising Age. His work on Ad Age’s American Consumer Project was the basis for his first book, Buyographics, published by Ad Age/Palgrave Macmillian. He has been awarded top honors in journalism competitions for content, design and innovative uses of technology. His research has been cited by global media outlets including the Wall Street Journal and The Atlantic.