mySociety developer Struan wrote a great update recently, describing everything we do when we integrate FixMyStreet Pro with a council’s system.
It was really for his colleagues to read, but it seemed like a shame to keep it only for our own consumption, because it’s a super-clear explanation, even for the less technically-inclined. So we thought we’d publish it here, as well. We hope you find it as useful as we did!
One of the benefits of FixMyStreet Pro’s Boulevard and Avenue plans is that, instead of sending emails to the council, we put reports directly into the council’s back end systems.
‘System’ can mean a lot of things, but for FixMyStreet we’re usually working to integrate with some sort of software that manages assets, often run by the council’s highways team. If you hear someone muttering about Confirm, Exor or WDM then we’re talking about one of these systems.
The most basic thing we can do when connecting FixMyStreet up to one of these systems is to take the information that a report-maker enters on FixMyStreet and put it into the equivalent fields in the council’s system. This is largely automating the process of cutting and pasting from an email that the council staff were probably doing before, and while that’s already a win for the council in terms of saving time and resources, we can usually do much better than this.
For example, we can pull a list of report categories out of the council’s system. This means that category changes don’t need to be made in both the backend system and FixMyStreet; they only need to happen in the backend system, and they’ll be automatically reflected on FixMyStreet without anyone having to remember to make the update on that side too.
This isn’t always practical, often because councils have many categories, not all of which they want to expose to the public. In those cases we still have to update the categories at our end. However, even doing this by hand in an integration allows us to add various bits of metadata to categories which are useful to the council. If we’ve automated this then all the metadata is pulled in as part of the automated process.
Depending on the council’s preferences/set-up, this metadata can include extra information to gather for certain categories, e.g. “how large is the pothole?” or information such as streetlight numbers associated with the category. These can help the council prioritise reports and also reduce the need to go back to the user to ask for further information.
The other thing integrations allow us to do, is both send — and more importantly for the council— fetch updates on reports. Not only does this mean that the council only needs to update the problem in their internal system; it enables better reporting on the current problem state.
While a member of the public can only mark a problem as fixed, a greater range of states are available to councils (Investigating, Work Scheduled and so on) which helps make it a bit clearer what’s happening with a report. Councils often have a fairly complicated internal workflow so another step in the integration can be mapping their internal status codes to those displayed on FixMyStreet.
From a public perspective, this also means that updates to a problem get to the council (we don’t email updates on reports; they’ve historically been presented as a way for FixMyStreet users to discuss an issue rather than to chase an issue or note a change in its status). This is also good for the council as it means if a member of the public isn’t happy with their response then they find out.
The gritty technical details
FixMyStreet’s native way to communicate with outside systems is Open311, an open standard for problem reporting.
Unfortunately, most council systems don’t speak Open311, so we need to write code that sends information in whatever format they use. We used to do this by adding code directly into FixMyStreet, which wasn’t too bad as the problem sending code is pretty modular and supports plug-ins for the actual sending process.
Once we started adding more integrations we wanted to move away from this approach, largely to avoid clogging up the main FixMyStreet codebase. As Open Source code, FixMyStreet has been deployed in many countries around the world, and it is unlikely that people running their own sites abroad would need this part of the code, and certainly not the whole range of different systems we were covering.
So now, most of our integrations communicate through a proxy that accepts Open311 requests, transforms them into requests to the council’s backend system and then transforms the responses back into Open311. The proxy is an evolution of the code our ex-colleague Hakim wrote some time ago for a previous integration.
Open311 is a pretty flexible standard, so there’s lots of room to pass custom information about, and that means we can hide quite a lot of the complexity in the proxy and never have to touch FixMyStreet’s code.
For councils where we can’t automatically fetch the list of categories, all the configuration is included in the proxy, making this invisible to FixMyStreet.
As more and more councils come on board, the process of adding integrations to the proxy will become easier. For example, now that we’ve set up a few Confirm integrations, any new clients using Confirm will require less code-writing and it’ll just be a matter of configuring mappings and categories.
And even if we’re integrating with a system we haven’t come across before, we now have a standard pattern of work, meaning that more of the code is in the specific implementation, and less in the set-up. That makes for a quicker, easier implementation all round.
Want to know more? Drop us a line and we’ll happily answer all your questions.
Image: Randall Bruder