Geocoding is something which we all need but not all MLS's provide Geocoded information for properties in their database, some do some don't. Our initial idea was to run geocoding as part of the import process, when we imported the properties from the MLS feeds we did a quick geocode lookup and updated the property with the latitude and longitude and then updated the property in the GetYourIDX database.
This appeared to work pretty good at first, however we soon found this was not a good idea.
- As the number of properties grew so did the number of Geocode requests we had to perform. Hitting rate limits and failed queries started to become a big factor.
- A properties geocode data will never change fo the life of the property. However, it was a quick performance tradeoff to relookup the data. We traded simplicity for a slight perfomance loss during import but this now.
- You just can't geocode thousands of properties in a very short time and expect every query to come back successful.
I am a big believer of "If things are getting complicated then theres going to be a better way" and sure enough thinking about it there was a much simpler way to Geocode all the properties.
Geocoding is a nicity, its not really 100% required from the time a property is first imported. If geocoded data becomes available an hour after a property is first imported then thats not going to be a problem. We update each property with the changed data rather than import every thing fresh each night so the geocoded information is never overwritten between imports.
So now we dont perform geocoding of properties during the import but instead have split it out into a seperate process and run it periodically to check for properties which require geocoding.
Cron still has its place in society