Make 2011 the Year of the Event
In this, my first blog post of 2011, I’d like to issue a challenge to the blogosphere to make 2011 the year of the event. There was no shortage of discussions about services in the 2000’s, let’s have the same type of focus and advances in event’s in the 2010’s.
How many of your systems are designed to issue event notifications to other systems when information is updated? In my own personal experience, this is not a common pattern. Instead, what I more frequently see is systems that always query a data source (even though it may be an expensive operation) because a change may have occurred, even though 99% of the time, the data hasn’t. Rather than optimizing the system to perform as well as possible for the majority of the requests by caching the information to optimize retrieval, the systems are designed to avoid showing stale data, which can have a significant performance impact when going back to the source(s) is an expensive operation.
With so much focus on web-based systems, many have settled into a request/response type of thinking, and haven’t embraced the nearly real-time world. I call it nearly real-time, because truly real-time is really an edge case. Yes, there are situations where real-time is really needed, but for most things, nearly real-time is good enough. In the request/response world, our thinking tends to be omni-directional. I need data from you, so I ask you for it, and you send me a response. If I don’t initiate the conversation, I hear nothing from you.
This thinking needs to broaden to where a dependency means that information exchanges are initiated in both directions. When the data is updated, an event is published, and dependent systems can choose to perform actions. In this model, a dependent system could keep an optimized copy of the information it needs, and create update processes based upon the receipt of the event. This could save lots of unnecessary communication and improve the performance of the systems.
This isn’t anything new. Scalable business systems in the pre-web days leveraged asynchronous communication extensively. User interface frameworks leveraged event-based communication extensively. It should be commonplace by now to look at a solution and inquire about the services it exposes and uses, but is it commonplace to ask about the events it creates or needs?
Unfortunately, there is still a big hurdle. There is no standard channel for publishing and receiving events. We have enterprise messaging systems, but access to those systems isn’t normally a part of the standard framework for an application. We need something incredibly simple, using tools that are readily available in big enterprise platforms as well as emerging development languages. Why can’t a system simply “follow” another system and tap into the event stream looking for appropriately tagged messages? Yes, there are delivery concerns in many situations, but don’t let a need for guaranteed delivery so overburden the ability to get on the bus that designers just forsake an event-based model completely. I’d much rather see a solution embrace events and do something different like using a Twitter-like system (or even Twitter itself, complete with its availability challenges) for event broadcast and reception, than to continue down the path of unnecessary queries back to a master and nightly jobs that push data around. Let’s make 2011 the year that kick-started the event based movement in our solutions.
[…] This post was mentioned on Twitter by John Januszczak and toddbiske, jlemmon. jlemmon said: Excellent points –> Make 2011 the Year of the Event http://bit.ly/icv97r […]
Todd, almost one year ago to the day (in a similar rush of new year enthusiasm) I made a similar call:
http://www.soabloke.com/2010/01/11/the-year-of-living-asynchronously/
Some progress through 2010…especially in the “non enterprisey” part of the industry. Let’s hope it continues in 2011.
Cheers,
Saul
[…] Todd Biske: Outside the Box » Blog Archive » Make 2011 the Year of the Event […]
[…] I’ve previously asked for this year to be the year of the event, let’s do so in a way that allows these events to feed into our internal activity streams and […]
Hi Todd. Nice post. I’ve thought of Twitter-like “follow” integration before and am sure this pattern will eventually be where most integration goes.
One point however is (and this is an issue I’ve often faced). If one is dealing with situations where it’s essential that every event is noticed by other systems, you can’t get away with not running regular cross-checks to ensure nothing has been missed and data is consistent. How do you see a “follow” pattern dealing with these uncertainties?
[…] Biske has recently written about the challenges of implementing an EDA and mentions the challenge of getting applications to share events. That particular problem will […]