Todd Biske: Outside the Box

Archive for the ‘Development’ Category

Amazon released their HTML5 Kindle reader this week, and I couldn’t keep myself from commenting on all of the talk of people saying/hoping/proclaiming that this was the beginning of the end for apps and Apple’s AppStore.

Hogwash.

I think it’s great that Amazon has released the HTML5 version of the Kindle reader, complete with integration into the Amazon Kindle store. I don’t see Amazon pulling their Kindle app from the app store though, and I think there would probably be a big revolt if they did.

It seems that a lot of the pundits out there think that all of the developers out there are just waiting to jump on a single technology that will support any device, anywhere. For developers that aren’t making any money on their products, that may be the case. I’m willing to bet that the lack of profits has less to do with having multiple code bases to maintain and more to do with the app just not being popular enough.

Sure, any development manager should be concerned about development costs, but developers sure don’t have a good track record of sticking with a single technology. You may get rid of your Objective-C code, replaced by HTML5 and a Java backend, and then all of a sudden the Java backend becomes a Ruby backend, and then a JavaScript/node.js backend, etc. You get my point. On top of that, most developers I know who are really passionate about developing enjoy learning the new technologies, so in reality, having multiple platforms to support may actually help from a job satisfaction standpoint.

But all of this isn’t even my main point. To me, the thing that drives it all is customer experience. When the iPhone first came out, everything was Mobile Web. Apple then backtracked, and came out with the App Store. I don’t know about you, but I can’t think of a single App that had a mobile web or iPhone-optimized web experience that was on par with the native apps that were created. Granted, part of that was due to Edge connectivity, but the real thing is that it was all about my experience as a customer. While HTML5 is very powerful, I still don’t believe that it is going to be able to provide the same level of experience that a native application can. Yes, it can work offline and utilize local storage to make it as app-like as possible, but it’s still based on an approach that’s more about a connected, browser-server paradigm.

There will be classes of applications for which HTML5 will be just fine. I’m willing to bet many of them will be replacements for applications that are already free in the app stores. That’s an optimization for the development team, since revenues are clearly coming from another source, whether it be advertising or eCommerce. For paid applications, though, customers are paying for the experience, and if the experience isn’t as optimized as possible, there are way too many alternatives out there.

All we need to do is look back at history to know that Apps are here to stay. Java did not result in companies dropping proprietary development languages for Windows, Mac, or Linux platforms. Yes, some cross-platform products did arrive and continue to thrive, but there’s still a thriving marketplace for native applications on the major desktop platforms. Will we see many mobile applications solely available via HTML5? Absolutely, but the native app store for iOS and Android will continue to thrive.

In this, my first blog post of 2011, I’d like to issue a challenge to the blogosphere to make 2011 the year of the event. There was no shortage of discussions about services in the 2000’s, let’s have the same type of focus and advances in event’s in the 2010’s.

How many of your systems are designed to issue event notifications to other systems when information is updated? In my own personal experience, this is not a common pattern. Instead, what I more frequently see is systems that always query a data source (even though it may be an expensive operation) because a change may have occurred, even though 99% of the time, the data hasn’t. Rather than optimizing the system to perform as well as possible for the majority of the requests by caching the information to optimize retrieval, the systems are designed to avoid showing stale data, which can have a significant performance impact when going back to the source(s) is an expensive operation.

With so much focus on web-based systems, many have settled into a request/response type of thinking, and haven’t embraced the nearly real-time world. I call it nearly real-time, because truly real-time is really an edge case. Yes, there are situations where real-time is really needed, but for most things, nearly real-time is good enough. In the request/response world, our thinking tends to be omni-directional. I need data from you, so I ask you for it, and you send me a response. If I don’t initiate the conversation, I hear nothing from you.

This thinking needs to broaden to where a dependency means that information exchanges are initiated in both directions. When the data is updated, an event is published, and dependent systems can choose to perform actions. In this model, a dependent system could keep an optimized copy of the information it needs, and create update processes based upon the receipt of the event. This could save lots of unnecessary communication and improve the performance of the systems.

This isn’t anything new. Scalable business systems in the pre-web days leveraged asynchronous communication extensively. User interface frameworks leveraged event-based communication extensively. It should be commonplace by now to look at a solution and inquire about the services it exposes and uses, but is it commonplace to ask about the events it creates or needs?

Unfortunately, there is still a big hurdle. There is no standard channel for publishing and receiving events. We have enterprise messaging systems, but access to those systems isn’t normally a part of the standard framework for an application. We need something incredibly simple, using tools that are readily available in big enterprise platforms as well as emerging development languages. Why can’t a system simply “follow” another system and tap into the event stream looking for appropriately tagged messages? Yes, there are delivery concerns in many situations, but don’t let a need for guaranteed delivery so overburden the ability to get on the bus that designers just forsake an event-based model completely. I’d much rather see a solution embrace events and do something different like using a Twitter-like system (or even Twitter itself, complete with its availability challenges) for event broadcast and reception, than to continue down the path of unnecessary queries back to a master and nightly jobs that push data around. Let’s make 2011 the year that kick-started the event based movement in our solutions.

I’ve been going through the TOGAF 9 documentation, and in the Application Software section of the Technical Reference Model, there are two categories that are recognized, Business Applications and Infrastructure Applications. They define these two as follows:

Business applications … implement business processes for a particular enterprise or vertical industry. The internal structure of business applications relates closely to the specific application software configuration selected by an organization.
Infrastructure applications … provide general purpose business functionality, based on infrastructure services.

There’s a lot more to the descriptions than this, but what jumped out at me was the typical black and white breakdown of infrastructure and “not” infrastructure. Normally, it’s application and infrastructure, but since TOGAF uses the term infrastructure application, that obviously won’t work, but you get the point. What I’ve found at the organizations I’ve worked with is that there’s always a desire to draw a black and white line between the world of infrastructure and the application world. In reality, it’s not that easy to draw such a line, because it’s an ever-changing continuum. It’s far easier to see from the infrastructure side, where infrastructure used to mean physical devices, but now clearly involves software solutions ranging from application servers to, as TOGAF 9 correctly calls out in their description of infrastructure applications, commercial of the shelf products.

The biggest challenge in the whole infrastructure/application continuum is knowing when to shift your thinking from coding to configuration. As things become more commoditized and more like infrastructure, your thinking has to shift to that of configuration. If you continue with a coding and customization mentality, you’re likely investing significant resources into an area without much potential for payback. There are parallels between this thinking and the cloud computing and software as a service movements. You should use this thinking when making decisions on where to leverage these technologies and techniques. If you haven’t changed your thinking from coding to configuration, it’s unlikely that you’re going to be able to effectively evaluate SaaS or cloud providers. When things are offered as a service, your interactions with them are going to be a configuration activity based upon the interfaces exposed, and it’s very unlikely that any interface will have as much flexibility as a programming language. If you make good decisions on where things should be configured rather than coded, you’ll be in good shape.

Surekha Durvasula had a very good post entitled ‘Why you need a stated “service versioning policy.”‘ In it, she presented 6 different scenarios where multiple versions of services may be required. If you don’t think you’ll need to deal with versioning, perhaps you should review these scenarios and determine how you’ll handle them when the need arises.

I received two questions in my email regarding my previous post on Service Lifecycle Management, specifically:

• Who within an organization would be a service manager?
• To whom would the service manager market services?

These are both excellent questions, and really hit at the heart of the culture change. If you look at the typical IT organization today, there may not be anyone that actually plays the role of a service manager. At its core, the service manager is a relationship manager- managing the interactions with all of the service consumers. What makes this interesting is when you think about exposing services externally. Using the concept of relationship management, it is very unlikely that the service manager would be someone from IT, rather, it’s probably someone from a business unit that “owns” the relationship with partners. IT is certainly involved, and it’s likely that technical details of the service interaction are left to the IT staff of each company, but the overall relationship is owned by the business. So, if we only consider internal services, does the natural tendency to keep service management within IT make sense? This approach has certain risks associated with it, because now IT is left to figure out the right direction through potentially competing requirements from multiple consumers, all the while having the respective business units breathing down their neck saying, “Where’s our solution?” At the same time, it’s also very unlikely that business is structured in such a way to support internal service management. Many people would say that IT is often better positioned to see the cross-cutting concerns of many of these elements. So, there are really two answers to the question. The first answer is someone. Not having a service owner is even more problematic than choosing someone from either IT or the business who may have a very difficult task ahead of them. The second answer is that the right person is going to vary by organization. I would expect that organizations whose SOA efforts are very IT driven, which I suspect is the vast lot of them, would pick someone within IT to be the service manager. I would expect that person to have an analyst and project management background, rather than a technical background. After all, this person needs to manage the consumer relationship and understand their requirements, but they also must plan the release schedule for service development. For organizations whose SOA efforts are driven jointly with the business, having a service manager within a business organization will probably make more sense, depending on the organizational structure. Also, don’t forget about the business of IT. There will be a class of services, typically in the infrastructure domains, such as authentication and authorization services, that will probably always be managed out of IT.

On question number two, I’m going to take a different approach to my answer. Clearly, I could just say, “Potential service consumers, of course” and provide no help at all. Why is that no help? Because we don’t know who represents those service consumers. Jumping on a common theme in this blog, most organizations are very project-driven, not service or product driven. When looking for potential service consumers, if everything is project driven, those consumers that don’t exist in the form of a project can’t be found! I don’t have a background in marketing, but I have to believe that there are probably some techniques from general product marketing that can applied within the halls of the business to properly identify the appropriate segment for a service. The real point that needs to be made is that a service manager can not take the field of dreams approach of simply building it, putting some information into the repository, and then hoping consumers find it. They have to hit the pavement and go talk to people. Talk to other IT managers whom you know use the same underlying data that your service does. Talk to your buddies at the lunch table. Build your network and get the word out. At a minimum, when a service is first identified, send a blast out to current project managers and their associated tech leads, as well as those that are in the project approval pipeline. This will at least generate some just-in-time consumers. While this may not yield the best service, it’s a start. Once some higher level analysts efforts have taken place to segment the business into business domains, then the right marketing targets may be more clearly understood.

While I don’t normally comment on press releases that I occasionally receive in email, one tidbit in a release from Active Endpoints about ActiveVOS™ 5.0 caught my eye:

Active Endpoints, Inc. (www.activevos.com), inventor of visual orchestration systems (VOS), today announced the general availability of ActiveVOS™ 5.0. …

Scenario testing and remote debugging. ActiveVOS 5.0 fundamentally and completely solves a major pain experienced by all developers: the question of how to adequately test loosely-coupled, message-based applications. ActiveVOS 5.0 includes a new BUnit (or “BPEL unit test”) function, which allows developers to simulate the entire orchestration offline, including the ability to insert sample data into the application. A BUnit can be created by simply recording a simulation in the ActiveVOS 5.0 Designer. Multiple BUnits can be combined into BSuites, or collections of smaller simulations, to build up entire test suites. Once deployed into a production environment, ActiveVOS 5.0 delivers precisely the same experience for testing and debugging a production orchestration as it does for an application in development. Remote debugging includes the ability to inspect and/or alter message input and output, dynamically change endpoint references and alter people assignments in the application.

Back in November, in my post titled Test Driven Model Development, I lamented the fact that when a new development paradigm comes along, like the graphical environments common in BPM tooling, we run the risk of taking one or more steps backward in our SDLC technologies. I used the example of test-driven development as an example. As a result, I’m very happy to see a vendor in this space emphasizing this capability in their product. While it may not make a big difference in the business solutions out there, things like this can go a long way in getting some of the hard-core Java programmers to actually give some of these model-driven tools a shot.

I’ve recently been trying to help out with an issue which has required me to roll up my sleeves a bit, and unfortunately it has brought pack all too familiar memories. Our computer systems (and often the documentation that goes along with it) simply makes things way too difficult. In my opinion, the thing that I’m trying to do should be a relatively straightforward function, but as I’ve dug into it, I just seem to run into an endless set of configuration parameters that need to be specified. Now, while I’m pretty far removed from my days as a developer, I still consider myself tech savvy and a very quick learner. I can’t help but think what the average joe must go through to try to make these things work. It’s almost as if these systems were written to ensure a marketplace for systems integrators and other consultants.

This all comes back to usability and human-computer interaction, areas that have always been a passion of mine. If your products aren’t usable, it’s simply going to create frustration and distrust. If your documentation or support channels are equally poor, the situation will continue to go down. What’s even worse is when there isn’t a workaround, and the only option left to a user is to find an expert who can help. As a user, I don’t like to be painted into a corner where I have no options. We need to keep these things in mind when we build our own systems for our business partners. If we design systems that aren’t usable and require that the business come find someone in IT every time they try to do certain options, that’s a recipe for disaster. If you don’t have a usability team in your organization, I strongly urge you to find some experts and start building one.

I received a couple comments on my previous post, and rather than respond in the comments themselves, I thought I’d use another blog post to address them since I don’t think many people subscribe to the comments feed. Before I get into them, I also wanted to call out this blog post from Anne Thomas Manes of the Burton Group. Out of a discussion on the Yahoo SOA Group about the relationship between 3-tier and SOA, she posted this entry which included this statement:

I also expect that the concept of “application” is likely to go away. Why is it that we as systems users have to be constrained by the limits of this artificial boundary called an application? Why do I have to shift my focus among multiple applications? Why do I have to copy data from one application to another? Why do I have to launch this stupid browser or that stupid application to get my work done? Why isn’t everything just accessible to me from my desktop (via widgets and gadgets) or from within my preferred operating context (e.g., email)?

It was this that prompted me to put together my original post, since I couldn’t find an entry in my blog that was specifically dedicated to this topic, although I had mentioned it as part of other entries. I had meant to include a link to Anne’s post in my first entry and forgot.

The first comment came from Brian “Bex” Huff who stated:

People understand the term “application,” but the word “solution” is a bit too nebulous.
Applications stand alone… what good is a widget to view an Excel doc, without the Excel application to create the doc in the first place?
I agree that IT should always *think* in terms of dynamic, evolving “solutions”… but the basic building blocks still include “applications”… as well as toolkits, frameworks, libraries, etc.

Bex actually made a statement that is indicative of the current culture, which was “Applications stand alone.” My opinion is that applications shouldn’t stand alone. Why shouldn’t we have the ability to present a UI component that can manipulate spreadsheets anywhere? Yes, there will always be cases where spreadsheet manipulation is the only thing we want to do, but there’s also plenty of cases where embedded spreadsheet manipulation would be better for users. What will enable this is thinking in terms of capabilities, rather than in terms of applications. My opinion is that an application is the result of packaging, rather than a unit of composition.

The second comment came from Rob Eamon. He wrote:

There will always be a collection of components/services that have been designed to work together to provide some set of functionality. And there will always be multiple sets of these. And there will always be a need for these sets to interact, possibly through mediation.
Disaggregating application components and making them independently accessible in many contexts seems very appealing.
But take the issues that the typical application development/support team faces and blow that up to the enterprise level. The fragile nature of such an approach will inherently stop it from becoming a reality.
IMO, the application is still a useful and reasonable building block and allows us to break down the enterprise solution space into manageable chunks.
Some of the application components might be useful as independent entities in a larger setting, but I donít think approaching everything that way is a wise course. IMO, it would inhibit flexibility rather than promote (counter-intuitive such that may be).

I wish Rob would have went into more detail on the “issues” that the typical application development/support team faces, because I can only guess what they may be. My first guess is that application teams currently have to deal with ever changing requirements and needs. If you increase the number of parts, you now have smaller components with new relationships associated with their use, and if we don’t manage it well, chaos will ensue. It should be noted that I’ve never said that this type of shift would be easy, and if anything I’d say it’s going to incredibly difficult. I’ve reflected on this in the past, specifically in this post, and wonder if it will take some company approach IT like this from the beginning, without any baggage to create the impetus to change.

Rob went on to state that in his opinion, the application is still a useful and reasonable building block. Here’s where I disagree, for the same reasons I used in Bex’s comments. An application is typically a “packaging” exercise, and those packages aren’t what we want for composition. The only part of an application that still has significant potential for being a “stand-alone” entity is the UI. I’d be happy to see an IT organization that makes an organizational/funding separation between development of UI code from development of services that are used by the presentation tier, much as Jeff Schneider suggested in this post from early last year.

Where I’ll agree with Rob is that this is not a change for the weak of heart. If a new CIO came in and reorganized the organization along these lines, the chaos that might ensue could certainly result in the perception that IT is even less agile than they were before. So, this isn’t a change that will occur in a year. This is a gradual, evolutionary change that will take time, but it will only happen if we’re committed to making it happen. I think a key to that is to get away from the application mindset.

Thanks to Rob & Bex for the great comments, and I’d love to hear from others whether you agree or disagree with my opinions.

Normally I’m not the guy to stir up a lot of controversy, but there’s one topic that I’ve mentioned on mailing lists, here, etc. that usually attracts a few opposing comments. That topic is the use of the term application in the enterprise. I’d like to see that term go away, or at least be limited to just the presentation layer of a solution. Why is that? It’s because it’s the boat anchor of IT, in my opinion. Think about it. How do we get out of the habit of thinking about monolithic point solutions for one small user group and into the habit of thinking about business capabilities that have potential for broader applicability when we have one hundred-plus organizations with the title “application development?” How do we deal with the organizational concerns when applications that were built for one group with one intent are now suddenly critical components for many other solutions with different user bases, each with their own set of priorities?

If nothing else, a switch from using the term “application” to the term “solution” would be a symbolic gesture that represents the change that must occur within IT. IT shouldn’t be producing applications, we should be producing solutions. Those solutions may still be self-contained like yesterday, or they may be a composite of pieces from many previous solutions, some new development, some off the shelf solutions, and some Internet-hosted offerings. The boundary of what constitutes “one application” are all but impossible to draw because of the many necessary interdependencies. Even if we limit the use of the term to just the presentation layer, it’s still debatable whether we should be using it. For example, are widgets on the OS X Dashboard applications? We don’t call them applications, we call them widgets. There’s some pretty sophisticated widgets, though. But wait a minute, won’t Word and Excel be around forever as applications? Well, the latest version of OS X introduced Quick Look, where with a press of the space bar, I can see the contents of an Excel spreadsheet without launching “an application.” The ability to present information in a tabular format is merely a capability. So, even in the sacred ground of the desktop, there are arguments that this notion of application launching may become less important to where the right thing simply happens.

I’m not a fool, and I certainly don’t think the use of the term application is going to go away in 2008, or anytime soon, but I do think that to continue the innovative use of technology, we need to make sure that the terms of the past aren’t locking us in to a single way of thinking about how computers are used. The continued need for better integration and more context-awareness in our technology solutions will only increase, and that pressure will continue to challenge any organization that is stuck with the application-centric view of technology as their boat anchor.

I attended a session on assessing maturity in IT given by Susan Landry. She outlined their maturity model which covers eight different dimensions. The maturity model has familiar levels if you’ve looked at my work, CMMI, or COBIT. Level 1 is ad hoc, level 2 is repeatable, level 3 is defined, level 4 is quantitatively managed, and level 5 is optimizing. The eight assessment areas are:

1. Application Portfolio Management (APM)
2. Project Portfolio Management (PPM)
3. Staffing, Skills, and Sourcing
4. Financial Analysis and Budgets
5. Vendor Management
6. Management of Architecture
7. Software Process
8. Operations and Support

The most interesting part of the discussion, however, was a single slide that discussed the interdependencies between these areas. For each pair of areas, the relationship was classified as either highly interdependent or moderately interdependent. Having done a multi-dimensional maturity model before, a big challenge is in determining whether or not it makes sense to get scored high in one dimension or low in another. In my SOA maturity model, I typically found that when scores were two or more levels apart, it was probably going to cause some imbalance in the organization. If the scores were even or a single level apart, it was probably workable. What I didn’t do, however, was to explore those inter-relationships and see if that theory uniformly applied. While Gartner didn’t provide a lot of depth in the inter-relationships, they did at least make a statement regarding it which can provide a lot of assistance in determining how to interpret the scoring and what actions to take.

In this session, Frank Kenney and Val Sribar, are discussing agility and application development in the context of a fictitious fable of a web unit in a business organization that is operating in an agile manner and how it works with the traditional application development organization. While I find the fable a bit far fetched, since I’ve yet to see a business unit pushing agile processes on IT, it’s usually the opposite, there is one key point that they made. They called out that not all business units may be ready for an agile approach, simply because they don’t change that rapidly. Hopefully, this will be a common theme throughout the conference. Any approach that claims to be one size fits all is probably going to run into problems. Whether it is agile, REST, Web Services, SOA, EDA, etc., there are always scenarios where it works well, and scenarios where it doesn’t work so well. A key to success is knowing not only how to apply a particular technique, but knowing when to apply it.

Further in on the presentation, they went through some different areas and discussed how they contribute to agility. In the subject of business process actions, they called out that there should be a single business person clearly in charge of a particular process. I think this applies not only for processes, but for services as well. There needs to be a service manager for each service in the organization. An individual may manage multiple services, but without a service manager, there will be problems.

On the subject of reuse, Frank Kenney pointed out that incentives, metrics, testing, methodology, and inventory management (registry/repository) is necessary to make it a reality. I think this is a good list, although I would have liked to see marketing included. While a registry/repository and methodology do focus on the consumption side, if a service or reusable asset is placed out there in a way that makes it difficult to find, the burden is on the provider to improve it, which involves marketing.

Near the end of the presentation, Val and Frank presented an “Application Group of the Future.” Val called out breaking out groups for User Experience, Business Processes, Business Information, an Integration Competency Center, Business Relationships, Software Quality, and Information Infrastructure. These groups would all be operating in an agile manner. In addition, the traditional development groups would also exist. These groups would all leverage common environments to support their efforts. There was one thing about this picture that I disagreed with, which was the placement of SOA. They sprinkled the entire organization with SOA experience, which makes sense, but they didn’t call out the separation between organizations focused on service consumption and organizations focused on service production. If anything, service production (unless it happened to be an automated business process) was buried in the tradition development group, and I don’t think that will cut it. On the bright side, one of the things that they called out which was very interesting, was the need for dynamic budgeting. Val made a comment that the web and mobile applications are challenging the way we book business early in the presentation. This need for dynamic budgeting is an analogous example to how a new agile IT, challenges the way we traditionally do budgeting. Annual budgets won’t cut it, we’ll need to make it more dynamic and responsive in real-time.