« AOSD 2005 Conference has started... | Main | Adopting AOP, and AspectJ 5 talks »

March 16, 2005

Grady Booch's keynote at AOSD

Grady Booch has just taken the platform to deliver his keynote, "The Complexity of Programming Models".

The opening question is : "how many lines of code are written each year around the world?". A. about 35 billion lines of source code are written each year. This is based on about 15 million or so software professionals world-wide, of which current estimation is about 30% actually cut code. (The percentage of professionals cutting code has steadily declined :( ). Each developer contributes about 7,000 loc a year. Over history, we have about a trillion lines of code written so far!

The general trend is that things are getting more complex... A given system means different things to many different stakeholders. What "simplicity" is for each stakeholder also varies. Simplicity has different points of view.

In the presence of essential complexity, establishing simplicity in one part of a system requires trading off complexity in another.

The best technology is one that is invisible... the task of the development team is to engineer the illusion of simplicity. In languages we see the tradeoffs in history. Control structures are a tradeoff between primitiveness and convenience. Garbage collection is a tradeoff between expliciteness and abstraction. VB and Smalltalk are tradeoffs between performance of development and performance of execution. Beans, aspects, services are a tradeoff between packaging for design versus packaging for development versus packaging for deployment...

A programming model specifies the semantic universe within which the developer labours and is defined by the languages, platforms, tools, etc. they work with.

For example, think about a web-centric programming model. There are a set of languages commonly used (HTML, CSS, XML, RSS, Java, ...), a variety of platforms (LAMP, J2EE) etc., a set of best practices, and a set of tools. These things combine to make the programming model for web services. And this is not simple... Question for introducing new things like aspects - are we making things simpler, or are we adding complexity. Other programming model examples are game development, high-performance computing, command and control, AI, domain-specific frameworks,... Check out Grady's Handbook of Software Architecture at http://www.booch.com/architecture.

A system is shaped by a myriad of design decisions by different stakeholders that work to balance the forces swirling around the system.

E.g. in civil architecture there are forces of compression and tension. "Any time you depart from established practice, make ten times the effort, ten times the investigation...".

In software some of the forces are: schedule, mission, cost, legal, ethical/moral, dependability, quality, performance, functionality, resilience, production, context, compatibility, resources. These are grouped in categories of business, values, operations, development, and environment.

Software is inherently complex due to complexity of the problem domain, difficulty of managing the dev. process, fluidity of software, and the fundamental challenges of discrete systems.

Software has limits, even if you don't think so. E.g. a space system that needed very accurate synchronized time - but the requirement violate the fundamental laws of relativity.

In the dev. process we face challenges of teams, multiple languages, platforms, tools, etc., scalability (size up, speed up, scale up, scale out http://www.intelligententerprise.com/db_area/archives/1999/991602/scalable.jhtml).

Discrete systems have non-continuous behaviour, combinatorial explosion of states, corruption from unexpected external evennts, lack of mathematical tools and intellectual capacity to model the behaviour of large discrete systems.

We can master essential complexity, but we can never make it go away.

How do we measure complexity? In biological systems we have a variety of techniques (http://www.carleton.ca/~hmasum/complex.html). The Kolmogorov measure says "what's the simplest structure that will produce this behaviour?". One very simple measure - how many words does it take to write down a description of the behaviours? Kolmogorov applied to software: "the relative size of a program capable of generating a given string".

If we don't know how to measure complexity, it is reasonable to suggest that we don't know how to measure simplicity.

Beauty/elegance is one thing we recognise. Elegance means simplicity and less new code, an elegant solution solves the whole problem.

Triggers of complexity include significant interactions, high number of parts and degrees of freedom, nonlinearity, broken symmetry, and nonholonomic constraints (localized transient anarchy). [No, I don't know what nonholonomic constraints are...].

As systems evolve, objects that we once considered complex become the primitive objects upon which more complex systems are built. Hierarchic systems are usually composed of only a few different kinds of subsystems in various combinations and arrangements... Systems can be decomposable (independent parts) or nearly decomposable (most sw components). In biology, the most resilient systems have the loosest coupling. See Simon, The Organisation of Complex Systems. Interesting finding: overall flexibilty increases by having fewer components but greater composability.

A complex system that works is invariably fonud to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work.

Self-organising systems are interesting to study. For example pattern formating in slime molds, flocking of birds, schooling of fish, wall building by ants... All of these have underlying simple rules. Can we build complex systems from simple rules, and let the complex behaviour emerge?

As complexity increases, the increase in functionality and understandability is exponential. We reach a barrier where we just cannot understand the system anymore.

Fundamentals never go out of style.

(or how to mitigate complexity)

Aspects are not a fad - there is something deep and profound about them, but we cannot ignore the fundamentals so always hold them in mind as you do dev/research. Fundamentals are:

• crisp abstractions
• clear separation of concerns
• balanced distribution of responsibilities
• simplicity via common abstractions and mechanisms

Abstraction... all abstractions are context dependent. All non-trivial abstractions are to some degree leaky (Joel on Software). There is no such thing as a perfect abstraction.

Gabriel: Worse is Better. Don't strive for "perfection" good enough is better. Both implementation and interface must be simple, although it is more important for the implementation to be simple.

The entire history of sw enginerering is one of rising levels of abstraction.

are you on that path?

To attack complexity, stick to the fundamentals, relax a constraint, make assumptions. It's better to be simple than to handle every possible case.

Architecture is the sweet spot for aspects... (interesting to see where this bit of the talk is going to go). Mature physical systems have stable architectures which have grown over long periods of time. In software-intensive system we haven't achieved this. Architecting software is different. Kent Beck: "Software architecture is what software architects do". Other definitions: http://www.sei.edu/architecture. Architecture establishes the context for design and implementation. Changing architectural decisions causes significant ripple effects. IEEE 1471-2000 defines software architecture. An architectural style is the classification of a system's architecture according to those with similar patterns. A pattern is a common solution to a common problem. An architecture is the result of weaving multiple views of the system from the perspective of different stakeholders. There are five fundamental views of a software systems (4+1 model): logical, implementation, process, deployment, and use case views. Not all systems require all views.

Logical view focuses on functionality, key abstractions, mechanisms, separation of concerns and distribution of responsiblities. This is where the aspect community is currently focusing. Challenge to the audience: what could aspects do in the other "views"?

Process view focuses on performance, scalability, throughput. Fruitful work for aspects here? Nothing happening yet.

Implementation view focuses on configuration management - the components used to assemble and release the physical system.Deployment view focuses on distribution, communication, provisioning. Use case view describes the behaviour of the system as seen by its end users and other external stakeholders. The logical view and the implementation view are tightly coupled.

Biological systems have crosscutting concerns: reproduction, protein creation, metabolism, cellular proliferation, heme synthesis, heat production,.. (and many more on Grady's slides). Concerns are not isomorphic to structure (there's no piece in the body that is the "temparature regulation" piece). In biological systems, these aspects evolved simultaneously and interdependently at each level of abstraction. What we have today with aspects is "cool stuff", but we can do even better than that when we look across the other architectural views. Some structures and behaviours crosscut components: security, concurrency, caching, persistent.

The role of AOSD: remember the fundamentals - don't make it more complicated. The current sweet spot for aspects involves elements of each of the fundamentals: especially with regard to building crisp abstractions and the separation of concerns for roles relative to packaging. This impacts primarily the interplay of the logical view and the use case view. Q. What are the semantics of an aspect? Tell me something that is an aspect, and something that isn't an aspect, and how we can classify the two.

The current pragmatic focus is upon transformation tools that focus on already visiblie artifacts. Tha harder focus - the most disruptive yet potentially the most valuable- is upon transformation tools that focus on deep semantic representations and then the creation of these traditional artifacts by reflection. Ie. source code as a pretty-printed side-effect, not a central object.

Make some simplifying assumptions by picking a single platform and see what you can do...

Closing thoughts: sw is fundamentally ,wickedly hard and it's not going to get any better. But the world doesn't need more technology - we need less, the best technology is invisible.

Posted by adrian at March 16, 2005 04:49 PM [permalink]

Comments

Post a comment