Presentation Roadtrip

It's been 3 weeks of presenting for me for two separate events; Microsoft Architect Councils in Christchurch, Melbourne, Canberra, Sydney, Auckland, and a BrightStar Enterprise Architecture conference also held in Auckland. The tripping is now over and it's time I put some background material up onto this blog.

For the BrightStar conference attendees I know you can get the presentation from the BrightStar site and you should have been provided with the link and credentials from Brightstar.

For the Microsoft Architect Council meetings, which were public events, I've got a copy of the presentation loaded on my public SkyDrive folder.

Looking back over the presentations it's now easier for me to see the main themes come through and to recognise which have raised the most interest from attendees.

In random order they appear to be.

1. The recognition of business value in the IT shared service line and the special function of architects to enable change, especially disruptive change in a company; something typically unrecognised in companies as typified by the statement fragment 'the business must make the decision...'.
   
2. The explicit recognition of the conflict of interest between building a product right and building the right product and why this leads you away from a single project manager role in your project delivery towards separate programme and product manager roles.
   
3. The discussion in the presentations about the normal but arguably short sighted pecking order of market facing teams over shared service lines and why this leads to the failure of brands and companies due to disruptive innovations.
   
4. Why companies fail to be creative because they deny their staff the time fully saturate themselves with all the information required to solve a problem, and the time for their subconscious mind to actually figure it out. You can't achieve innovation in a project, projects are defined once you know what you're planning to do.
   
5. The importance of addressing complexity and efforts to do this from modular programming through object orientation and service oriented architecture with lots of references to the work of Roger Sessions and his recent complexity papers.

All in all it was a great experience that benefited me probably a lot more than the attendees so a big thank you goes out to Microsoft and BrightStar for getting me along.

From a recent trip to India

In Mumbai an elephant is stuck in a traffic jam while a young woman wearing dirt covered rags sits on a pile of rubbish and combs her hair, in another world she could be a model. In the far reaches of a rough shop in Agra sits a derelict life size wooden statue of a Hindu god, intricately carved, old and dirty, why isn't it out the front in place of the tacky tourist baubles? On the road to Delhi a camel pulls a cart in the early morning while just off the road men squat and presumably undertake morning ablutions. In Bangalore high technology campuses with green grounds and gates and walls and guards provide paradise away from the daily grime of the city. Contrasts run so great that senses are initially sent awray until a form of acceptance comes in and you now longer lift an eyelid at the sights in front of you. I liked it, a lot.

barcamp agile Wellington and the power of the individual

http://barcamp.org/BarCampAgileWellington

...is underway and the it's proving more thought provoking than I'd assumed. The development process isn't something I'm directly focused on in my everyday work but it's certainly an important area for me. The biggest theme striking me today is the focus on the individual which I find fascinating.

There was a trend for awhile that worked hard to deliver results through process. Methodology x, prescriptive guidance y. The dehumanisation of the worker.

It just doesn't work that way - we're human after all. It really is all about individuals. Teams yeah, sure - but every member of that team is an individual with ideas and beliefs. You either recognise it and work with that or you ignore at your peril.

I love the blogging world today and I love the way guru characters have appeared that lead through the posting of their ideas. They're enormously important at driving company directions and in defining respect and appreciation from those that watch and listen. Companies such as Google, IBM, Microsoft and Sun all have these people, but there are plenty of companies that don't yet. Big companies, large consulting companies and product companies. I have no faith in their ideas, direction, or products not until I see a passionate, articulate voice from someone I trust.

Schneier on Security: More "War on the Unexpected"

One can only hope that the increasing information transparency we see occurring on the Internet and in modern computing technology continues...

Schneier on Security: More "War on the Unexpected"

Wellington Weather...

...is incomprehensible!

Yesterday morning I awoke to howling westerlies and heavy rain in Paraparaumu, then exited the train in Wellington (50km distance) to a warm, dry, windy day with the sun shining!

This morning I awake to incredibly heavy rain, take the train to Wellington and find it cold with a strong damp southerly.

What will it be tomorrow? Heck, what will it be tonight?

Living here you know that you just don't know. Looking on the bright side, at least we have something to talk about.

And as for tonight, best advice I can give is follow the MetVUW site (and keep looking out the window). Based upon the forecast for 7pm, I'm guessing it may have cleared up again!

Heat Mapping

I've recently had the pleasure of uncovering some new data visualisation -visualization for you Americans- capabilities, in particular heat mapping a measure that's a function of two input variables.

Generating eye catching representations of information is crucial in the corporate world, and yet for the most part, business data visualisation is very poor. Back in the 90s I was busy with research for a physics PhD and I lived by gnuplot and a Windows package called Origin. In transitioning into the business world I also transitioned into Excel. I've enjoyed using Excel and I'm a really great fan of Excel 2007 and especially the data mining tools that Microsoft now provide with it but the problem with the graphs, is that everyone else at work generates graphs that look, well, exactly the same. So, I thought I'd go and take a look at what else is now available.

First off I discovered that Gnuplot and Origin still exist and their capabilities have advanced far beyond what I used when I produced a thesis on a 386 linux pc with.. was it 120MB of disk But, then I discovered Python, the scipy (scientific Python) and numpy (numeric python) libraries, and the MatPlotLib library. Now these looked especially useful because of the image mapping capability which could be used to generate a heat map.

Working with a range of people with different backgrounds in a corporate environment you're constantly trying to re-represent information in differing ways to achieve some connection with your audience. For some people an Excel bar chart works great, for others scatter plots, for others they'll want error bars etc. So you want a range of graphing tools available. Now Excel is good, especially Excel 2007, but there's still a range of visualisations that it can't perform, especially image mapping or heat mapping and some 3D plot types.

Now MatPlotLib can help to fill some of the void, particularly with the image mapping. Check the screenshots to see a range of examples of image (heat) maps and polar plots mixed along with more conventional 2D plot types.

In one example I looked at the pattern of customer usage through our Internet channel as a function of customer age and their total relationship with the company. To do this with MatPlotLib I first collected the data from our database environment, I then used Excel to manipulate the data into an array format (same format as an Excel surface plot), then saved it to a file, I then loaded it into a Python array and used the imshow() command to generate a heat map. I regenerated the graph over successive half year intervals keeping the color range set to run over a constant interval, then combined the images together into a movie with Windows Movie Maker (you can also do this with command line tools such as memcoder which part of the opensource mplayer suite).

The output of this exercise was a movie that showed customer activity through the Internet increasing over time across all age ranges but accentuated by the degree of total transactional relationship they had with the company.

One of the biggest problems you face with generating heat maps/image plots is getting a full range of data. If the value you want to plot is a function of x and y, and you're not actually taking measurements, but just relying instead on historical data you'll probably end up with a lot of missing data points. The imshow() command interpolates data points to smooth over the holes.

The easiest way to get MatPlotLib on your pc/laptop is to install the standard Python distribution and then over the top install the IPython shell. IPython provides you with a scipy item in your programs list which gives you a Python command prompt with all the necessary libraries pre-loaded.

If you're interested in other visualisation tools then I've since found a good background reference at IBM: http://www.ibm.com/developerworks/linux/library/l-datavistools/. It's say's it's for Linux but in reality most of this software runs on Windows (and in fact if you look at the numbers of downloads on some of the packages the majority of downloads are for the WinOS).

The Business Shutdown Statement

http://www.andredurand.com/2007/05/10.html#a689

It's not a day goes by I get the very same. As if I'm not thinking about the business!

The Value Of An Employee

Some recent project activity I've been involved with has reminded me of the all too commonly occurring scenario that plays out in corporate offices up and down the land each day; the relegation of the employee to support while the contractors populate the projects. (Disclaimer : I've been a contract resource, an employee, a project manager, and team manager so I've covered pretty much all the bases...)

Now, it never ceases to amaze me that organisations persist in driving project delivery with their contract staff in preference to their corporate staff. The usual argument is to better manage resources and budget for asset development enabling easy capitalisation of time, depreciation of asset cost, and flexible staffing levels with project load, but consider this.

• How will your employees feel if they're forever stuck on support and bug fixing while the glory of new project delivery (and the happy celebrations on project completion) are forever the domain of your contractors?
  
• How will you stop your people believing that your employees are second rate staff to the contractors you bring on for the big projects?
  
• How do you successfully manage operation of the project deliverables when the operational staff were not involved in the project development?
  
• How do you continue long term development and idea innovation on a system produced by one team and operated by another?
  
• And just how many projects actually really do successfully deliver generating the actual value originally envisaged in the business cases? Wouldn't it be better to manage the cost up front knowing you can capitalise after the fact?

If you're building a valuable asset then you should be building a commensurately valuable human structure to continue development over time, not just hand over and dash off to the next engagement.

It's a naive view that sees a complex system developed, deployed and left alone. Again, anything valuable at a point in time needs to change to remain valuable in response to a changing environment. It's simply ridiculous to imagine a complex system can be simply bounded by the original project definition.

Relegating employees to operational activity because of concerns over managing budget is a strategic error for any company.

Richard Rumelt on making choices and Disruptive Innovation

 

Thanks go to the McKinsey Quarterly Journal for this great interview. You'll need to register (it's free) to read the original article.

Now the title of this post isn't exactly what Rumelt wrote but the theme runs strongly through the interview.

While responding to questions on the nature of corporate strategy Rumelt gives the example of the resurgence of Apple (and really Steven Jobs) through the iPod:

"Jobs didn't give me a doorknob-polishing answer. He didn't say, 'We're cutting costs and we're making alliances.' He was waiting until the right moment for that predatory leap, which for him was Pixar and then, in an even bigger way, the iPod. That very predatory approach of leaping through the window of opportunity and staying focused on those big wins - not on maintenance activities - is what distinguishes a real entrepreneurial strategy."

And,

"Enter Jobs. He was perfectly positioned because he was a bit of an insider in the entertainment industry but didn't have any of those asset positions that were being threatened. He didn't need to make a fantastic leap of imagination into the far future. He found a set of ideas that needed to be quickly and decisively acted upon."

Two other good points in the interview are the power of writing down thoughts in sentences over bullet points and the concept of value denial.

Computerworld Article Response

Well I was quoted in Computerworld yesterday(http://computerworld.co.nz/news.nsf/tech/DABFB2798D9A6E50CC257336007EAB04) as a result of statements made during my voice of the customer track presentation on Kiwibank at TechEd 07 in Auckland. During the presentation I made a point that platform infrastructure is an important enabling factor in exploiting future opportunities; and that at Kiwibank we were embarking on a desktop upgrade, initially to XP and at a later date to Vista.

During question time at the end of the presentation I was asked to explain in more detail why we were not going directly from Windows 2000 to Vista. The answer is simply that we need to complete our natural desktop replacement to ensure people get a reasonable performance. This is underway and from my perspective, being mostly interested in application deployment, it is not a great inhibitor to future progress. Windows XP does provide organisations with the ability to deploy the .net 3 components for presentation, workflow management, and communication and while I'm sure that from an infrastructure perspective there are many good reasons to deploy Vista for improved management, I'm satisfied with the operating system for application hosting.

The remainder of the article referred to some of the more interesting ways in which we're trying to take advantage of a range of new technologies within the bank; if I had had time in the presentation I could've shown many more.

New Web

These notes come from Michael Platt's Web 2.0 presentation at TechEd07 Auckland.

Why TCP?
Why HTTP?

Why not UDP and BitTorrent...
This follows on from the SAF06 event and Bill Gates talk. Current web protocols were designed with low bandwidth environments in mind; now there is a high bandwidth environment and the time is right for a disruptive innovation. A new read/write peer to peer protocol could easily supplant HTTP with the PUT/POST/GET verbs.

If we do this then REST could provide a model for a new implementation.

Basic support available in WCF1.0 but it's native in Orcas.

Question could be what REST support exists in Silverlight?

Interesting point raised on process management and the Robotics SDK which contains graphical process designer. In a bidirectional web environment you need process support.

TechEd 2007 Auckland Presentation

Just finished! I can never tell in advance how the presentation will go until it's performed in front of the obvious. It's only then that you can gauge the feed back and know which areas of the presentation were actually important to the audience.

In this particular case I think the following areas stood out:

• Innovation is not the result of projects
• My rant on incubators never succeeding
• The requirement to sell internally your ideas
• The need to think ahead two, three or more years and use that to build your strategy for your technology development

Big thanks go out to Lou Carbone for his keynote, best keynote speech I've seen, and I took a hint and used my time before the presentation to add some pictures - they speak a thousand words.

Main message from Lou's speech was that emotional clues are everywhere; and you're daft if you don't take heed.

Surfacing a new Internet

The message has been clear for months; the internet has been limited through protocols designed in 1994 andwe're now on the verge of a new electronic world. I watched Bill Gates give this message last December at the Strategic Architects Forum in Seattle and it's now becoming very clear what he had in mind.

First the release of the Silverlight beta with vector graphics; then the Silverlight alpha with a downloadable sandbox; and now Surface. I've been telling people in my industry ever since that the world will be dramatically different in 5 years time. I don't think it's been well understood but maybe now it will be different.

When you think of a core Internet application such as Internet Banking you realise that a customer's expectations of what Internet banking is has only been based upon 10 years of experience at most. Five years from now this experience will be very different. Silverlight and Surface will enable whole new styles of user interaction. Interacting with the bank becomes a flick of the wrist!

Now, one last memory from that conference in Seattle: the statement that the Internet is increasingly becoming a mirror of reality. Seems more accurate every day...

Almost Forgotten

A few weeks ago it was ANZAC day in NZ; a day of remembrance originally defined by the failed WW1 landings at Gallipoli, but now extended to encompass those Australians and New Zealanders injured or lost during all the wars since. A day also celebrated in Turkey for the creation of a secular state: for one world a disastrous invasion; for another the rise of a modern state.

Now, I usually find these days disconcerting; there seems too great a yearning for days past, as though something about those historic events made us better people, proved our metal, gave perhaps a reason to celebrate our forefather's heroic achievements. It seems a dangerous thing.

Now that might be true but on this occasion, I think I've found something else that warrents mentioning.

My family background on my father's side is Polish. That side of my family came to NZ as refugees in the 1940s and 1950s via the Soviet Union, Persia, and Palestine. Of 1.5 million Poles deported by Stalin in 1939/1940 to Siberian labour camps, only about 700 made there way to start new lives in NZ. The original arrivals consisted of orphaned kids, followed after the war, thanks to the Red Cross, by those close family members that managed to find their children or siblings. It was by this mechanism that what remained of my polish ancestry came together in NZ.

The impact upon families of the trip from labour camps was severe with many dying along the way including members of my father's family. The men of fighting age went into the army, navy or airforce and variously split into those fighting for the West and those fighting for the Soviets.

I recently came across a web site which attempts to document some of the Polish forces during the war, a force which included members of my family. I know that the internet is unreliable but the site seems to confirm what I heard from my family as I grew up.

The Polish forces were a major part of WW2 and yet also now largely forgotten.

The Soviet controlled Polish army fought through to the battle for Berlin and put a Polish flag temporarily on the Brandenburg gate next to the Soviet flags, albiet temporarily as it was rapidly taken down. They numbered 396,000 through the war with 23000 killed or missing in action.

The British controlled forces numbered 255,000 across airforce, army, and navy with 13,000 killed or missing in action.

This means that ignoring the initial invasion of Poland, and the 15,000 officers believed killed by the Soviets in camps in 1940, Poland still fielded a combined total of approximately 650,000 servicemen through WWII making it the 4th largest allied armed forces.

Comparing the casuality rate with that suffered by the US forces during the war is enlightening: 5% of those fighting in the West were killed or missing; 6% of those fighting in the East were killed or missing. As many Poles fought in WW2 under British and Soviet command as did US Marines but they suffered twice  the casualty rate (http://www.usmm.org/casualty.html). And that's excluding the initial invasion of Poland by Germany and the Soviet Union in 1939.

Yet you'll never hear that mentioned on the History channel!

With Yalta and Stalin's desire to incorporate Poland into the Eastern Bloc the efforts of those 2 armies was effectively sealed. In 1946 the British command demobilized the Polish army. They were not allowed to join the London victory parade and according to an interview on the Polish Solder website they were given identity cards with "Enemy Alien" written on them.

In the case of my family 3 members were mobilized into the army, navy and cadets. All survived and ended up coming to live in NZ, a country that interestingly fought alongisde the Polish 2nd Corps throughout North Africa and Italy.

I think it's about time that the efforts of these forgotten soldiers were better brought to light.

SQL Server 2005 Data Mining Classification Matrix

Finally figured it out, so easy, and yet so poorly described in the documentation; the classification model tab in SQL Server 2005 data mining shows you the accuracy of your model prediction. Figuring out how to use it had me confused for quite some time. If you're as daft as me then perhaps the following might help.

The easiest way to discover how to correctly read the classification matrix is to generate a table from your testing data set (you do split your original data into training and test data sets don't you...!?) with the ID, the actual outcome you're trying to predict, and the predicted value. You can do this in the Mining Model Prediction Tab. It's easier there because the option to save to table in your database is accessible from the little disk icon in the top left corner. You choose the ID, the actual and the predicted value in the bottom half of the screen. The first time I looked at this it was a bit of mystery how to drive that half of the screen, I'm guessing you've mastered that part.

Load up the table that is saved from your mining query into SQL Server's query window and count up the number of entries that are correctly predicted; and the number of actual values for each state of the actual attribute. You'll then see how it maps to the classification table. Basically, the succinct description at the top of the table is correct. Columns correspond to actuals; rows to predicted values. The bit they leave out that would've been useful to me is that the total number of cases of any one actual value you get from summing vertically; the total number of predicted cases of a value you get from summing the row.