Thursday, December 07, 2006

SAF 2006 - A Retrospective View

It's a few days since the event now, the small details are beginning to fade away, and I can look back now and think about the overriding themes. Of these, one strikes me more than any other: a statement from Garry Flake which went roughly like "the Internet is an increasingly complete reflection of reality".

Garry's presentation ("How I learned to stop worrying and love the imminent Internet singularity") took a look ahead at the future. He picked up on an idea published in 1993 by Vernor Vinge: the idea that the exponential speed of technological improvements will produce super-human capabilities, making the future unknowable, an idea that I think harks back to Arther C Clarke and the concept that any sufficiently advanced technology constitutes magic. Vernor coined this occurence a singularity because we have no knowledge of the future beyond that point. This is an interesting thought, especially with respect to the Internet. Garry highlighted (as had Charles Fitzgerald in his earlier presentation, "Software and Services") that information technology is most strongly advancing in two areas: memory and networking. Both of these are improving well ahead of the rate of improvement of processing power. (But as for other areas of technology, I personally doubt we're making such significant progress. Don't you think that living from 1900 to 1970 would have been just as remarkable as living from 1940 to the present?)

With a near future characterised by a massive increase in easily accessible storage and network bandwidth it shouldn't be surprising that more and more information about our daily lives and the world around us will be stored and catalogued for later use. The stored information will increasingly reflect the detail of the real world and the Internet will increasingly become a reflection of reality.

To me this is a wonderful concept because it means that we have a representation of reality that's accessible, can be queried, and is an enabler to learn about our world and improve it. To business it should also be wonderful because it opens up opportunities in many more ways than are possible now: mining, characterising, segmenting, predicting, combining, and in general just responding better to customer needs.

The reflected electronic world of information and the services that operate on that information will also provide value from the integration across organisation boundaries. If you can spot the opportunity that links a group of disparate services together you could be clipping the ticket to your first billion. This is the basis of the mashup and we're sure to see making unexpected outcomes from the smart combination of diverse services and information.

The feflected reality was not the only nugget to come out from the SAF, there were also several other ideas that came through the presentations and roundtable workshops I attended.

Bill Gates highlighted the limitations and short life ahead of the existing web browser model; Ray Lane emphasized the uneconomic application development environment present in the US and the likely impact of globalisation; Norm Judah gave a great overview of how Microsoft manage their own internal services; Garry Flake presented on the Innovators Dilemma, and there was a lot of discussion across all presentations on the long tail effect.

Bill Gates' brief discussion of the future of the browser came up during the Q&A session at the end of the SAF. He made the point that the web browser was designed in the 1990s to fulfill a need to deliver an interface to a system over a limited bandwidth link. That environment is not true now and we know it will be increasingly invalid going forward. Network bandwidth is increasing rapidily so you can envisage a future where you connect to a system on the Internet and a much more functional application interface is downloaded and run on your device using some sort of application hosting environment (perhaps based on a virtual machine created for the session, or a sandboxed machine such as you get with .NET and Java). The standards for this form of interactivity have a long way to go but the direction is clear. I wonder how long before this will impact the next generation of Internet sites?

Ray Lane's presentation of the economic problems of the application development market highlighted the large number of existing small development companies, the available venture funding, the commoditization occuring in the development process and as a result the market contraction we can expect (up to 70% of existing companies going). However, Ray also highlighted the new opportunities created by the long tail and networking effects hence the name of the presentation, "The Personal Enterprise".

Norm Judah's presentation of "The Last Architectural Mile" showed us how Microsoft run their own services. This eye opener showed them using their own toolsets to manage their data centres with millions of servers. The use of the Business Scorecard Manager to present their current status was great as was the simple idea that success of failure occurs in the last mile. Delivering into production is so important and yet, while complex IT systems are designed with great thought and effort, the human processes that operate those systems are often not.

Garry Flake's singularity presentation also introduced the innovator's dilemma (first coined by Clayton M. Christensen and described in his 1997 book The Innovator's Dilemma), that innovation comes from the bottom not the top. As companies become established new start ups innovate and then take over the entranched company's market. Companies that survive show an ability to sustain innovation by being prepared to make significant change (or destroy themselves as Garry put it). I wonder to what extent you can say that we've seen this with the likes of IBM and Microsoft? Certainly IBM have moved a long way from the days when the mainframe was their core deliverable. One thing that does mark both companies is their massive investment in R&D, billions of dollars per year. Garry is an example of the impact of Microsoft Labs, his team form Live Labs, a collaboration with MS Research that is developing the functionality you can now get now through www.live.com.

The concept of an increasingly accurate reality represented on the Internet was an overriding theme throughout the event. The impacts are diverse; the wealth of information, the potential outcomes from better analysis, providing richer choice, responding better to people, clustering, trending, making decisions at the edge, networked opportunities, the long tail. I've tried to put some order to these ideas in the bullet pointed list below and in the posts under this you'll see the notes I recorded at the time of the presentations. Some of these notes I posted on the Internet as soon as I'd finished writing them in the lecture rooms, and others I posted over the last couple of days from the notes I'd taken on my laptop. Hope it's helpful.

Change Drivers: Opportunity Creation from Changing Technology

  • Storage and networking improving much faster than local cpu
  • Parallel processing increasingly applied to overcome limitations in cpu speed increases
  • This is opening up new opportunities due to network effects
    • The long tail
      • The affect of the many in the tail can be greater than the few in the head
    • The internet is increasingly mirroring the physical world

The Internet as a Mirror of Reality

  • Responding to people
  • Understanding people
  • How?
    • You can't manage what you can't measure
  • Analytics
    • Application to numerous areas
      • Process improvement
      • Understanding and responding to human behaviours
      • Processing of different data types
        • Text
        • Numeric
        • Image data
        • Sound
      • The internet increasingly mirrors real life - the data on the net is becoming an increasingly complete representation of real life
        • Brute force statistical analysis
          • Trending
          • Clustering
  • Personal choice
    • Make options available
    • Natural selection and evolution
  • Beware human faults
    • The Innovators Dilemma
      • Look down as well as up
  • Take advantage of the masses - the long tail
    • Opportunities exist due to the accessibility of the many
  • Decision making at the edge
    • Responding to your customers at the edge of your organisation versus centralised product development
    • Responding at the edge is equivalent to greater customer focus
    • Create networked opportunities
      • Connect individual to individual
    • Great local example - Seattle traffic reporting via the internet

SAF06 - The Personal Enterprise

Ray Lane

 

Commoditisation

  • Cost of software reducing
  • Costs for software 10% of current
  • 70% of companies will go
  • $4.7 billion going in as venture capital in US market
  • Nicolas Carr – IT doesn’t matter...

Globalisation

  • US market leadership lost
  • Existing model uneconomic
    • Too many competitors within a limited region

Market Demand

  • SMB market growth strong
  • Enterprise market limited

Future

  • Leverage community power
    • Mash ups etc
  • Enabling user generated content
  • Apps from services
  • Personal history
    • Activity
    • Alive
  • Context
  • Relationship status and value and history

Seven Laws for Success in the Personal Enterprise

  • Serves individual need
  • Viral adoption
  • Contextual and personal
  • No train
  • Delivers instantaneous value
  • Utilises social networks
  • No IT

Examples

  • WebEx
  • Skype
  • IM
  • SuiteTwo
  • Google desktop
  • SFDC
  • Monster

SAF06 - High Performance Computing

Ryan Waite, Group Program Manager for High Performance Computing

Background

  • 30 people in the group developing HPC
  • 10GFlops $30m 1991
  • 10GFlops $4000 2006
  • Departmental cluster growth higher than top end growth
  • Product development fundamental to Microsoft
    • Continued Office product development needs to demonstrate new functions and value
    • HPC seen as a possible avenue to do this in the future
  • Financial Services seen as a key sector

Usage

  • Weather Research and Forecast model
    • Community project
    • 3000 users
    • 360k lines only 750 changes to port to windows
  • Excel Services
    • Store spreadsheets on sharepoint
    • Run on the server
    • Run excel services on Compute Cluster
  • Monte Carlo
  • Risk
  • Speedo fabric design
    • Weave and affect on fluid dynamics
  • Procter and Gamble
    • Plastic bottle design
    • Packaging analysis
  • MRI scan of heart and blood flow
    • Fluid dynamics model
    • What if analysis of surgical changes

Compute Cluster Cost

  • Cheaper than standard windows 2003 (~30% less)
  • Can’t run office servers on it

Grid Computing

  • Definition not clear
  • Microsoft refer to
    • Compute grid
    • Data grid

Dataset Sizes are going to be MASSIVE

  • New Sky Telescope 2015
    • 80TB data in a scan on one night
    • Rescan next night
    • Identify what changes
    • Repeat!

SAF06 - Software and Services

Charles Fitzgerald, GM Platform Strategy

Change Drivers

  • Broadband
  • Services
  • Commoditisation
  • Complexity
  • Consumer focus

Challenge

  • To find the value
  • It’s never going to be easy because of competition

Services Infrastructure

  • 20 million on a slow IM day
  • Infrastructure construction dramatic
  • $200 million on data centre capacity
  • Next to hydroelectric generation facilities
  • Millions of servers
  • Concrete, copper, steel, electricity

The Tyranny of Or, the Power of And

  • In the past we've had to make a lot of choices
    • This technology or that
    • This way of delivering or that
    • This company's services or that company's
  • Today we have the ability to combine services more easily
    • Mashups
    • Partnerships
    • Give the customer choice, offer both options

Broadband Inflexion Point

  • Web browser designed for limited bandwidth
  • Broadband beyond current web 1.0

Saturday, December 02, 2006

SAF06 - Dynamics

Satya Nadella

The Last Mile of Productivity

  • Connecting people to process

Microsoft have 30 CRM instances deployed across different areas

Dynamics Investments

  • Role based experiences - Office
  • Collaboration - Sharepoint
  • Contextual intelligence - SQL Server
  • Services - Visual Studio
  • Live enabled - Windows Live

Dynamics architectural tenants

  • Composable role based experience
  • Model driven
  • SOA deployment

Demonstrations

  • Dynamics CRM v3c
    • Lead management gadget for Vista sidebar - hooks into CRM web services
    • Relationship mapping with images on each member of the relationship
    • 3D home design and layout
  • Exception management demonstration
    • Exception notifications go into sidebar via a gadget
    • Click on the notification and a sharepoint site comes up with exception problem identified (in this case an order problem)
    • Drills down onto order details
    • Combines it with mapping from virtual earth
      • Order origination and delivery points identified
      • Where the order is now
      • Collaboration with the truck driver via a wireless device
      • Messenger used - hopefully not while driving
    • Sharepoint search used to identify missing customs document required for shipping
  • Business Intelligence
    • Sharepoint dashboard
    • Mapping
    • Tabular business score card from Performance Point Manager with traffic lights
    • Opening detail data with Excel Analysis Services
    • Another dashboard with more detail
      • More detailed mapping
      • Pie charts and graphs
      • Drill down with ProClarity
        • Performance map
        • Size of box shows size of sales
        • Colour shows margin a la heat map
        • Drill down into geographic areas
  • Dynamics Live CRM
    • Hosted on the Live site a la SalesForce.com
    • Dashboard
      • Charts
      • Scorecards
      • Drilldown
    • Outlook style interface for activities, calendar, queued items
    • Customer data
    • Collaboration tools
    • Workspace creation for working with partners
      • Collaborator uploads documents rather than emailing
    • Marketing campaign creation and management
      • Internet ads
        • MSN ad centre
        • Choose regions and target pages
        • Performance tracking
      • Email

Model Driven Development

  • Research derived customer model
  • Workflow driven
  • Role specific tools

Friday, December 01, 2006

SAF06 - How I learned to stop worrying and love the imminent internet singularity...

Garry Flake

Microsoft Live labs

Live Labs is a partnership between MSR and Windows/Windows Live.

Singularity - introduced by Vernor Vinge in 1993 - the idea that the exponential speed of technological improvements will produce super-human capabilities, making the future completely unknowable.

Hmmm.

Is it the end of humanity??

Making a case for the internet singularity

  • Democratization
    • Greater and greater power in devices
    • Lower and lower entry barriers
    • Content
      • documents
        • office, html
      • images
      • movies
      • audio
      • publishing
        • blogs
      • software
        • opensource
      • research
        • wikipedia, search
      • metadata
        • tags, play lists
  • Power law distributions and long tails
    • The weight of the tail is greater than the head
    • Coined by Chris Anderson an a Wired article
    • Implications
      • Consumers start to become producers
        • Playlists
        • Remixes and mashups
        • Aggregations...
      • The small producers may outweigh the large producers
      • But is it all worth reading/listening/watching? I'm doubtful...but the important thing is that the good stuff bubbles to the top.
  • Internet ecosystems and network effects
    • The value of a network increases as a function of the number of participants
  • The Innovators Dilemma
    • Cray killed by SGI
    • SGI killed by Sun
    • Sun killed by PC
    • PC killed by cellphone?
    • The first companies must destroy themselves to survive - innovations come in from the bottom
    • Companies often fail to look downward - they only look upward
    • Online world causes the innovators dilemma to run much faster - months versus years

The internet as a mirror

  • We are putting more and more information on the internet
    • The internet becomes more and more a reflection of the physical world
  • Because it is on the internet it is accessible
  • Analysis occurs across the internet
    • For example, data mining of natural language based upon internet hosted information (text)
    • For example, genome data
    • For example genome research papers on the internet so that data miners could access both the research and the data - programmatic data miners can access both the papers and the data and result in new discoveries.
    • This is a circular process
  • The body of knowledge increases

Live Labs

  • A little under a year old, reports to Ray Ozzie
  • A virtual organisation that aspires to hit a suite spot between
    • Users and businesses
    • Short versus long term
    • Problems and solutions
    • Science and engineering
    • (This sounds dubious to me...)
  • Garry then gave a demonstration of Microsoft Live Labs- Photosynth which is worth viewing if you haven't yet tried it.

My thoughts - lots of opportunity if you know how to mine it. Analytical techniques and tools become so important in this.

SAF06 - The Architectural Last Mile

These are my notes from Norm Judah's keynote this Thursday morning at SAF06. Norm is CTO of Microsoft Worldwide Services and IT. These notes are taken during the talk and published straight away - wifi is great.

Norm firstly asks what is architecture?... the art of simplicity.

Norm highlights a number of recent architectural changes:

  • The power at the edge of the network
    • Traditionally big in the middle, dumb on the outside
  • Boundary changes
    • Boundaries between companies
    • Management impacts

Success of failure occurs in the last mile

  • People
  • Human processes are not often re-engineered

Build systems not for the people of today - but the people of tomorrow. Look at how your teenage kids behave: instant messenging, youtube - living online behaviours in a manner that does not make sense to those of us that are in midlife or older. So don't automatically assume that because you can't imagine using this system, that your views will apply for the future.

Processes and human behaviour is important. IT spend does not automatically increase productivity. MIT study by Erik Brynjolfsson (http://web.mit.edu/sloan-msa/Papers/2.3.pdf)shows that process is more important, but also that process + IT investment achieves much greater productivity.

Demonstrated Community Campus - Microsoft only site. Remarked that it's easy to create the site but hard to get people to use it which highlights the importance of managing human behaviour, of process.

A lesson from this is that you need to measure in detail. People will register but not use a system. So a count of registrations has no value. Look instead for how parts of the site are being used.

They prepared an adoption maturity model based on CMMi. Don't build an immaturity model

  • Negligent
  • Obstructive
  • Contemptuous
  • Underming

Refer to Capt. Tom Schorsch, http://www.stsc.hill.af.mil/crosstalk/1996/11/xt96d11h.asp, and the original notes on CMM from Finkelstein, "A Software Process Immaturity Model," ACM SIGSOFT, Software Engineering Notes, Vol. 17, No. 4, October 1992, pp. 22-23.

Make heros! Publish and market the subject matter experts. Give awards for most posts, most effort. Create individual importance.

Another lesson: operational architecture crucial to project success. Since most things go wrong due to operational errors you need to explicitly design for the operation. You can not manage it if you don't measure it.

Norm gave a great example of using the Business Scorecard Manager to report on Exchange availability in Microsoft. This forms the mail SLA reporting. The report showed 33,000 connection attempts blocked for the day - hacking attempts!

The scorecard contained a section called Major Problem Reviews (MPR). These were categorisations of the types of problems.

Desired Configuration Management

  • Reduce configuration changes
  • Proactively manage upcoming change