|
Information System Management Making Data Warehouses Talk Sense |
||||||||||||||
|
|
||||||||||||||
|
|
||||||||||||||
|
Purpose of this document
Operating in the information age, gathering, analysing and creating information is more and more dominating company resources. Doing it right determines success. This document is written as a handle on how to address Information Management effectively and efficiently.
This document
describes
- What an Information System is
- How to build an Information System
- Why most Information Systems fail
How to read this document
To understand this document it is important to read the chapter Information defining information as used in this document. Other chapters can be read independently.
Contents
Building an Information System
Why do Information Systems fail
Executive Summary
Conducting business requires insight in the internal - and external environment. Information workers need accurate and up to date information to make informed decisions. The ability to manage information has become a critical factor for corporate success in any business. It is the main competitive differentiator in information or knowledge intensive industries.
The function of an Information System is to optimise decision-making by delivering • the right information • in the right format • at the right time. The information-need changes continuously. To be useful the supplied information must keep matching the evolving information need of the individual users.
An Information System must understand business needs to deliver the right information. Interaction with users is crucial to determine the best content and format for new information. Furthermore to be able to continuously deliver necessary information the Information System must be able to anticipate on future information needs and gather data beforehand. Relationships with the data sources are necessary to maintain a reliable source for information. The relationships with business and sources together with company wide information value awareness are the base of an Information System.
A Data Warehouse is only the technical implementation of the production function of an Information System. It is a tool supporting the Information System. It consists of a centralized historical database with data management and reporting functions. It maintains data integrity and consistency and supports the administration of business rules and information definitions. Currently most Information Systems are built as Data Warehouses, i.e. a historic database with reporting functionality. They often lack the flexibility to adjust and evolve on a continuous basis. They also lack the organisation and relationships with the business needed to keep up with business information demand and manage new data sources. Even when delivered with the agreed functionality and in time, these Data Warehouses will eventually fail to perform the primary Information System function: delivering information as needed by the business.
Creating an Information System means building a data management application (e.g. a Data Warehouse) and developing skills, relationships and a company mindset to information value. Creating and improving a small flexible system is best used to develop the required skills. Interaction with users delivering direct value creates the necessary business relationships while adjusting to the evolving information needs. User involvement and valuable results also creates the information value awareness needed to create the right mindset. Once these first results have lead to a basic Information System with basic skills and relationships, new information can be included in the system on a larger scale.
It is a mistake to try and create too much in one go or speed up this first development. It will result in poor results and lacking skills and relationships, thus in a non-functional Information System.
An Information System is a strategic asset of a company. The Information Organisation managing the system represents the core of this system and is an integral part of the company. As a primary process it should be placed in a separate department comparable to the finance department, controlling all information assets of the company to the benefit of the whole company. To ensure continuity in knowledge and experience core responsibilities in the system must be kept within the company and cannot be outsourced.
Information
What is information?
To define information in this document, a more general term is introduced first.
|
Insight |
Clear or deep perception of a situation |
Insight can be seen as the effectiveness with which the environment is perceived. For a manager insight is critical.
The success of a company depends on the quality of business decisions. The quality of business decisions depends on the right interpretation of the environment. Does the employee know and understand the threats and opportunities in the environment and can he make the right decision? Can he see through the environment to the next steps, stay ahead of competitors and threats, and anticipate on future situations and take appropriate actions?
To obtain insight an information worker, someone who needs information to perform his tasks, uses experience and expertise to process information.
|
Information |
A message received and understood that reduces the recipient's uncertainty |
Information can be a description of a business process; a weather forecast; a long-term budget proposal; the production report for January or an interactive analysis application filled with last year’s accident statistics. Information can be obtained from hallway gossip, the smell of coffee or a slight hesitation in a conversation. It can be any message as long as it is received and understood and increases the recipient’s insight. In daily use data in any form is often referred to as information. In this document ‘information’ is used as the meaning of data as perceived by the user and as the representation of this meaning in an understandable and accessible format. This means that just data is not information. It is information if the intended user can understand it and it improves insight.
Obtaining the right information is a
continuous iterative process of interaction with the environment. It is
gathering information (i.e. meeting people, reading reports) and interpreting
and combining the information to a clear view on the environment. Decisions are
based on this view and thus based on the available information. The information
need evolves with every change in the environment. It even changes with every
change in available information.
Diagram information with limited scope
Information Management
In the information age information has become one of the main assets. The effective use of information is increasingly determining the success of a company. Creating awareness of information value and the ability to manage information company-wide will have great impact on business results. Since information is used in every aspect of business and generated in every process, a centralised managing function is needed to structure, combine and distribute it effectively and efficiently.
Where the finance department monitors and manages the financial aspects of business and the legal department handles all legal affairs, the Information Organisation should manage this important business asset.
One of the most important and difficult tasks is determining valuable information and using it. Where legal issues and financial matters tend to identify themselves automatically, although often too late, with information this is not always straightforward. Data is generated in all processes in and outside the company. Determining the potential information value of these processes and creating a system to effectively use it requires specialized skills. It means combining in-depth business know how with in-depth system/process knowledge. When the information value is close to the process, e.g. obtaining direct process efficiency, this link can often be made by the department itself. When information value is business-logically separated from the information source this can be very difficult. Optimising supplier and customer relations inter-departmentally is a simple example. Identifying tactical processing strengths in one department and using them commercially in another could be another issue.
Where money can be transformed and traced easily, information is an asset that needs processing and care. Handling it wrong can render valuable information useless, on the other hand information can be copied and used on different locations multiplying its value. Like money the optimal use of information requires strict procedures for handling it. As for money this depends on the specific organisation.
Information Benefit
The benefit of information is a result of information value and information cost.
Information Value
The business value of information is the improvement of business by better decisions. This means maximizing profits and minimizing risks. On a strategic level a better perception of the situation can lead to an advantage by choosing the optimal investment strategy. Tactical it can mean that competitors can be outsmarted and orders won by a faster and more client specific approach. At an Operational level the right information can be used to optimise production processes leading to a better product or cost savings. Next to this, the advantage of a successful information system is that the time spent on gathering information by specialist employees, e.g. (sales) managers, can be reduced so that more time remains for the actual work on hand.
To be valuable information must be:
|
Information needs to address relevant business issues |
|
The meaning of the message must be understood. Complex indicators might be a good measure for business performance they are of no use if it is unclear how they can be influenced. |
|
Displaying the same information on different reports must result in the same figures. When the information is different this must be clear in the report. |
|
The message must have a digestible format. Irrelevant or too detailed messages blur the picture. “Too much information creates a swamping effect”. |
|
Missing critical information can result in a misinterpretation of a given situation. “A little information is dangerous”. |
|
Information integrity must be known. Unreliable messages can contain information as long as the status is known. |
|
Information must be provided at the right time. This usually means as soon as possible. This can conflict with aspects like reliable or concise. Sharing confidential information in a closed group before publishing it demands timing. |
These items are not given in order of importance. The importance of each point is determined by the way in which current information does not meet this criterion. Not meeting either of these criteria will probably render the information useless.
Information Cost
Besides potential value there are certain costs related to obtaining information. Managing information is also identifying and managing these costs. The cost of information can be divided into production-, development- and maintenance costs.
|
the costs of determining the right information contents and format and how to produce it. These costs depend on information specialist and user skills, system content knowledge and available tools and sources. |
|
the costs made to generate and distribute a single report. They are mostly depending on the manual interaction needed to produce the information. |
|
the costs made to update a report when data sources change or simple changes in the contents are made. These costs depend on the frequency of change and the costs of implementing and testing report adjustments. |
A weekly progress report created by a manager combining several production reports into one has low development and maintenance cost but high production costs.
A large number of individually automatically produced reports based on operational sources have little production costs but maintenance and development costs are high. Each report has to be adjusted and tested individually. Intelligence can be copied from similar reports but logic and correctness must be determined for every single report.
These costs are made in the following fields:
- External costs: consultants, programmers, information delivery bureau
- Personnel costs: extra wages and missing capacity for work at hand
- Software/hardware/infrastructure: acquisition and maintenance of machines, databases and tools
Information System
An Information System supports business by delivering business information in an effective and efficient way. To support business an Information System must be able to supply information when it is needed. It must follow the changing information needs of the business.
An Information System is not just an application where data is gathered and transformed into information. It is a system that develops and maintains the information applications in a company. It must include connections to all functions of the company and the relevant external environment. An Information System consists of three stakeholders working in cooperation.
q The information users
q The data suppliers
q The Information Organisation
When creating an Information System the three parties must be brought together and made aware of their responsibilities for the system to work successfully.
Information users
The information users are responsible for using the information and supplying feedback. Besides determining and optimising information contents and format in cooperation with the Information Organisation, the information users are responsible for maintaining information definitions. The business must actively drive the Information System, if not, the delivered information will stop matching the business needs and will become useless.
Data suppliers
Every company has a
number of systems administrating business processes. These systems provide data
on which information is based. The data suppliers,
internal and external, are responsible for administrating business processes or
gathering other data and delivering this data to the Information System. Changes in these systems that have an impact on
existing reports must be communicated to the Information Organization.
Source system development must include communication with the Information System team to make sure the Information System can adjust to production system updates. For new applications, with information value, the Information Organisation should be included in the entire process.
Information Organisation
The Information Organisation manages the Information System. Its task is supporting business by delivering the necessary information. It manages the Information System and forms the liaison between Business Information needs and operational data sources. The Information Organisation has the following responsibilities:
q Analyse and foresee business information needs.
q Proactively manage source systems contents and - development.
q Maintain a reliable and functional technical information infrastructure (i.e. Data Warehouse and related applications).
The first two points require in-depth knowledge of both business issues and source system contents and development into one function. Close contact with the business and consultation on system development is crucial for the success of the Information System.
As the information keeper of the company the Information Organisation must be aware of all possible information needs in the company. Then, being actively involved in all system changes and developments, it must weigh every change and development on its added information value. Include this value in the decision process and make sure the information issues get the attention they need.
Maintaining the Data Warehouse requires technical skills and knowledge to set up, maintain and enforce Data Warehouse rules and guidelines. Creating content independent code, retrieving business logic on-line from the administration database when needed for processing data.
Information System benefit
The benefit of an Information System lies in increasing total information value while minimizing the information cost. This does not always mean a decrease in information cost.
Information value increase: through formal definitions the data becomes consistent and the information becomes clearer. Centrally storing intermediate data and information definitions increases the possibilities for quickly analysing data to create new information.
An Information System where business information is administered formally can reduce information costs considerably. Creating a platform containing all sources of information directly available for queries increases development speed for new information. Production costs are kept low by using economies of scale on skills and performance. Separating programs and business logic reduces maintenance costs; there are fewer programs to maintain and business logic can be maintained without using IT staff
The cost of information becomes clearer when the information effort is organized centrally instead of having distributed staff devoting half their time in producing periodic reports. This might lead to the impression that information costs are increasing, make sure the advantage of reducing the workload in the rest of the organisation is noticed. Setting up the organisation for information management and acquiring the skills and knowledge needed takes time and requires dedication of educated employees. Defining and administrating all information definitions might take more effort than merely creating a report from scratch. The first reports might cost more. However, information based on existing definitions and available data will be created at only a fraction of the cost. Adding a new data source to one report is cheaper than adding it to the Data Warehouse. When it must be included in multiple reports, the costs of adding it to the Data Warehouse is shared where adding it to each report individually would multiply the costs.
All the information definitions and rules and guidelines will not limit the flexibility or freedom to analyse data and develop new reports. They just ensure that the delivered information is of value and available to the entire organisation, now and in the future.
Building an Information System
Keep focused on the function of the system i.e. delivering valuable information that increases insight.
An Information System consists of an Information Organisation with relationships with Information Users and Data Suppliers. To provide the information in a cost effective manner usually a Data Warehouse is used as storage for historical data and an administration of Business Logic.
Three parts of an Information System
Building an Information System is developing an information organisation to develop and maintain necessary relationships and a functional Data Warehouse.
Information Users
Aiding the user is what the Information System is built for. To develop and improve the system user input is crucial. Lose the users interest and the system will stop developing and become useless.
Building and maintaining a relationship with the information user is fairly straightforward in thought. Deliver what the user needs, deliver it quick and keep delivering. However, determining the best content and form for information is not easy. It requires the cooperation of both information engineer and business expert. The best results are obtained in an interactive process where quickly evolving prototypes and feedback creates a feeling for the users needs and the current information possibilities. Not only does the user get an idea of what he is asking for and will understand what he eventually gets. The information engineer also gets insight in the users needs. In future situations he will be able to determine the potential value of information origination elsewhere for this particular user and proactively include it in the system. Losing the relationship is much easier. Let him wait too long or don’t let him know what he is getting and he will no longer come to you for feedback. Making the user understand the value of the delivered information is what keeps his interest.
Data suppliers
Building relationships with data suppliers can be difficult. Suppliers usually have no direct gain in supplying the data. It only complicates their jobs because they have to include the Information Organisation in all modification plans that might affect the data. Data delivery usually includes setting up procedures and changing schedules. These are not the type of actions maintenance organisation want to do on a regular basis. New data sources should therefore only be added once they are needed for information. However, when adding a new data source, try to include as much data as possible. There is not much extra effort involved at this stage and extra data is bound to be useful in the near future. Every adjustment of interaction with source systems can lead to time consuming coordination and acceptance procedures which you don’t have time for once you need it.
Data Warehouse
The Data Warehouse has three main functions. •It operates as an historical database where operational databases do not keep the history. •It creates a buffer neutralizing source system changes towards the delivered information. •It serves as a reliable source for developing and delivering new reports. To set up and maintain these functions effectively Data Warehouse expertise must be included in the Information System. Without the proper set-up of the Data Warehouse The Information System will fail.
Information System Development
There are only two constants in an Information System, Growth and Change.
When building an Information System growth and change must be incorporated in every piece of the system. Continuous growth and change implies that an information system is not something you build now and use for a number of years and then build another. It is a living system that evolves continuously. Not only the organisation, but also the relations and applications it maintains. For the Data Warehouse it means that the architecture and design are under constant stress. Once the tension gets too high due to incompatibility of the current architecture with the needed functionality the architecture has to be altered. The same applies for the organisation and the relationships.
An Information System interacts with many different groups and depends on successful relationships and information awareness. Building a successful Information System requires developing these relationships and creating this awareness. Since key users, i.e. tactical and strategic managers are usually hard to get hold of and easily distracted, it is important to deliver valuable results in short development cycles. The key to a lasting information system is to maintain the ability to create new information in a short period. Maintenance cost and effort must be as low as possible since maintenance cost will limit development of the system in the future.
First start
In the beginning the system (i.e. all stakeholders and the application) endure relatively big changes. The information users must be able to relate to the new information and the system has to be adjusted to new developments. In this stage it pays off to start as small as practically possible, keeping the team and the application small improves flexibility and understanding in early development stages. Having two streams of information developed simultaneously adds overhead to the system. It doubles the needed resources while cutting the results in half by extra distraction due to communication and doubling start-up related mistakes. Getting the Information System organised requires continuous redesign and rebuilding in the beginning. Simultaneous development not only doubles the extra work by not using the previously gained experience but creates an extra round of redesign to integrate the results. Once the Information System has matured to fit the organisation. • the relationships with the environment have evolved • a basic set of information definitions is available • development procedures and guidelines are created and • an initial historic database is filled. Using this system new information can be developed and integrated into the Information System in parallel.
Besides gaining Data Warehouse experience and determining the best architecture to fit the company needs a successful quick first implementation can show the value of the system immediately to the company. Paving the way for future developments is a major goal of the first application.
Continuous development, continuous evaluation
Make sure you keep thinking about what you are doing and keep doing it. Business information value can best be determined when it is available. Information is only worth as much as it is influencing company decisions for the better. The objective is not to stop creating better information but to make sure it is worth the effort. Do not expect perfect information. Perfect information has an infinite price and can only be given afterwards. The aim is to make sure you get sufficient information and keep improving it as long as it’s worth it’s while.
Moreover, in this context less is more. Make sure all information that is created is used. Remove old reports when they are no longer in use. Distinguish periodically needed information from ad hoc requests. Ad hoc requests give a very clear information need and probably information gap in the current system. Check if information is not already present and keep in mind the possible value for others. The information might be useful to others.
Keep the information as clear and simple as possible. Users must be able to understand the provided information. Comparing figures from different reports can only be done effectively when the meaning of the figures is clear.
Development cycles
Developing the information system consists of continuously following two cycles. The short cycle adds new information to the system the long cycle adjusts the structure of the system to the added functions.
Short cycle (adding information)
The short cycle is user driven adds new information to the system. Adding information to the system requires strong involvement of the stakeholders needing the information. It is impossible to create information without the cooperation of the users. Adding information to the Information system begins by determining the user.
1. Identify the user with an information need. Without a user needing the information there is no value.
2. Determine the best information and the necessary sources. The user can best determine information value by using it. The information is determined in an interactive and iterative process where information specialists and business users cooperate in creating the optimal solution (maximal info with minimal cost).
3. Define the information and the source data. Once the necessary information is clear it must be described and registered to make it available for future use.
4. Integrate the new information and source data in the Information System. Integrate sources in the periodic data download, create quality checks and make the information available to the users.
Long cycle (structural changes)
The long cycle consists of two phases. They are not directly guided or invoked by the system users, the long phase of regular development is interrupted by short revolutions where the application structure and the organisation is adjusted to fit the current needs.
1. New information is integrated in the system in the current structure, following the short cycle. Integrating new information in the system will put a strain on the architecture of the application or the organisation when it does not completely fit in the existing structure. Small adjustments are made to make it fit.
2. The system is adjusted to the new demands once too many exceptions must be made to fit the necessary information. This can mean redesign of the application or restructuring the organisation or the procedures.
Restructuring the Information System should have as little effect on existing information as possible. It is intended to simplify maintenance and new developments.
Take small steps
The development of an Information System depends on the development of the information needs. It is a mistake to make over ambitious demands at initial stages. By the time they are met the demand will have changed and the information delivered will no longer match the current need. Small quick steps make sure that the information is understood correctly and is optimised for the time of delivery. Large steps usually imply more developers whose activities must be coordinated. This kills speed and flexibility and thus the ability to move quickly without losing the overall structure. Make sure the new functions are integrated in the system. Where this takes a little effort with each new addition, omitting this phase will increase maintenance costs and eventually reduce flexibility to a point where the system is no longer worth wile.
Determine the right information and include it into the system
Determining new information usually requires building the actual report with the right data. This might lead to the impression that when the report is ready the job is done. This is not the case. When generating a report the information shown in the report must be defined and incorporated in the Information System. All data used to generate the information must be added including periodic refreshment, cleansing and standardisation. While this might seem to be costly and cumbersome it pays off very soon. Increased adding of information definitions and historic data creates a library from which new information can be obtained easily. This drops development, maintenance and productions costs of future reports dramatically while enhancing speed, quality and value due to shared routines and definitions. Not incorporating the information fully in the Information System increases the risk of creating double functionality and delivering unregistered information. This will undermine the integrity and flexibility of the Information System and destroy it in the end.
Information System responsibility
A functional Information System is an operational, tactical and strategic tool. If managed properly with the interest of the company in mind it becomes vital for the position of the company. It combines business expertise and information knowledge throughout the company. An Information System is critical to the core business and any other business. Therefore the responsibility for the Information System can never be outsourced. No external party can put in place and maintain an Information System without creating a complete dependency to said party. The responsibility and key development of the system must remain in the hands of the company. Consultants and developers can be used to create parts of the system but may never be made responsible over the system. Acquiring dedicated employees that will perform these tasks is a mayor issue and must be initiated as soon as possible.
Why do Information Systems fail
An information system fails when there is no positive impact on business justifying the effort spent on building and maintaining it. Information Systems fail because the information delivered is not or cannot be used to increase company results or when costs for maintaining the Information System rise to a point where positive effects are no compensation for the costs.
Most Information Systems fail to deliver the maximum gain just by not delivering the needed information. By not incrementally following the evolving information need the system is not optimised to the users needs. On the other side, the users do not obtain awareness of information value they would have when being involved in continuous development.
The value of an Information System depends on the way the system influences business for the better. Current implementations are often created in big projects incrementally delivering a set of functions based upon previously analysed needs. Where this approach can work for administrative applications with fixed functionality it often fails to • include flexibility needed to effectively adjust to the changing information needs • create information awareness in the organisation • develop relationships needed to develop new information • introduce an information organisation with sufficient know-how and responsibility to manage and control the information system.
q Without relationships or flexibility, it will not be able to support decision making optimally and will not deliver the information needed.
q Long development steps limit user interaction and limit information usage. Information is obsolete before it is available or only data is delivered because determining information was not possible at the time the system was designed.
q External development of an Information System threats system continuity. To keep an Information System functional, continuous effort must be put in integrity and flexibility. With every adjustment an investment is necessary, failing to do so will render the system useless in a short period.
q Without information value awareness, available information will not be used to its full potential and new information will not be developed.
q Without an organisation managing information needs and possibilities, new opportunities will be missed and operational chaos can compromise the system.
Summarized: a Data Warehouse without Information Organisation or relations with the business will not deliver information.
The Solution
When developing an information system just begin to make sure it generates valuable information and not data. It is better to create a report that is perfect for a specific need then to make a report based on a predefined compromise between several users. The created information will have value and increases insight into information needs and the way the business is looked at. Adding more and more information to the system and solving the problems as they occur will eventually lead to a system as mentioned. In fact, that is the way any functional Information System is developed. Having an experienced team helping you take the first steps and educating your staff in the first period can make all the difference.
Developing an Information System is not easy, so you should not make it harder then it is. The right Information System will deliver according to the energy spent on it. By not focussing on the right goals, it is easy to spend too much and get very little in return.
- Keep focused on delivering valuable information.
- Make sure the relations and the information organisation keep in line with the business
- Include all new information in the Data Warehouse to share it with the whole company
Start off with a very small team and get used to the fundamentals of using and maintaining an Information System before launching a company wide solution. The first version of an Information System creates the Architecture and Organisation around which the other functions must be created. The first steps are flexible and the outcome in skills and experiences as well as the first Data Warehouse determine the success of future developments.
Support growth and change in every item of the system.
The organisation and architecture of the system initially start off flexible and loose but will gain structure and standards with every new step. Getting all definitions right the first time is next to impossible so having to change them in the development process is inevitable. When starting small and slowly adding functionality, changes and system redesign are relatively easy in the early stages. Basic functional needs will surface in this process resulting in a system best fitting the organisation.
To support rapid development and a company fit architecture the initial development of the system should be the responsibility of a small, dedicated team. Working close to the business this team must be able to understand the business and help in determining the desired information need. External specialist help can be used to provide extra skills and knowledge to support the critical development stages. Eventually the development and maintenance should be the responsibility of a structured Information Organisation managed by company personnel. Their role is to ensure the strategic advantage that a truly functional Information System brings.