Showing posts with label dataware house. Show all posts
Showing posts with label dataware house. Show all posts

Why Datawarehouse? Do your company need one?

Monday, June 29, 2009

Data Warehouse As Absolut-e See It

Absolut-e has many times solved organization business issues using data warehouse solutions. We have highly qualified architect consultants who know how to provide solutions to organizations using data warehouse technology. These consultants get better every day on what they do. Data warehouse will be described fully.

What is Data Warehouse?

Data warehouse is a repository for storing organizations historical data. There are numerous historical data that a company may want to store.

Example is data on employees, customer,product,how efficient company's order processing system, Which Plant is providing more profit , trend of last year vs this year much more.

It is important to understand that the data warehouse contains raw data. Data stored in the data warehouse is usually retrieved in the form of information which means the data analyst can perform complex queries and analysis. Queries used to access the information have to be written well to avoid slowing down the operational system. The typical reasons why DWH would be used are as follows:

  • It is used as an investigation tool to find new knowledge. Typical example is to learn more about customers.
  • An organization may decide to use an existing data to make strategic decisions that may have an impact on how they do business
  • DWH help to integrate variety of application systems
  • Sometimes database alone may be inefficient to run analysis on data. Other times databases may be impossible to run trends due to missing data
  • In the real world organizations have more than one operational system, so company-wide reporting could not be supported from a single system
  • Reports in operational systems often require writing specific computer programs which tend to be slow and expensive

The above reasons for DWH creation are solid and it is proven over and over that there is a real need for DWHDWH, which are as follows: in the business world. It is important to mention that there are different types of

  • A Multidimensional Data Model
  • Cuboid
  • Data Cubes – Data modeled and viewed in multiple dimensions, and defined by dimensions and facts
  • Data Cubes
  • Fact Constellation Schema
  • Star Schema
  • Snowflake Schema

What is involved in designing DWH?

The general concept of DWH is to bring data together from a variety of existing databases to support management and reporting needs. The principle is that data should be stored at its most elemental level because this provides useful and flexible basis for use in reporting and information analysis. This is not always the case, because of different focus on specific requirements that require different methods for designing and implementing data warehouses.

For a DWH to be designed the consultant need to understand and analyze business needs and create a business analysis structure The consultant has to be conversant with different views, which are To-Down, Data Source, Data Warehouse, and Business Query. With Top-Down View information has to match current and future business need, and allow selection of relevant information. Data Source View captures, stores, and manages the information by means of data modeling. In the Data Warehouse View, Fact tables and dimension tables and Recalculated totals and counts, and source information In Business Query View data is viewed from the end user view point

The DWH is accomplished by process designs, using the following approaches: Top-Down, Bottom-Up, Combined, and Combined, software engineering point, and step list. Top-Down approach is used when adequate technology is available and understood, especially, when business problems are clear and well understood. It is accomplished with overall designing and planning. The Bottom-Up approach begins with prototypes and experiments. This approach is useful when modeling and the technology is new. It helps to evaluate benefits before making significant commitment to the design. One would use Combined Approach to plan and strategize while retaining rapid development from bottom-up approach. Waterfall or Spiral methodologies may be used in terms of software engineering point of view. In the case of step list business processes, grain, dimension and measures are utilized.

What benefits our clients gain from our data warehouse projects?

Some benefits of using a DWH are as follows:

  • It enhances end-user access to a wide variety of data
  • It enable business decision makers to obtain various kinds of trend reports, for examples are the customers pattern of purchasing, and the item with the most sales in a particular area or country for the last five years.

A data warehouse can significantly enable commercial business applications such as Customer Relationship Management (CRM) , SAP (ERP), Peoplesoft, Oracle EPM , Sales Force, and sales analysis and trends.

What are Absolut-e recommendations on designing and using DWH?

We recommend that you use consultants who have solid understanding of data and it’s manipulation. As the saying goes “garbage in garbage out”. Absolut-e consultant’s have enormous experience of designing and deploying data warehouse applications successfully. Absolut-e will use the correct tools to develop your data warehouse, such as Business Intelligence (BI). BI is a term that refers to applications and technologies which are used to collect, provide access to, and analyze data and information about organization operations. BI enables companies to have a more comprehensive knowledge of the factors affecting their business. For example, metrics on sales, production, and internal operations. Also, it can help companies to make better business decisions. Other benefits of using BI tools are as follows:

  • BI applications and technologies can help companies analyze the following: changing trends in market share, changes in customer behavior and customers’ preferences, spending patterns, company capabilities and market conditions.
  • BI systems can help companies develop consistent and solid business decisions leading to better results than basing decisions on guesswork. Also, it can enhance communication among departments, coordinate activities, and enable companies to respond more quickly to changes that may include financial conditions, and customer preferences.
  • BI applications and technologies can enable companies to make more informed business decisions, and they may give that organization a competitive advantage.
  • BI enables companies to gather information on the trends in the marketplace and come up with ground-breaking products or services in anticipation of customer’s changing demands. Companies need to have accurate, up-to-date information on customer preferences, so that they can quickly adapt to their changing demands.
Why Wait ? check out our free consultation now : www.absolut-e.com
Email us: sales@absolut-e.com to schedule your free consultation & how Absolut-e provide solutions to your data problems & build Decision Support System (DSS)

Comparison of Bill Inmon and Ralph Kimball paradigm:

Saturday, June 27, 2009

Comparison of Bill Inmon and Ralph Kimball paradigm:

In the data warehousing field, we often hear about discussions on where a person / organization's philosophy falls into Bill Inmon's camp or into Ralph Kimball's camp. We describe below the difference between the two.

Bill Inmon's paradigm: Data warehouse is one part of the overall business intelligence system. An enterprise has one data warehouse, and data marts source their information from the data warehouse. In the data warehouse, information is stored in 3rd normal form.

Bill Inmon: Endorses a Top-Down design
Independent data marts cannot comprise an effective EDW. Organizations must focus on building EDW.

"You can catch all the minnows in the ocean and stack them together and they still do not make a whale,"
Bill Inmon, January 8, 1998.


Ralph Kimball's paradigm: Data warehouse is the conglomerate of all data marts within the enterprise . Information is always stored in the dimensional model.
Kimball model also proposes the data warehouse bus architecture. This architecture is comprised of:
- A staging area (which can have an E/R or relationally designed 3NF design or flat file format), which cannot be accessed by an end-user of the data warehouse bus (Presentation Layer) .
-The Data Warehouse Bus itself which includes several atomic data marts, several aggregated data marts and a personal data mart but no single or centralized data warehouse component.
The Data Warehouse Bus:
- Is dimensional;
- Contains transaction and summary data;
- Includes data marts, which have single subject or fact tables; and
- Can consist of multiple data marts in a single data base.


There is no right or wrong between these two ideas, as they represent different data warehousing philosophies. In reality, the data warehouse in most enterprises are closer to Ralph Kimball's idea. This is because most data warehouses started out as a departmental effort, and hence they originated as a data mart. Only when more data marts are built later do they evolve into a data warehouse.

Ralph Kimball: Endorses a Bottom-Up design
EDW effectively grows up around many of the several independent data marts – such as for sales, inventory, or marketing

"...The data warehouse is nothing more than the union of all the data marts...,"
Ralph Kimball, December 29, 1997.

The main difference is Top-Down vs. Bottom-Up Approach.
Let us see the advantages & disadvantages:

Advantages of Top-Down Approach: [Bill Inmon Approach]:

  • A truly corporate effort, an enterprise view of data
  • Inherently architect ed- not a union of disparate DataMarts
  • Central rules and control
  • May be developed fast using iterative approach
Disadvantages of Top-Down [Bill Inmon Approach]:

  • Takes longer to build even with iterative method
  • High exposure/risk to failure
  • Needs high level of cross functional skills
  • High outlay without proof of concept
  • Difficult to sell this approach to senior management and sponsors
Advantages of Bottom-Up Approach [Ralph Kimball Approach]:
  • Faster and easier implementation of manageable pieces
  • Favorable ROI and proof of concept
  • Less risk of failure
  • Inherently incremental; can schedule important DataMarts first
  • Allows project team to learn and grow
Disadvantages of Bottom-Up Approach [Ralph Kimball Approach]:
  • Each DataMart has its own narrow view of data
  • Permeates redundant data in every DataMart
  • Difficult to integrate if the overall requirements are not considered in the beginning
Kimball encourages to have perspective of both “vertically” and “horizontally”
while gathering business requirements while developing/designing data marts using Bus Architecture to integrate in to Enterprise Data warehouse.

Vertical
  • Don’t just rely on the business data analyst to determine requirements
  • Inputs from senior managers about their vision, objectives, and challenges are critical
  • Ignoring this vertical span might cause failure in understanding the organization’s direction and likely future trends
Horizontal
  • Look horizontally across the departments before designing the Data Warehouse
  • Critical in establishing the enterprise view i.e Bus Architecture
  • Challenging to do if one particular department if funding the project. [Even one departments funds this project , but all the departments together become the company]
  • Ignoring horizontal span will create isolated, department-centric databases that are inconsistent and can’t be integrated while considering for Enterprise Data Warehouse
  • Complete coverage in a large organization is difficult , but CEO or CTO represents all of the departments. They should have the proper understanding of the full business and recommend what is good for whole company not just one department.
  • One rep. from each dept. interacting with the core development team can be of immense help
  • Creating a Business Decision Making group when there is a difference in the perspective from each department would help in resolve these kind of conflicts.
New Practical approach by Kimball
  • Plan and define requirements at the overall corporate level
  • Create a surrounding architecture for a complete warehouse
  • Conform and standardize the data content
  • Implement the Data Warehouse as a series of Supermarts, one at a time
[Notes: Soon I will have detailed steps & process to be followed by Kimball , please check back or subscribe to my blog]

SUPERMARTS

  • Totally monolithic approach vs. totally stovepipe approach
  • A step-by-step approach for building an Enterprise Data Warehouse from granular data
  • A Supermart s a data mart that has been carefully built with a disciplined architectural framework
  • A Supermart is naturally a complete subset of the Data Warehouse.
  • A Supermart is based on the most granular data that can possible be collected and stored
  • Conformed dimensions and standardized fact definitions
Data Warehouse to Data marts


Operational Source System (OSS) To Directly transforms data to Enterprise Data Warehouse Let us call -- ETL Process (1) .

Enterprise Data warehouse to Data Marts
Let us call -- ETL Process (2).






Enterprise Data Warehouse











[Further comparison will be added often ]