Results...

Different Data Cube generation time with different dimension, and different dimension number is given below:

These two curves are almost similar and they are linear with respect to the tuple size. And the data for the District wise Division Data line [Fig 1] is steeper than the Month wise year Data line [Fig 2]. Because Division level Hierarchy has more follower than the Year level Hierarchy. So, querying with hierarchy which is followed by more hierarchy, then the resulting line will be steeper. But both of them are linear with respect to tuple size.

These two curves are almost similar and they are linear with respect to tuple size. And the data for the District wise Division Data and smaller Range wise Range line [Fig 4] is steeper than the Data District wise Division Data and Month wise year Data line [Fig 3]. Because range hierarchy is concrete hierarchy wile Year hierarchy is a discrete hierarchy. So, Query with respect to concrete hierarchy will take more time than the discrete hierarchy.

In the Fig 5 all the curves seem to be straight line. That is all of them are linear with respect to tuple size. But the difference is the slope. Time with higher dimension has steeper line than the time with lower dimension count. And in the analysis of the algorithm we show that the slope is (number of dimension x average tuple count). So, the results support the mathematical analysis of the algorithm. Again from the analysis of the data we see that, for the fixed tuple size, time with respect to dimension number increases linearly. That also supports our mathematical analysis. So, the time is linear with respect to nay of them.

Fig 1: Cube generation time for District wise Division Data

Fig 2: Cube Generation time for Month wise Year Data

Fig 3: Cube generations time for Month wise year Data and District wise Division Data

Fig 4: Cube generations time for District wise Division Data and Smaller range wise Range Data

Fig 5: Cube generations time for different number of dimensions with different dimension value

In the Fig 6 the black line indicates the normal algorithm’s tuple vs. time relation and the cyne line indicates the hash mapped algorithm’s tuple vs. time relation, that we have developed. From the figure we found that normal algorithm is polynomial. We also found that our developed algorithm is linear.

Fig 6: Time comparison between normal and hash mapped algorithm

Here is the result of the space required to hold the data cube. In the Fig 7 we see the graph is almost linear, hence the space required is linear with respect to tuple size. This line cut the X-axis, but in reality the size will be fixed at 8KB, which is the block size of windows file system.

Fig 7: Transaction Table size vs Space required

Pseudo code and Mathematical Analysis

I have describe the cube generation algorithm based on HashMap. First we get the entire dimension in a vector. Then iterate over the vector and get the taransactionID and Amount for each dimension in the first while loop. And another HashMap is build with respect to DimensionID and previously build HashMap. Then we iterate through the second HashMap in the next while loop. And find the common transactionID which belongs to all the Map, and push them as final result.

Algorithm:

HashMap dimensionID2hashMapPointer;

Iterator dimensionIterator = dimension->begin();

while(dimensionIterator != dimension->end())

begin

HashMapPointer transactionID2Amount = new HashMap();

Query newQuery = BuildQuery();

ResultSet queryResult = get the query result which contain transactionID

and Amount;

Iterator dataIterator = queryResult->begin();

Repeat

tuple = dataIterator ->tuple();

transactionID2Amount = tuple;

dataIterator = dataIterator->next();

until (dataIterator != queryResult->end())

end

HashMapPointer firstDimension = dimensionID2hashMapPointer->begin();

Iterator mapIterator = firstDimension->begin();

While(mapIterator != firstDimension->end())

begin

Boolean found = true;

String resultString = DimensionID+”#”;

for all other dimension

begin

Boolean newfound = Is transactionID is in current HashMap;

found = found && newfound;

resultString += currentDimensionID+”#”;

end

If(found)

resultList->add(resultString+”TransactionID+amount”);

end

Complexity:

Let,

The query has Dimension Number = n;

The fact table has tuple count = t;

Average tuple count for any query = t_a;

HashMap has constant searching time = s;

So, the first while loop has time = O(n x t_a);

The second while loop has time = O( (t_a) x (n-1) x s )

= O((t_a) x (n-1)) [ as s is constant]

So, the total algorithm complexity = O(n x t_a);

Time Complexity:

The cube access time is linear in this development. It should be polynomial as the dimension of the cube increases, but in this development Hash map algorithms were used to make the polynomial time linear. This can be considered as a great achievement in this system. So the cube generation time = O(n) here n is the number tuples in the fact table.

Space Complexity:

The space required for holding the cube is almost linear with respect to tuple size. Except if tuple size is too low. For very few tuple sizes the space will be constant and after a threshold value the space will increase linearly.

Schema

We have used snow-flake schema for storing the transactional data. In Fig the used schema diagram with the hierarchies are given, where we have shown the ERD of the schema. Here we have shown the hierarchy and the fact table. This is a sort of snowflake schema, and in the ERD there is two reference of BranchInfo related to the transaction table. They can be described as one is for where the transaction executed, and other branch reference through AccountInfo is for account to which the balance is changed. Thus they may not same. Hence this information is not redundant. And we get bank information from the account information. For inter bank transaction or inter account transaction here will be two entries in the fact table, one for debit and other for credit. In the diagram we see that there is a range hierarchy. Range is a concrete attribute but in the schema we made it discrete attribute to ensure faster query response. Query with a concrete attribute require more time than the query with discrete attribute. So, we transfer the concrete attribute into discrete attribute. Similarly we made the time hierarchy, and the last type hierarchy is for transaction type. This will define what type of transaction is defined by the entry.

All these are based on banking system. A data warehouse for banking system.

Comparative study of Data Warehouse and Data Mart

Before we are going to compare data warehouse and data mart first we have to define what “data mart” is.
In simple words we can define data mart as “specialized, subject-oriented, integrated, volatile, time-variant data store in support of a specific subset of management’s decisions”. It is actually a specialized version of a data warehouse, where it contains a snapshot of operational data, predicated on a specific, predefined need for a certain grouping and configuration of select data.
There can be multiple data marts inside a single corporation; each one relevant to one or more business units for which it was designed. Data Marts may or may not be dependent or related to other data marts in a single corporation. If the data marts are designed using conformed facts and dimensions, then they will be related. In some deployments, each department or business unit is considered the “owner” of their data mart which includes all the “hardware, software and data.” This enables each department to use, manipulate and develop their data any way they see fit; without altering information inside other data marts or the data warehouse. In other deployments where conformed dimensions are used, this business unit ownership will not hold true for shared dimensions like customer, product, etc.
So, we found that data marts are actually some sub-set of a Data Warehouse, where they are maintained for the ease of the business people. There is no way that, when a data mart reaches certain size, it becomes data warehouse, or some collections of data marts comprise or become a substitute for a data warehouse. No matter how many characteristics they have in common, data marts and data warehouses will always be different. They can not be used interchangeably, and it’s only when they’re paired together that they operate at their optimal potential.

Concept of Data Warehouse

A Data Warehouse is a database that collects, integrates and stores an organization’s data with the aim of producing accurate and timely management information and support data analysis. That is, a kind of special data base designed and maintained with the goal of decision support system of any operational system. Data are collected and copied to the data warehouse so that queries can be performed without disturbing the performance or the stability of the main system or the operation body.

According to Bill Inmon, who is recognized as the “Father of Data Warehouse”, the formal definition of a Data Warehouse is a computer database and its supporting components are:
• Subject oriented, meaning that the data in the database is organized so that all the data elements relating to the same real-world event or object are linked together;
• Time variant, meaning that the changes to the data in the database are tracked and recorded so that reports can be produced showing changes over time;
• Non-volatile, meaning that data in the database is never over-written or deleted, but retained for future reporting; and,
• Integrated, meaning that the database integrates data from most or all from an organization’s operational applications, and this data is kept consistent.
Data warehouses are generally batch updated at the end of the day, week or some period. Its contents are typically historical and static and may also contain numerous summaries.