The Secrets to Big Data and Information Optimization Revealed in 2013 Research Agenda

Managing the access, storage and use of data effectively can provide businesses a competitive advantage. Last year I outlined what the big deal is in big data, as the initial focus on the volume, velocity and variety of data – what my colleague Tony Cosentino calls the three V’s – is only one small piece of how organizations should evaluate this technology. The more balanced approach is to include what he calls the three W’s – the what, so what and now what, which shifts the focus to an outcome-based view that can handle the time–to-value urgency found in business. Big data analytics can help assess the volume of data, while the velocity of data that is potentially in-motion is best handled by what we call operational intelligence. Beyond these, techniques and technology such as predictive analytics and visual discovery facilitate extracting more value from big data. Along with a wide variety of data, these tools help organizations focus on optimizing information assets. We will soon conduct benchmark research into information optimization to determine how organizations are dealing with their information today and what steps they are taking to improve. In-memory computing will surely be one of those steps, as it can significantly improve the time-to-insight equation.
Big data does not magically become valuable, nor is it easily implemented. Organizations must realize they still require good information management competencies that address the full lifecycle of access and integration across in-house and cloud computing resources. Our research into data in the cloud has found that most organizations are not prepared to handle this broad, distributed data dilemma and has become very important. Technology to harvest and integrate data must become  easier to use for both business and IT users, which means overcoming the prevailing reliance on spreadsheets, which our research has found is a larger data challenge than most people realize.
Our research agenda for 2013 calls for us to examine not just the forms of big data technology, but also the impact and value of big data tools that organizations can use to maximize the value of their information and drive better insights. Realizing the vision of greater intelligence across processes and teams takes an investment of time, effort and money. Accomplishing such a feat requires focusing on information competencies to support big data effectively, which requires an assessment of the information processes to deliver data to business effectively. Our research into operational intelligence found that the use of events is a critical part of the big data environment. At the same time the skills of master data management and data governance do not go away, and in fact become important to address the business accuracy question that inevitably pops up when more data becomes available to be utilized. Our research into product information management has found that the drive for data quality is changing organizations’ approaches, and that a comprehensive information strategy using MDM and PIM together across business and IT can yield significant benefits compared to an IT-only approach. Our Product Information Management Value Index found some startling results about which vendors really meet the business needs of organizations. Product information is one of the many priorities as well as customer, employee and finance that need to be part of a big data effort and information optimization set of processes.
To embrace big data and optimize the use of information across an entire enterprise requires not just competencies and methods to ensure effective deployment, but also the ability to understand the business cycles for information and design the technology accordingly. Our research this year will examine the varying types of big data technology, including data appliances, Hadoop, in-memory computing and RDBMSes,  all of which our big data research in 2012 indicated have a nice growth pattern going into 2013. The number of Hadoop tools in particular has expanded dramatically in the last year, making it easier for existing staffers to utilize that technology without having to hire dedicated programmers, as was the case for early adopters.
In addition, we have seen that large-scale in-memory computing architectures can provide significant value when it comes to working with big data. A new generation of data appliances arrived on the scene in 2012. Of course it is critical to explore new methods to provide analytics and gain insights from big data and not assume that your existing providers will provide you a competitive edge with innovative approaches. Also, businesses must not limit themselves to the use of structured data when they can also utilize varying forms of content to help get a more integrated view of information. Big data can become strategic, but businesses need some technical acuity to design the best possible architecture for their needs.
It will be critical to understand best practices for big data in 2013 and not get caught in the neverending cycle of evaluating technology. IT organizations need to deliver value to business iteratively in order to be seen as contributing to the value business expects from technology investments. It is also critical that IT and business analysts work together to find the right big data approach. They must reduce time spent on data-related tasks; our technology innovation benchmark research found that more than 40 percent of organizations currently spend too much. Supporting a variety of data will be critical, as even location data and resulting analytics, which we are researching, is a much larger priority in business than most people in IT realize. It might very well be that businesses must adopt a distributed set of big data technologies to meet their collective needs.
Being efficient in blending big data with existing applications requires good data integration. Our assessment in 2012 found a large selection of vendors in this area, but only some are exploiting the integration points of big data technology and where they might exist in cloud computing environments. Successful data integration might require a deeper examination of data virtualization, which our information management research found to be a growing priority. This year will see a new crop of information and business applications that exploit the value of big data and are aligned to specific line-of-business needs. At the same time the use of dedicated analytics designed for this approach, which we call big data analytics, will require examination in 2013; my colleague Tony Cosentino has plans for new research on lessons learned and the technology providers in this area, as he outlined in his agenda for 2013.
Businesses must make sure to examine new methods of assembling and harvesting information from existing applications and systems for business, including those that might not be classified as big data but can deliver the value required for business needs. What’s old can still be new, and the role of data integration has never been more important to automate the flow of data in the enterprise. We will research further into data integration in 2013 to determine best practices and build upon our existing Value Index on Data Integration with a new report that highlights the expansion of data integration vendors’ support for big data and cloud computing.
As information becomes potentially more easily managed, organizations need to expand upon traditional business intelligence and look at the role of predictive analytics, which our research has found is essential for optimizing business processes and aiding critical business decisions. We have found the largest obstacle to predictive analytics is difficulty integrating new tools into existing information architectures. For many organizations, our research has found getting the basics in business analytics requires a dedicated approach. With the commoditization of hardware and memory and the concomitant increased computing potential, as well as the emergence of cloud computing options, these new methods for taking advantage of big data become more cost-effective for a spectrum of small and big businesses.
All of these are exciting advancements in the science of information management, and CIOs should make a point of learning about and investing in all of them. Being more intelligent with big data is the mantra for 2013. Organizations that heed lessons learned and research the right path forward will reduce their risk of not delivering the value their businesses demand. While a big portion of the technology sector attaches itself to big data, being pragmatic and assessing the right path forward will be the most important best practice for 2013.

Original Source

Read More

ETL to ELT Conversion Testing in Data Warehouse Engagements

With the ever changing business environment and an extremely volatile economy, the financial services and the banking industries are facing acute challenges in quickly generating proficient analytical information/reports through data warehousing, which enables businesses to make informed decisions.

In the recent past, the architectural design of data warehouses has predominantly been ETL (extract, transform and load). However, with emerging technologies taking center stage, the architectural vision of data warehouse systems has gone through a radical shift. This shift has led to a new data warehousing process, as the previous data models were not equipped to handle the exponential increase in user data. Often, businesses were also unable to meet their critical data-related business needs due to their inability to incorporate changes into their large single data repositories. In turn, that translated into a longer waiting periods for reports that were requested by business users. Eventually, organizations ended up specifically dedicating resources to "extract data,” performing additional and often redundant analytics and reporting activities outside the data warehouse itself.
Thus, several manually intensive and expensive processes have to be established to fill in the gaps created by the limitations of the architecture design of the ETL-based data warehousing systems. The strategic solution was the implementation and transition to the data warehouse design technology ELT (extract, load and transform).

Importance of Data Warehouses in Banking and Financial Services

Banking and financial services involve numerous data intensive processes, from withdrawing money from an ATM to suggesting the most lucrative investment portfolio to a high value client. This makes data warehouse testing extremely critical in this domain. Figure 1 shows some examples of scenarios that involve data warehouse processes in specific financial services domains.
Figure 1: Data warehouse system dependability in financial services sub domains

ETL, Focusing on Desired Data Output

The ETL data warehouse technology is primarily based on the report requested by the business user. The approach is top down. First, a decision is made regarding the desired output data (i.e., the format and content of the report demanded by the user) and then the required data is extracted, transformed and loaded, as seen in Figure 2. With the focus on output, the input data needs to be processed to remove any data anomalies, ensuring high quality of data for transformation and loading.
Figure 2: The standard ETL process

When data warehouse systems were initially conceptualized, the ETL-based systems met all the business needs, as the focus was on designing the output first. Moreover, data that was relevant only for the processing needs was required. However, these systems had their limitations when needs related to future expansion arose, like the re-modification of output data, which would need a complete redesign and change to the input architecture of the data warehouse itself. This would be a very expensive and cumbersome process for an organization to achieve with the traditional ETL process.

Data Warehouse Technology ELT

The concept of ELT data warehouse technology came into existence because of the high business demand for error-free, high-quality input data in data warehouse systems. Here the approach is bottom-up. The data is initially extracted to a staging area, where the business rule and integrity checks are performed.
With ELT, the load and transformation processes are decoupled from each other, as shown in Figure 3. Hence, the development is split into smaller manageable chunks, which make project management, change management and the development of data warehouse systems much simpler.
The ELT data warehouse systems overcome the shortcomings of ETL systems, which paves the way for this transition across industries, especially in financial services and banking.
Figure 2: The ELT Process

Achieving Thorough ETL to ELT Conversion Testing

If an organization decides to transition from ETL to ELT, from a QA perspective, a lot of planning and strategy is required to ensure that the conversion is thorough and successful. In any typical re-engineering project, especially one that includes an ETL to ELT conversion, the following validation points need to be looked at (Please note that the ETL process is mentioned as the source and ELT process as the target wherever applicable):
Structure of the target table: The ELT output table structure is usually not normalized for easy accessibility to the input data. Processes and columns need to be mapped to ensure consistency between the ETL source systems and the ELT target systems.
Data type of the target table: Usually, the structural compatibility between the source and target database needs validation. Accordingly, test cases are framed to warrant data quality in the targeted database. Lastly, the output format of the data is validated to ensure no truncation has occurred to the values of data.
Intermediate phases/models: Usually, the loading or the staging area is divided into intermediate models that include:

1. Input data model, which contains data that is clean and completely decoupled from end users.
2. Reporting data mart, which contains data that is useful for generating reports for the end user.
3. Reporting view, which contains formatted data in a predetermined template.

An effective QA includes testing of all three stages, as shown in Figure 4. The data is also compared across stages to ensure its accuracy and consistency.
Figure 4: Sample intermediate stages in an ELT system

Relationships in output data models: As stated in the previous sections of this article, the main advantage of ELT processing is the easy maintenance of individual stages due to the clear segregation of each stage from the other. Let us consider a scenario where a customer’s loan information is stored in a source table and a record consists of the customer’s first name, last name, address, document name, type, loan type and the loan amount. The target ELT system would require tables on the customer, loan and the document information. After the transition from the ETL to ELT system, the following data relationships would need to be validated to ensure that they are still intact after the transition:

• Customer table to loan table.
• Loan to document table.
• Document to customer table.
• Static table testing of relationship codes.

Backup database: The master and duplicate databases would need to be validated for consistency in terms of database structure, framework and data between them.
Process validation: This entailstesting the data loading and scheduling of jobs by checking the error logs and output of the scripts used for loading. The analysis of error logs is done to report any inconsistencies (e.g., an actual error occurrence not reported in the error log or vice versa). The ELT jobs are then triggered in sequence to test the ELT data warehouse system behavior.
End-to-end validation of the ELT system: All the stages (i.e., the source, input data model, reporting model, reporting view and report flow) are to be tested using predefined input data to ensure that it moves through all stages in the expected way, generating the desired output. This end-to-end validation of the ELT system is essential for detecting inherent defects due to the interdependency of code/data (if any) between all the stages.
ELT isn’t a solution to all the problems of data warehouse systems. In some specific cases, a hybrid solution of both ETL and ELT methodologies would be the appropriate model to adopt, depending on business requirements. To illustrate this,  consider a situation where a data warehouse has multiple source systems. The data from these sources can be brought to a common staging area through the ETL process and then specific ELT processes can be further applied to generate user-specific reports.
However, having an ETL, ELT or ETLT as a data warehousing solution is not enough. A clear understanding of the vision of the data warehouse output from an end user’s perspective is essential for a successful data warehouse testing validation. It’s also important to comprehend the business context and clearly define the input data, their interrelationships and the associated processes in the data warehouse system to generate correct and relevant reports for the business/end user.


Original Source

Read More

5 Recommendations for Developing a Big Data Analytics Strategy

Big Data, it seems, has reached the main stream. Everyone’s talking about it and everyone’s wondering whether they should do it, according to new research from IBM. But it remains to be seen whether businesses and business decision makers really understand what Big Data actually is, or how to develop a strategy get the most of it.

IBM Big Data @ Work Survey

The findings of this research entitled "Analytics: The real-world use of big data" were based on the IBM Big Data @ Work Survey that was carried out in the middle of last year in 95 countries across 26 industries and which surveyed 1144 professionals working around big data.
The first thing it found was for many, even the term big data was confusing largely because it is used in association with so many different IT disciplines. It was used by respondents in the survey to describe all kinds of different things including: large quantities of data, social media analytics, advanced data management capabilities and a lot more.
However, it also showed that a number of pioneering enterprises are already starting to achieve breakthrough business outcomes.
While there are many ways of defining big data analytics, for the sake of this study IBM defined it as the ability to extract new insights from existing and newly available internal sources of information. It also refers to the technologies used to extract meaning.


While the study goes through many different aspects of big data, like the four elements of big data — volume, variety, velocity and veracity — it also offers five principal recommendations for organizations to progress their big data strategy and maximize the business value from enterprise data. Those five recommendations include:

1. Start with customer-centric outcomes
2. Develop a strategy for the entire enterprise
3. Start with data that is already available in the enterprise
4. Identify business priorities and build the strategy on that
5. Develop business case based on measurable outcomes

1. Customer-centric Outcomes

It is absolutely essential, the research says, that enterprises focus their big data strategies on efforts that will provide most business value. For most, this means starting an analytics strategy with customer analytics in order to provide customers with better services, which in turn should lead to better customer retention.
While this may seem obvious, the difficulties in doing this are growing every year as individuals become digitized and better informed of their choices.
The result is that enterprises will need to understand their customers as individuals, and need to invest in new technologies and advanced analytics to do this.
The ultimate goal here is not just to get to know the customer, but also to connect in a way that the customer sees as useful, which may come in the shape of more timely, informed or relevant interactions. Big data analytics, for its part, offers insights from big data that are increasingly important in these kinds of relationships.

2. Enterprise-wide Big Data Blueprint

The blueprint of your big data strategy must cover the overall vision, the strategy and the requirements not on a departmental basis, but on an enterprise basis. This should result in the creation of a common, enterprise-wide understanding of how the enterprise intends to use big data to improve business objectives.
With it the enterprise will be able to identify key business challenges to be overcome, the business process requirements that define how big data will be used, as well as the architecture, data, tools and hardware needed to make the blueprint a reality.
It also provides the basis for developing a roadmap to guide the organization through practical approaches to the development and implementation of its big data strategy.

3. Start With Existing Data

To achieve short-term results as the big data implementation starts and gathers steam, enterprises need to be realistic about what they can achieve initially. For those that have implemented a successful strategy already that is providing business value, the easiest place to gather insights is from information that is already in the enterprise.
Doing this enables an enterprise not only to use easily available data but also the skills and software that are already in place. This provides immediate benefits as they make the business case for extending the big data analytics to include more complex sources of information and types of information.

Most successful strategies have started analyzing existing repositories of information while scaling data warehouses to handler larger volumes of information for future insights.

4. Business Priorities, Skills Investments

As the market matures, businesses are being forced to choose between a growing number of analytics tools while, at the same time, having to deal with a critical shortage of analytics skills in both the US and Europe.
Big data success hinges on finding a way around this, which has seen IBM set up special analytics colleges in the US and Canada.

But for the moment, businesses will have to work in a market as it exists now and that means investing in tools and skills. As part of this process the research suggests that new career models will emerge for individuals with the requisite balance of analytical, functional and IT skills.
For those that already have the skills in-house, enterprise must focus on professional development and clear career progressions; investment in these people at the moment should be a top priority for executives.

5. Measurable Outcomes

To develop a viable big data strategy and to ensure that there will be ongoing interest and investment from decision makers, enterprises need to ensure that the case for ongoing investment is based on quantifiable business outcomes. In other words, business leaders need to be able to see the advantages.
Businesses can do this by ensuring that there is active involvement and sponsorship from one or more business leaders when the original strategy is being developed and when the first implementations take place. Also of crucial importance here is ongoing cooperation between the business and IT departments. This should ensure that the business value of all the investments in big data analytics is properly understood.
This paper and the research around it have a lot of insights for businesses that are considering their first big data deployments. Like all other IT elements in a business it requiems thorough planning before execution. This paper makes a good starting point.

inShare37

Original Source

Read More

Facebook Advertising Works - If You Don't Make These 3 Mistakes

There are still marketers who do not believe Facebook advertising works. For those people, this article is for you. By avoiding these three mistakes, learn how your brand’s Facebook advertising efforts can be effective.
I’m still surprised when I meet a person who doesn’t believe Facebook advertising is effective. It is — if you implement a few best practices and avoid common mistakes.
Last year I read an article about Facebook ads by Nicholas Carlson of the Business Insider. The article, “Facebook Ads Don’t Work Only Because Marketers Keep Making These Mistakes,” addressed key things marketers should and shouldn’t do when investing their efforts in Facebook advertising.

There were three issues Carlson wrote about in the piece that really hit home with me and I’d like to elaborate on them here.
These are the three things that anyone who is advertising on Facebook must stop doing immediately:

1. Acquiring Low-quality Fans through Incentives or Gimmicks

Brand managers often believe the more page Likes they acquire, the better. This kind of thinking is all wrong. When it comes to Facebook fans, the principle of quality over quantity applies.
Luring new fans in with advertising incentives, like seasonal deals and giveaways — and asking them to Like your page in order to access the deals — is effective, but it’s not a solid long-term marketing plan. Providing value to fans shouldn’t be a one-time thing, but rather an ongoing effort implemented by your brand.
One of the biggest misperceptions about social media is that popularity and influence are one and the same. This is simply not true.
In fact, the idea that quantity matters more than anything else is the reason that buying Likes is still in practice. It is more common for brands that are new to Facebook because purchased Likes quickly boost their Page like count. The issue with cutting corners — and buying Likes is definitely taking a shortcut — is that although a Page may have a large number of “fans,” quantity does not guarantee engagement.
In fact, brands that purchase likes often see dramatically lower engagement than pages that acquire fans organically. The reason? Pages that don’t spend the time acquiring the right kind of fans, a.k.a. quality users who have a true interest in the brand, are wasting their efforts on users who bring no value to the table.

2. Ignoring Targeting and Optimization

There are more than one billion active Facebook users, and Facebook’s Ads Manager is like a one-stop shop that lets you reach the ones who are most relevant to your business. With a decent understanding of your brand’s target market, it’s simple to reach the types of quality customers who will most likely find interest in your brand.
Facebook offers simple targeting options such as gender, age and education, but if your brand wants Facebook ads that perform really well (and who doesn’t?!), a more sophisticated targeting approach is necessary.
When creating a Facebook ad, it takes just a few more steps to take an average Facebook ad that will yield so-so results to a rockstar ad that will bring in high conversion metrics.

Step 1: Evaluate the ad image

A traditional Facebook ad that appears on the right-hand column of the Facebook homepage is tiny (100 x 72 pixels), so it’s important to have a simple image that portrays a clear message. It’s even more important to be thoughtful about the image you choose.
Before selecting an ad image, it’s best to ask the following questions: Will this image be appealing to the users I’m trying to reach? If the image were a standalone display ad, how well do I think it would perform with my target?

Step 2: Perfect the headline

After the ad image, the next place a user's focus is pulled is the bolded headline copy of a Facebook ad. Use the headline to speak directly to your target audience. Try to appeal to their needs, wants and/or desires. Often the best way to do this is by asking a simple question that’s relevant to the message of the ad.
For instance, if you’re advertising your product, instead of using your ad copy to talk about the features of the product, use copy that’s relevant to the benefits of your product. Does your target market include people who often work on deadline and are always in need of more time? Is one of your product’s benefits that it saves time? If so, an example of an effective ad headline would be: “Need a time-saving tool?”

Step 3: Target users by specific interests

The best advice when creating a Facebook ad is to always target using “precise interests.” The more specific, the better. Choose “precise interests” by identifying both broad and narrow interests. One good practice is to include competitors into the “precise interests” section. For example, if you’re Pizza Hut and you’re advertising a really large promotion, target Little Caesars' fans with your Pizza Hut ad.

3. Thinking that Facebook Ads don't Drive Performance

If your brand has done a good job creating well-targeted ads, the ads will drive performance. Carlson mentions this powerful example in his post for Business Insider: A company by the name of PetFlow gets 20 percent of its transactions and 30 percent of its new customers directly from Facebook.
The reason most people believe their Facebook ads don’t drive performance is because they’re most likely not tracking their performance the right way. After creating Facebook ads, the next most important step is to identify the way you’re going to track conversions.
Conversions metrics differ depending on the type of ad that’s running. To track a user-to-fan conversion, or the number of new fans your Page was able to earn as a result of an ad, use Facebook’s Ads Manager. To track an ad’s performance based on the number of sales you acquired, enlist the help of a third-party link tracking company like Improvely.
There is plenty of proof that Facebook advertising can be an effective tool. The key to a successful campaign is to first focus on quality and then create a calculated targeting and tracking plan. Test this advice the next time you create Facebook ads and see if your brand’s faith in Facebook advertising is renewed.

Original Article

Read More

Seven Drawbacks of Traditional OLAP

In 1993, E.F. Codd, the acknowledged founder of relational databases, introduced the term Online Analytical Processing (OLAP). OLAP is intended for the non-data processing professionals like the business expert, and is expected to be intuitive, rapid, and flexible.

 

  Intuitive user interface: clear a technical roadblock for business personnel to operate freely

  Rapid analysis procedure: accommodate the fast-changing business environment to seize opportunities and make decisions

  Flexible computing ability: able to confront many complex business computation

When the traditional OLAP product was just introduced, it experienced 2-digit growth rates in the global market for years before clients start to complain about that projects would easily fail, no obvious effect, and the time span is a bit too long. Since 2004, the growth of OLAP drops dramatically, owing to the seven major drawbacks of traditional OLAP tools(what’s the remedies of OLAP’s drawbacks?):

 

1. Pre-modeling as a must

Regarding the business data, the traditional OLAP tools do not allow for the immediate analysis without pre-modeling. These tools without a good OLAP engine cannot convert the data to a pattern in which business personnel can operate directly.

 

Assume that you are requested to analyze the profit for a telecommunications enterprise. Firstly, you need to draft a star or snowflakes model involving the date, region, client gender, client’s occupation, credit ratings, and other dimensions. Then, adapt the model to the practical database through repeated modifications. For example, remove the dimension of “customer’s occupation” from the model if it is not covered in the business data; replenish the ignored dimension of “hierarchy of consumption” to the analytical model; and use the ETL tool to synchronize the “sales network” from another database to this database. The finalized stable model will be filled with data regularly at scheduled times for use in analysis. Even so, you will find that the data is not up to the convention or short of historical data. Then, you will have to establish an additional data warehouse or data mart; In case no key data from the model is found, then you will return to modify the business system instead.

 

Modeling is a time-consuming procedure of a great many steps. Users thus have to pay the expensive cost in advance.

 

2. Great dependence on IT

Although business personnel is the intended user of OLAP, they will still have to work with the IT pros because the traditional OLAP tools requires a complex modeling procedure and its users have to write a great numbers of codes/scripts/SQL.

 

The below jobs cannot be completed without the assistance from technicians: Propose a common analytical model; Map dimensions of model to the names of fields, tables, and views of the business database; Use ETL tool or Perl scripts to migrate and integrate all required data to a same database if they are in different databases; Establish the reasonable scheduling rules, and fill data to the model by writing SQL statements or stored procedures; Deploy OLAP application to the server, and write .net/java application for user to use. Not to mention setting-up the database warehouse or data mart and modifying the business system to replenish the key data, business personnel is unable to handle without IT involvement.

 

The traditional OLAP tools depend heavily on the involvement of IT pros, and cost great human resources. What’s even worse, the IT pros without business expertise cannot fully understand the analysis goal. The model built by such IT pros may not be trustworthy enough. OLAP projects built through months or years of efforts often deviate away from the requirement of actual business, which gives rise to lots of project failures.

 

3. Poor computation capability

The data computing refers to a procedure of processing and transforming data through a series of specific steps toward a concrete goal. Data computing is a basic feature of OLAP. The traditional OLAP tools are of insufficient computational capabilities and few computational methods such as drilling, slicing, rotation, and simple column computation. This is because their architectures are old, lacking the innovation, and hard to strike a balance between user-friendliness and flexibility. Taking the below computational goal for example, it is hard to implement with the traditional OLAP tools.

 

How to find the workshop whose defective rate continuous to drop for 3 months in a row?

How to make statistics on students whose score on each course is above B?

 

In the 1st month since the new product enters the market, how many days will it takes to reach the 1/3 of the total sales in that month?

 

Lacking the computational capabilities impedes the flexibility of OLAP tool greatly. Analyzers are confined to a narrow and small area, incapable to analyze freely, and even have to resort to the 3rd party to perform this kind of computation. In the similar business computations, OLAP is often abandoned in an awkward situation.

 

4. Short of Interactive analysis ability

Data analysis is the most important feature of OLAP. Not like the data computing, data analysis is an interactive procedure requiring the perfect step-by-step computational mechanism. Toward the obscure goal, users need to observe the current data and make the reasonable assumption, and then verify/falsify the assumption to ultimately achieve the rather complicated business analysis goal. Unfortunately, the model of traditional OLAP tools is too old to provide so free a style of interactive analysis.

 

For example, why the sales improved greatly?

The possible assumptions include: orders flood in, sales forces beefed up in many respects, and any large orders are placed. Of which, the procedure to check the large orders is as given below:

1. List the order data, and then filter the data of recent 6 months.
2. Compute the average sales of each order.
3. Multiply the average sales by 300%, as the criterion of”large order” standard.
4. Group the data from step 2 by month, and filter the result from step 3 to get the large order.
5. View the result and lower the criterion for comparison if few orders are up to the criterion, and raise the criterion for comparison if there are too great orders after filtering.
6. Count the large orders of each month.
7. Compute the increment of large orders in each month.

 

Considering the above result, users may still need to keep investigating the cause of emerging large orders through computation until the clear and reliable basis for decision-making is found. The cause may be the vocational training for sales persons starts to bear fruit.

 

Traditional OLAP tools suffer from the limited interactive analysis abilities. They are unable to provide the above-mentioned flexible step-by-step computational capabilities, unable to solve the complex business computing, or provide the true basis for decision-making. So, quite a few clients just take OLAP tools as an expensive and apparently single-purpose reporting tool for presentation. They are trying to get more leverage, but only find the limited uses of OLAP for the time being.

 

5. Slow in reacting

Traditional OLAP tools require pre-molding and cooperation between people of various departments. Therefore, it is usually slow in reacting to the business analysis demands.

Assume that a toy manufacturer needs the below information before Christmas: Of the top 100 cities with the largest population, which cities require the sales effort being strengthened immediately. Then, you will find that no population data of the city in the existing model. The usage of traditional OLAP tools is as given below:

 

Business personnel download population data from census bureau or Wikipedia, and then use Excel and other similar tools to list the top 100 cities by population. The above step will not take more than 1 hour. But in the step followed, they will have to ask IT pros for help, because these data cannot be imported to OLAP for use directly.

 

Firstly, to coordinate: the business personnel raise a request to his/her superior. The superior will request the support from R&D department, and assign a project leader who will investigate the request initially, and build the project schedule and budget. The R&D department will approve and set up a development team. If cost is not a priority, then the development team can be standby all time for the business personnel.

 

Then, to implement: Perform the demand analysis in details and confirm it is to modify the model, schedule the task and export from Excel or database for other models to use. In addition, there are also steps of design, implementation, deployment, and verification as well as communication with database administers Web application administer, and programmers. When it comes to the final acceptance, further improvement is still required because IT pros may not be in the picture and are on the different page with business personnel. The improvement would be unavoidable unless unfortunately the Christmas was gone and they missed it.

 

The traditional OLAP is slow in reacting, requires great workload, and takes a bit too long time to implement the goal. Facing the fast-changing challenges, the enterprises will miss commercial opportunities, and find themselves in a disadvantage position in the intense competition.

 

6. Abstract model

The traditional OLAP tools convert the data of 2 dimensional from database and Excel to the multi-dimensional. To use the OLAP tools freely, business personnel must correctly understand the concepts of slicing, rotating, drilling, and other concepts as prerequisites. The abstraction of model hinders the business personnel from analyzing freely.

 

For example, to business personnel, the employee data, regional data, and product data are simply 3 lists or 3 sheets in Excel. Even the most ordinary business personnel know how to group, sort, and filter these data. However, things changes once these data is converted to the multi-dimensional data, business personnel have to say good bye to the data organized 2-dimensionally as flat as computer screen, and image these data as a cube. Never having the business personnel seen such data in their work or life before, they feel a bit tough to operate these abstract data.

 

The abstract model requires analyst to think 3-dimentionally. This gives rise to the difficulty to understand. Users are hard to convert the business language to the abstract multi-dimensional operations. Therefore, it is hard to truly implement the goal of on-line-analysis.

 

7. Great potential risk

Traditional OLAP tools have a huge potential risk due to the lacking of the computation and low interactive analysis ability, and the implementation relies on the cooperation with IT pros. The procedure and the cycle are a bit long.

 

The analyst are often forced to abandon the OLAP due to its poor computation ability that results in the failures to submit data of huge amount, and great difficulty to provide valuable references for the decision-maker. Lacking the interactive analysis ability, unable to solve the complex business analysis problem or find the true cause and solution, these drawbacks give rise to the faulty conclusion of analysis, and bring about great direct loss to the enterprise. In the analysis aimed to the business, too much work is relied on the IT pros. No wonder the procedure and cycle is lengthy, huge manpower and physical resources are consumed, while users are still unable to react effectively and efficiently to the constant changing business environment. Quite often, the ”stars” business is exceeded and ”cash cow” business is captured by others too. The analysis conclusion often deviates from the original goal of analysis, and the wrong decision may be easily made.

 

These potential risks can easily incur the failure of OLAP project, and bring about unrecoverable loss to the enterprise.

 

All in all, the traditional OLAP tools do not implement On Line Analytical Processing according to its true essence. They are just the “OLAP in its narrowest senses or the subset of OLAP”. It is neither intuitive nor fast or flexible.

 

• Original article

Read More

Using Data Analysis to Avoid 4 Common Causes of Business Failure

Although figures vary, it’s estimated that between half and 75% of businesses are destined to fail within the first three years. One of the startling things about the sheer number of businesses that go under in the first few years, is the fact that it’s often for entirely avoidable reasons.
All CEOs and business owners are susceptible to the same mistakes and blind spots, irrespective of the size of the business, geographical location or industry. So whether you run a local car rental business or a world-leading telecoms company, if you repeat these common mistakes your business is doomed to fail.
1. Poor Market Research
Too many businesses dive into new markets, introduce new product lines or acquire new businesses without carrying out the necessary market research or due diligence. The failure to conduct proper market analysis is a common oversight and as a result business owners and CEOs are often blindsided by their lack of preparedness in dealing with unanticipated events.
Solution:
To avoid this potentially ruinous scenario, companies should carry out a full-scale market analysis by setting up a web monitoring system that will surface any information relating to their target market: who are the main players? what is the price comparison and market performance of each competitor? In addition, companies should conduct customer sentiment analysis to discover reactions to competitor brands and to determine if there is sufficient demand for your product or service. Based on all of this data, you should have a clear idea of what level of market penetration can be achieved with your proposed new offering. The most important reason for carrying out a thorough market analysis is to avoid any major surprises which can derail your business early on.
2. Failure to Listen to Customers
Ignore your customers at your peril. It’s a fairly basic piece of business advice and, one would think, an easy one to remember. But there are so many instances where customer needs are not given the priority they deserve, that it’s still one of the number one reasons that businesses go to the wall each year. Getting a business off the ground takes a lot of focus and determination, along the way business owners and CEOs get distracted by other issues and customer needs start to slip down the list of priorities.
Solution:
One of the best ways to stay informed about your customer needs is to listen and learn about their purchasing habits, find out what trends they are following and get plugged in to the challenges they face in their daily lives. By tracking what your customers are saying about your company or brand on social media, web forums and discussion groups you have a clear idea about where to position your brand and how to anticipate evolving consumer trends.
3. Lack of Innovation
New technologies are emerging all the time and the companies who take advantage of the latest innovations are the ones who manage to stay ahead of the curve and achieve growth. Companies such as Kodak, Nokia and RIM have been severely penalized for neglecting to fight off the threat from breakthrough technologies and new innovations.
Solution:
By monitoring the latest technologies available in your market or scanning technology patents to see what’s coming on stream in the future, your business will have more than a fighting chance of surviving the next wave of innovation. Forward thinking business should set up an automated technology watch to provide answers to the following questions:

• What are the emerging technologies in our sector?
• Which technologies used in other areas or sectors could benefit us by replacing our current technologies ?
• Which technologies could be used to design new products, new packaging, new production or distribution methods?
• Could any technologies be combined to improve our products?
• How might emerging technologies affect our competitors or the arrival of new players?
• Which companies master the key technologies we have identified and could we envisage an alliance or acquisition in the future?

4. Poor Sales
Nowadays companies need to look beyond traditional sales channels to generate new customers. If your sales pipeline is running low you cannot possibly sustain your business into the future. Your growth potential is dependent on broadening and diversifying your customer base and that requires a steady stream of new sales prospects.
There is a wealth of information on the web which can be used to identify new sales opportunities. Once companies learn to harness this information correctly, the insight it will provide to front line sales teams will reap huge rewards.
Solution:
To immediately start growing the number of sales opportunities at their disposal, businesses should set up a 360 degree monitoring system which will notify sales teams of the following events:
- New investments in plants/facilities: this means the customer will be in a position to purchase new equipment.
- New products launches: this may lead to the setting up of new production lines.
- New acquisitions: will the customer modernize its acquisitions’ existing machines/equipment?
- New contracts: Will there be a subsequent increase in production?
- Personnel changes: Should you get in touch with the new R&D director that has just been appointed?
While there are no guarantees in business, companies who manage to avoid the pitfalls above can at least ensure that their business has the best fighting chance of making it in today’s turbulent economy

 This post originally appeared on Digimind's Market Intelligence Hub.

Read More