The UNICOM Intelligence Data Model: An introduction for survey research executives

Software should help to free your business of constraints--not add them. It should facilitate the adding of value and uniqueness to your offering in the marketplace--not get in your way. What we as software suppliers must do, is create an environment in which many different ways of working can be planned and executed.

The industry is now beginning to see growing client demand for ever more complex analysis and data mining, customer relationship management (CRM), Web-based publishing of interactive databases, and executive information systems and marketing information portals that integrate multiple data streams. As we rise to meet these challenges, we need the tools to overcome the fundamental inefficiencies that arise from the difficulty of integrating and adapting applications, sometimes from multiple vendors, so that we can create comprehensive, cost-effective solutions for our research clients.

Software products are merely tools in the construction of processes and solutions. A supplier of software tools may offer advice and consultancy on best practices and innovative thinking, but cannot, and should not, prescribe or limit a way of working. The UNICOM Intelligence products are a growing set of tools that enable the creative implementation, management, and adaptation of survey research processes. At the core of the UNICOM Intelligence architecture is a technology called the UNICOM Intelligence Data Model.

The key to the UNICOM Intelligence architecture and its benefits lies in the UNICOM Intelligence Data Model. The UNICOM Intelligence Data Model is a methodology (in programmer's terms, an “API”) for managing data of differing formats, structures, and storage mechanisms, so that applications that use the UNICOM Intelligence Data Model can read data from or write data to any source, regardless of the storage technology. The user application is written using a standard interface, and the UNICOM Intelligence Data Model hides the details of the underlying, package-specific storage architectures. Thus, the application can “know about” many data formats, and be insulated from any changes to them, without the programmer even knowing that they exist.

Further, the UNICOM Intelligence Data Model provides functions to handle tasks that are common to many market research applications. It gives the programmer the “hooks” to handle, for instance, multiple response questions, hierarchical data sets, tracking of study versions, and simple aggregation, simply and quickly, so that this functionality need not be reinvented for each study. This helps free the programmer to focus on the unique aspects of the application--where the value is added.

The UNICOM Intelligence Data Model is not a data repository, nor is it a data format or schema for storing data. It is a system that gives programmers the ability to efficiently develop applications that need not be concerned with, or limited by, any particular data format. This frees the programmer to adapt applications to new and unforeseen scenarios without the need to spend time dealing with how data is formatted.

There are already numerous examples of this technology in action, of which UNICOM Intelligence Reporter - Survey Tabulation is just one. It is an easy-to-use tabulation package that has been specifically designed for the market research industry. It does not understand, or even need to know about, the format of the underlying data. Knowledge of any particular data format is contained in the relevant data source component (DSC). You can think of the DSC is an “adapter” that describes a specific data format to the UNICOM Intelligence Data Model. This means that data in any format for which a DSC is available, can now be analyzed using UNICOM Intelligence Reporter - Survey Tabulation without the need for any changes whatsoever in UNICOM Intelligence Reporter - Survey Tabulation itself. In fact, no additional work is required by the user at all. The additional data formats just come “for free” with the DSC.

The UNICOM Intelligence Data Model is at the hub of the UNICOM Intelligence Reporter - Survey Tabulation architecture. Each UNICOM Intelligence Reporter - Survey Tabulation product is, in effect, a spoke that communicates with all of the others and with third-party applications through this hub.

The key benefit for the survey research community will lie beyond simple transparency of data access. Whether your company adopts a package that meets your current needs or invests in extensive, proprietary development, you'll want to know that the software supports your ability to customize and extend your systems, products, and business processes--with our help, with one of our partners, or on your own.

A second real-world example might help to illustrate this last point. The construction of paper questionnaires for scanning often involves some fairly extensive formatting, as well as a time-consuming step in which the fields on the paper questionnaire are mapped into the scanning software. We've combined third-party software with our UNICOM Intelligence Data Model and a proprietary application to create a solution that automates much of the formatting--and reduces the mapping process from one that often requires a day or more, depending on the questionnaire, to a matter of minutes. Further, changes to the questionnaire can be handled automatically, both in the formatting and in the mapping to the scanning software. This saves multiple iterations of the manual mapping process, and enables agencies to be more flexible and responsive.

In general, the biggest data processing beneficiaries to date are those who deal with data from numerous different data collection sources. Instead of converting, remapping and exporting data, and creating a myriad of new files, you can now access these disparate data sets directly with any analytical tools that are UNICOM Intelligence Data Model-enabled (such as UNICOM Intelligence Reporter - Survey Tabulation). We are already analyzing data seamlessly from UNICOM Intelligence Interviewer Server, Quantum and Quanvert, and IBM SPSS Statistics, and other UNICOM Intelligence data formats, such as Surveycraft, are set to follow shortly. Interest has also been shown by a number of outside organizations including competitors, the Triple-S Committee, and a number of market research companies that have their own proprietary systems. This developing consensus behind the UNICOM Intelligence Data Model will position it, we believe, as an industry standard for the interchange of market research data.

Perhaps less obvious, but potentially more far reaching, are the benefits that can be derived by the development community. The UNICOM Intelligence Data Model is not only time, cost, and labor saving in itself, but also an open system enabling developers to add functionality and new capabilities to the market research process using standard tools, systems, and applications with remarkable ease.

“I cannot emphasize strongly enough just how much is possible. I am not a programmer, never have been, and choose not to become one, yet I have written, with negligible expertise and in a very short time, at least half a dozen mini-applications in Visual Basic that manipulate market research data and perform useful additional functions. Examples include a mini-application that counts words in a questionnaire authored in In2form or Quanquest for use when calculating payments to translators (it took three hours to develop); a tool that exports a spec file for Quinput (two days' work); an example application that uses UNICOM Intelligence Interviewer - Paper and Word to generate a printed questionnaire from data in any data format (such as IBM SPSS Statistics .sav file, Quanvert database, etc.) that took two hours. Others around me have done many other exciting things (probably far more efficiently!) and have demonstrated these publicly at trade shows, conferences, and training courses. Some of these are even given away free as examples in the UNICOM Intelligence Developer Documentation Library (DDL), downloadable from the World Wide Web. See, for example, the source code that is included in the DDL of a simple tool that can display data (including multipunched data) in any format (whether that is IBM SPSS Statistics, Quanvert, UNICOM Intelligence Interviewer Server, or any other third-party data file) in an Excel spreadsheet.”

The UNICOM Intelligence Data Model is “open” in two ways: first, it is built around open, industry-standard (Microsoft) technologies; and second, it provides an open, published interface (API) for the use of one and all. The first allows UNICOM Intelligence Data Model-enabled programs to interact smoothly with a large body of other standards-based software, allowing them to be combined easily as components of more complex systems. The second allows others outside UNICOM Systems, Inc.--clients, partner companies, and even competitors--to easily take advantage of the benefits offered by the UNICOM Intelligence Data Model.

The use of standard Microsoft technologies provides efficiency benefits at every level--whether you're simply loading a data set into Excel with a few lines of VBA, or you're creating a complex tracking environment or a marketing information portal. Moreover, because you can work with survey data independently of the underlying storage format, you are not tied to any one product or software supplier. The result is greater flexibility and improved business processes.

The UNICOM Intelligence Developer Documentation Library (DDL) is a free resource which UNICOM Intelligence makes available on the World Wide Web to developers who wish to utilize the open interfaces of the UNICOM Intelligence infrastructure in order to develop or customize tools, features, interfaces, or bespoke applications. We wish to welcome everyone, including our competitors, to join us in opening up to our customers this new world of efficiency and data transparency.

UNICOM Systems, Inc. is supporting industry-standard efforts wherever possible. In the market research area, our director of development has worked with the Triple-S committee, and will contribute in any practical way to the converging of our UNICOM Intelligence Data Model with this published standard.

First, consider an integrated, global data system for a 50-country customer/partner satisfaction project. The survey is just one element in a global CRM effort designed to build loyalty and satisfaction among channel partners of a very large information technology manufacturer. Surveys are conducted in partners' native languages, and the questionnaire contains both standard, global elements, and questions customized for regional operations.

▪Integrating multiple data collection modes and third-party software packages to provide efficient merging of data for fast, combined operational reporting and for extensive analysis without incremental data preparation and conversions. This challenge alone can add considerable operational cost using traditional approaches.

Second, consider a data mining project for a supermarket chain that combines behavioral information from a loyalty card program and volume data from point-of-sale systems with surveys conducted on Palm Pilots and/or kiosks on the retail floor. The UNICOM Intelligence Data Model will make survey data from UNICOM Intelligence or third-party data collection systems available to analytic tools such as Clementine, where it can be merged with other data streams and used to analyze and predict buying patterns. In this way, retail floor and shelf layout, item selection, and clustering can be optimized to maximize high-margin and overall volume, and highly targeted promotional strategies can be evolved to increase customer value.

Moving to more operational issues, question/questionnaire databases, or libraries, have been built by a number of research agencies and manufacturers to reap the benefits of standardization, in terms of both measurement consistency and process efficiency. In this context, the UNICOM Intelligence Data Model provides a single place in which to store--for any particular question--specifications and text information needed for:

Finally, the UNICOM Intelligence Data Model can also help research agencies to better understand and--as a result--more efficiently manage their operations through rapid process management reporting. One field and tab organization, for example, is using the UNICOM Intelligence architecture to integrate data from interviewing log files, interviewer time clock logs, an employee timesheet database, and an interviewing scheduling system. The UNICOM Intelligence Data Model makes this data available to a batch program that generates management reports on:

Additional UNICOM Systems, Inc. modules are to be deployed to assist in interviewer scheduling and productivity analysis, thus improving facility utilization. This firm is generating more effective management information more rapidly and distributing it more effectively via their intranet thanks to the UNICOM Intelligence Data Model and the UNICOM Intelligence architecture.

Our goal at UNICOM Systems, Inc. is to build on the deep, long-term partnerships we have with our clients. We must earn and sustain your trust--you must know that we as a company are out to help you deliver the most effective possible solutions for your customers. UNICOM Systems, Inc. has a clear-cut commitment to open standards and systems, in order to allow our customers maximum flexibility to extend and combine our products to create differentiated solutions that, in turn, add value for their customers. More and more, UNICOM Systems, Inc. products are offered both as turnkey solutions and as components with published programmer's interfaces, as developers' tools that support our clients in the endeavor to innovate.

This isn't the traditional thinking in this industry. Companies have felt that protecting their secrets meant protecting their client bases. Today, however, it is important to leave this kind of tactical thinking behind. As the market research industry changes, as it faces threats from consultancies and from low-cost and DIY research, as demand emerges for more and more sophisticated data warehousing, analytic, and marketing information applications, the stakes are far too high. Our clients will--and should--be intolerant of company policy that “traps” them within a product line and limits their flexibility to innovate.

It is important to all of us, as an industry, to think strategically about open systems and to support industry standards. By adopting standard technologies, by publishing our API's, and by working with industry standards organizations, we can help our customers to compete effectively in this changing marketplace.

UNICOM Intelligence is a technology platform which includes an integrated suite of software products for all aspects of the market research process. The core of this strategy is to place the data at the center of the whole process in an open format, to promote wider use of that data throughout the enterprise. The open UNICOM Intelligence platform is also designed to provide clients with fully scalable, customizable tools that allow greater process efficiency and competitive innovation.

Businesses of all sizes drive their success using UNICOM Intelligence software and expertise for data collection, data management, and insightful analysis and reporting. With a background in market research and business intelligence, UNICOM Intelligence offers an extensive technology suite with unrivalled breadth and depth, proven consulting skills, and a truly global service through our international network of offices.

Behind the new UNICOM Intelligence Data Manager functionality will be a new scripting language for the automation of the above tasks as well as a vehicle for the technical user to develop more reusable routines for customized tasks that may be repeatedly needed in any particular installation.