Exporting batches of UNICOM Intelligence Interviewer - Server data to IBM SPSS Statistics
In an ongoing project, you may want to export data in batches. For example, at the end of each day you may want to export all of the data collected since the end of the previous day. The UNICOM Intelligence Developer Documentation Library comes with three sample DMS files that are designed to enable you to do this. The samples are called mrInterviewFirstBatchToSav.dms, mrInterviewNextBatchToSav.dms, and mrInterviewNextBatchNewVersionToSav.dms and are designed for use in a multiversion project. There are some version changes that will require special handling, but this is described below.
The sample mrInterviewFirstBatchToSav.dms transfers the current UNICOM Intelligence Interviewer - Server data to a new IBM SPSS Statistics .sav file. The two samples mrInterviewNextBatchToSav.dms and mrInterviewNextBatchNewVersionToSav.dms use the read capability of the IBM SPSS Statistics SAV data source component (DSC) to find the existing case on the IBM SPSS Statistics .sav file that has the latest interview finish date and time (by reference to the value of the system variable DataCollection.FinishTime). Only those cases that have a later interview finish time than this are then exported from UNICOM Intelligence Interviewer - Server to the .sav file.
mrInterviewFirstBatchToSav.dms
Use this sample to export the first batch to IBM SPSS Statistics. It does the following:
▪Exports all non-test data that has the Completed status to a new IBM SPSS Statistics .sav file.
▪Saves the output metadata.
mrInterviewNextBatchToSav.dms
Use this sample to export the second and subsequent batches to IBM SPSS Statistics when there has been no change to the metadata since the last batch export. It does the following:
▪Uses the output metadata created by running the mrInterviewFirstBatchToSav.dms file as the input metadata for the export.
▪Exports to a new IBM SPSS Statistics .sav file all non-test data that has the Completed status and an interview finish time that is later than the latest case already on the .sav file.
▪This sample does not save the output metadata. This sample is designed to be run when there has been no change in the metadata since the previous batch export. Therefore there is no need to save the output metadata.
This sample fails if you run it before running the mrInterviewFirstBatchToSav.dms file. It will not transfer any records if you run it immediately after running mrInterviewFirstBatchToSav.dms without collecting any additional data in between.
mrInterviewNextBatchNewVersionToSav.dms
Use this sample to export the first batch after there has been a major version change. It does the following:
▪In the OnBeforeJobStart Event section, it extracts the most recent version of the project's .mdd file and merges it with the output metadata file used for the previous exports so that the alias names used for the previous exports will be reused wherever possible.
▪Exports to a new IBM SPSS Statistics .sav file all non-test data that has the Completed status and an interview finish time that is later than the latest case already on the .sav file.
▪Saves a new output metadata file.
Additional versions are created in the output metadata file with the result that the versions in the output metadata file will not match the versions in the project's main .mdd file. Therefore the version names stored in the case data in the DataCollection.MetadataVersionNumber variable will not correspond to the versions of the same name in the output metadata file. However, this should not present a problem when using the output metadata file for subsequent exports.
You should use this sample once each time you export the data using a new version. You should then use the mrInterviewNextBatchToSav.dms file for subsequent exports using that version until the version changes again. However, before running the mrInterviewNextBatchToSav.dms file, you will need to amend the name of the input .sav and .mdd files to reflect the name of those created by mrInterviewNextBatchNewVersionToSav.dms.
If you have amended the samples to include only a subset of variables and the version changes do not affect these variables, you would not need to run the mrInterviewNextBatchNewVersionToSav.dms file nor make the adjustments described below.
When a batch includes data collected using two major versions
You would need to adjust the process as follows:
1 Take a backup copy of mrInterviewNextBatchToSav.dms.
2 Change the InputDataSource section in mrInterviewNextBatchToSav.dms so that it selects the earlier version of the metadata and change the WHERE clause to additionally restrict the transfer to data that was collected using the earlier version. For example:
InputDatasource(mrInterviewData)
ConnectionString = "Provider=mrOleDB.Provider.2; _
Data Source=mrRdbDsc2; _
Initial Catalog=" + InputMDDFile + "; _
Location="Provider=SQLOLEDB.1; _
Integrated Security=SSPI; _
Persist Security Info=False; _
Initial Catalog=short_drinks; _
Data Source=LocalHost"; _
MR Init MDM Version=7; _
MR Init Project=short_drinks"
SelectQuery = "SELECT * FROM vdata _
WHERE (DataCollection.FinishTime > '@LastFinishTime') _
AND (ContainsAny(DataCollection.Status, {completed})) _
AND (DataCollection.MetadataVersionNumber = '7') _
AND NOT (DataCollection.Status >= {test})"
End InputDatasource
3 Run the modified version of mrInterviewNextBatchToSav.dms.
4 Run mrInterviewNextBatchNewVersionToSav.dms to transfer records from the later version.
5 Restore the backed up copy of mrInterviewNextBatchToSav.dms.
See also