UNICOM Intelligence interview scripting guidelines
Overview
This section contains guidelines, recommendations, and best practices for improving the performance and maintainability of UNICOM Intelligence interview scripts.
Script writing rules and recommendations
▪Follow the error handling guidelines that are outlined in
Dealing with errors. For example, include an error handler in all scripts.
▪Use custom validation functions to avoid Goto statements.
The Goto approach makes the script more difficult to follow and maintain. For example, the following example shows a Goto statement that is used to re‑ask the question when the constant sum validation fails. Each attempt to answer the question is stored in the interview history, which requires respondents to press Previous multiple times when they back up in the survey.
ConstantSumQuestion:
ConstantSumGrid.Ask()
Dim category, total
total = 0
For Each category in ConstantSumGrid
total = total + category.Item[0].Response
Next
If total <> 100 Then
ConstantSumGrid.Errors.AddNew("ConstantSumError", _
IOM.Questions.ConstantSumError.Label)
Goto ConstantSumQuestion
End If
In the following example, the constant sum validation is implemented as a custom validation function.
ConstantSumGrid.Validation.Function = "ValidateConstantSumGrid"
ConstantSumGrid.Ask()
Function ValidateConstantSumGrid(Question, IOM, Attempt)
Dim category, total
For Each category in Question
total = total + category.Item[0].Response
Next
If total <> 100 Then
Question.Errors.AddNew("ConstantSumError", _
IOM.Questions.ConstantSumError.Label)
ValidateConstantSumGrid = False
Else
ValidateConstantSumGrid = True
End If
End Function
The use of custom validation functions also makes it easier to reuse validation logic across multiple questions.
▪Use Enums instead of numeric values, or string literal values, to improve readability. For example, use TerminateStatus.tsScriptStopped instead of1.
The IOM and MDM library type definitions are registered for use within interview scripts. For Enums from libraries that are not registered, either define the Enums as Constant within the script, or use a comment that indicates the Enum name. For example:
.Properties[ "Autocommit Isolation Levels" ] = 256 ' READ_UNCOMMITTED
▪Do not ask questions in Functions or Subs. Restarts do not work correctly when restarts occur on the proceeding question. The script in the Function or Sub, that occurs after the .Ask() statement, is not run.
The only exception to this rule is when the .Ask() statement is the last statement that is run in a Sub (only a single question or page can be asked). The following script is acceptable:
Sub AskPromotionQ(IOM)
Dim score
score = IOM.Questions.FinalScore
If score >= 100 Then
IOM.Questions.PromotionHigh.Ask()
ElseIf score >= 50 And score < 100 Then
IOM.Questions.PromotionMedium.Ask()
Else
IOM.Questions.PromotionLow.Ask()
End If
End Sub
The following example will not restart correctly after a timeout or restart:
Sub AskPromotionQs(IOM)
Dim score
score = IOM.Questions.FinalScore
If score >= 100 Then
IOM.Questions.PromotionHigh.Ask()
ElseIf score >= 50 And score < 100 Then
IOM.Questions.PromotionMedium.Ask()
Else
IOM.Questions.PromotionLow.Ask()
End If
' NOT recommended. Only a single Ask statement
' can appear in a sub-routine and it must be the
' last statement executed
IOM.Questions.PromotionFinal.Ask()
End Sub
The .Ask() statement must never be included in subroutines.
▪When MyQuestion is a variable (not known in advance), use name-based lookup in collections to avoid linear searches. For example:
IOM.Questions[MyQuestion].Ask()
Most UNICOM Intelligence collections support name-based look-ups.
When Q1 is a defined question, use Q1.Ask() as it is always faster than IOM.Questions[“Q1”].Ask().
▪Use IsNullObject to check when an object is NULL. Comparing an object to
NULL returns
True when the default object property returns
NULL. Use
IsNullObject to check whether the object is
NULL, without expanding the default property. See
IsNullObject for more information.
▪Use FindItem to search for an item in a collection that might or might not exist.
FindItem uses name-based collections look-ups. For more information, see
FindItem.
The following example best demonstrates how to search for an object in a collection. Instead of using the following script:
Function CheckIfQuestionExists(IOM, questionName)
Dim i
For i=0 to IOM.Questions.Count-1
If LCase(CText(IOM.Questions[i].QuestionName)) = LCase(questionName) Then
CheckIfQuestionExists = True
Exit Function
End If
Next
CheckIfQuestionExists = False
End Function
...use the following instead:
Function CheckIfQuestionExists(IOM, questionName)
' FindItem returns NULL if the question doesn't exist
' Return True if something other than NULL was returned
Dim question
Set question = FindItem(IOM.Questions, questionName)
CheckIfQuestionExists = Not(IsNullObject(question))
End Function
▪Use the Browser Capabilities property collection to determine whether the browser supports JavaScript. The practice is more efficient than using the browser client to determine JavaScript support, avoids issues with browser scripting incompatibility, and keeps the logic within the survey routing. Properties that are not covered by the Browser Capabilities property collection (for example, a client’s IP address) must still be collected by other means. For more information, see BrowserCapabilties IInterviewInfo properties collection.
▪Avoid storing XML in CHJ cache files. In some cases, projects create large CHJ cache files. The large size is the result of the XML file information, that is read by the script, being stored in the CHJ cache file (sometimes multiple times). For example, the following script reads the XML and stores its full content as a text string.:
set fso=CreateObject("Scripting.FileSystemObject")
set thisFile = fso.GetFile("c:\myinputfile.xml") 'get file
set txtStream = thisFile.OpenAsTextStream()'open for reading
set file=txtStream.ReadAll()
txtStream.Close() 'close data stream
set myXML=CreateObject("Microsoft.XMLDOM")
myXML.LoadXML(file)
The state of the interview routing script is stored in the CHJ cache file after each .Ask() statement, in cases where the interview needs to be stopped and the restarted at a specific position, or the user navigates back to a question. Each time the file variable is assigned in between two .Ask() statements, the script stores a copy of the file value at the .Ask() statement. Some cache files store the information multiple times, which dramatically increases the file size.
The issue can be avoided by clearing the file variable prior to the next.Ask() statement (using file=""). The variable state is small when it is captured at the .Ask() statement. There is no need to read the file into a string because the XMLDOM can read directly from the file, which avoids the need for the file variable. The XMLDOM object is not stored in the cache file. The routing script can be simplified as follows:
set myXML=CreateObject("Microsoft.XMLDOM")
myXML.load("c:\myinputfile.xml")
Performance guidelines
The guidelines for improving interview script performance are as follows:
▪Place scripts, that need to run only one time, outside of the loop when using loop logic. Placing scripts outside of the loop is important for scripts that are time intensive, such as scripts that create a database connection or retrieve a file from a web server. For example, instead of using the following script:
Dim category
For Each category in Q1.Categories
' The database connection is recreated for every category
Dim connection_string, connection
connection_string = "Provider=MSOLEDBSQL.1; Data Source=" + _
DATASOURCE + ";Initial Catalog=" + DBNAME + _
";Integrated Security=SSPI;Persist Security Info=False"
Set connection = CreateObject("ADODB.Connection")
connection.Open(connection_string)
' Use the connection for this category
...
Next
Use the following script instead:
' Create the database connection once for all categories
Dim connection_string, connection
connection_string = "Provider=MSOLEDBSQL.1; Data Source=" + _
DATASOURCE + ";Initial Catalog=" + DBNAME + _
";Integrated Security=SSPI;Persist Security Info=False"
Set connection = CreateObject("ADODB.Connection")
connection.Open(connection_string)
Dim category
For Each category in Q1.Categories
' Use the connection for this category
' ...
Next
▪Remove redundant script. When you remove questions from an interview script, the tendency is to remove just the Ask() statement without removing the logic that is used to initialize the question.
▪Set the style, validation, and response constants in the metadata section when they apply to all routing contexts. For example, use the following script:
' Set the control type style property in the metadata section
Names "Names" loop
{ use NameList } fields (
Types "Types" loop
{ TypeList use \\.TypeList } fields (
Rating "Rating"
style(Control(Type = "DropList")) categorical [1..1] { RatingList use \\.RatingList };
) expand;
) expand;
Avoid the following script:
' Where possible, avoid setting styles in the routing script
Names[..].Types[..].Style.Control.Type = ControlTypes.ctDropList
Setting properties a single time in the metadata avoids the need to set the property for every loop iteration. The practice improves routing script performance and reduces the size of interview cache file (the property changes do not need to be saved to the cache file).
Set the properties in the routing section when styles or properties are conditionally set, or apply only to specific routing contexts.
The following sections provide more performance guidelines.
Sharing data between surveys
A single sample table can be used to share participant information between several projects. Information that must be shared between multiple projects can be stored in the sample table, rather than in the respondent data. The practice is typically called project chaining and can be used to avoid connecting to another project’s data from a survey. Connecting to another project’s data causes the other project’s metadata to be loaded for each interview. This behavior negates the efficient use of memory that is provided by the project object cache. The additional connections are also not cached, so connections are not managed as efficiently as the connections managed by the system. Therefore, connecting to another project’s data must be replaced, where possible, with the project chaining strategy.
The overall steps are as follows
1 The respondent clicks a URL to start the MAIN survey.
2 The URL either contains all required authentication information, or the respondent is asked to supply the missing information.
3 The system calls AuthenticateSampleRec to find the sample record and validate the authentication information. The special chaining code recognizes the initial project, typically due to a Project sample field that is Null or because the project is initiated with a specific value.
4 The MAIN project starts, and the respondent steps through the survey. The MAIN survey writes any sample information that might be required by later projects in the survey chain.
5 The respondent completes the survey.
6 The best next project for the respondent can be calculated in either the interview or sample management scripts. Updates are stored in the Project sample field. It might be easier to place this logic into the sample management script as all projects can then use the same script (all chain logic is stored in one script). Placing all logic in a single script also allows the sample management script to be updated without activating with the UNICOM Intelligence Interviewer - Server Admin Participant Rules activity (if necessary). However, the sample management script must be updated for all projects because the script is not shared, even though the sample management table is shared.
7 The system calls ReturnSampleRec to return the sample record. There might be some special Serial handling required. Each record in the chain can be assigned the same Serial, which can lead to nonconsecutive serial numbers in each project. Multiple Serial sample fields might also be required and the ReturnSampleRec script must be updated to properly store the Serial.
8 The end of interview page contains a redirect, or a URL, that points back to the MAIN project or to the next project in the survey chain. The respondent is redirected, or clicks a control, to continue.
9 The system calls AuthenticateSampleRec to find the sample record and validate the authentication information. The special chaining code recognizes that the record is in theCOMPLETED queue, and checks the Project field to decide whether the chain is complete or if the next project in the chain must be started. If a different project must be started, the project's InterviewProperty is updated with the appropriate value.
10 Repeat steps 4 - 9 for extra projects, as required.
More changes are required for restarts (to support starting the correct project with the correct Serial value).
Project chaining
The following changes are required to support project chaining:
1 Activate with shared sample.
2 Redirect from one project to another.
3 Update the sample management script to allow next project to be started.
4 Share sample data between projects.
Each of the changes are discussed at a conceptual level and with the aid of a simple working example.
Activate with shared sample
Activate the main project with sample management and an uploaded sample table. A special sample management script is required.
Activate the other projects with sample management. Specify the main project’s sample table for the sample database, and use the same special sample management script as the main project.
Redirect from one project to another
Either the interview or the sample management script can be used to select the respondent's next project. When the interview script is designed to select the respondent's next project, a change must be made to all scripts that start other interviews.
The scripts must also setup a redirection at the end of the interview. The redirection can be an HTML redirect, or you can create a control at the end of interview page that links to a URL. A template can make the EndOfInterview page resemble a normal interview page, which leads respondents to believe that they are taking one large survey rather than a chain of smaller surveys.
Redirects must be setup with the EndOfInterview text or a banner on the end of interview page, rather than on the last interview page. When the interview redirects from the last page, the main interview stays active until the timeout period (the interview timeouts rather than completing). The sample record is not returned until a later time, which results in an indeterminate sample state.
In addition to using sample fields, you can also pass some interview information on the URL (I.User1 to User9). The parameters are available to the interview script as IOM.Info.UserN.
Update the sample management script to start the next project
The following items must be taken into account when you update the sample management script to support chaining:
▪Allow authentication from the COMPLETED queue (depending on the current segment of the chain).
▪Track the current project and when the chain ends.
▪Track multiple serial numbers (if necessary).
▪Start and restart the correct project with the correct serial number.
The sample management scripts that are delivered with UNICOM Intelligence support simple web chaining between two project, and can be used as an example.
' +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
' ToDo - Set bChaining to True to enable chaining for web projects
' If chaining is enabled a FIELD_PROJECT SampleField must be included and the
' proper project names specified as PROJECT1 and PROJECT2. In addition, extra
' serial fields must be specified As FIELD_SERIAL1 and FIELD_SERIAL2.
' These settings are used in AuthenticateSampleRec.
Dim bChaining
bChaining = False
Const FIELD_PROJECT = "Project"
Const PROJECT1 = "Project1Name"
Const PROJECT2 = "Project2Name"
Const FIELD_SERIAL1 = "Serial1"
Const FIELD_SERIAL2 = "Serial2"
Share information between projects
Any information that is collected in one project, but required in another project, must be written into the sample record. For information on strategies for reading and writing categorical data, see
Working with sample records and
Converting data to the required format.
The amount of data that is shared must be limited to what is necessary. However, the system can handle many sample fields. A large project that is used for load testing contains 37 sample fields. However, there are instances when over 250 sample fields are used. System performance depends on the size of each field. Many sample fields can be supported when fields are sized efficiently.
Reduce the number of variable instances
Avoid creating loops with many iterations, or nested loops with many iterations at each level. The number of variable instances for nested loops is the product of the number of elements at each level.
As a guide, if the total number of variable instances for a loop construct exceeds 10,000, reconsider the way that the loop is structured. The AnalyzeMDDs.mrs script can be used to extract summary information, including the variable instances for existing MDD files. This script can be run before a project is activated, or nightly on the staging cluster to identify projects with large loops that might require restructuring. You can also use the activation events to validate the metadata.
To understand how the variable instances add up, consider the following example.
The relatively simple set of nested loops creates 7,886 variable instances. Adding Q6 to grid Level4 increases the variable instance count to 11,726. When you investigate reducing the loop size, start at the top of the loop by reducing iterations, or remove unused iterations and questions, and work down your way through the levels.
Investigate the following criteria:
▪Reduce iterations.
▪Use database questions.
▪Store an iteration identifier when each respondent answers a small iteration percentage.
▪Remove unused questions and categories.
▪Use nocasedata when data is not used directly in analysis.
Reduce iterations
Database loop questions are one approach for reducing the number of loop variables instances. Database questions create iterations for only the maximum number of iterations that are asked, not for all iterations. For more information, see
Database questions. The basic concept is that a numeric (or text) identifier is stored instead of storing a category value. No additional routing script changes required. The analysis of database questions is slightly different as the derived
DBCodes variable is used during analysis.
Alternatively, the technique that used by database questions for creating a numeric loop and storing the iteration category in a nested question, can be used. The following example uses Level2 from the previous example:
Level2 "Level2" loop
{
Type1 "Type1",
Type2 "Type2",
Type3 "Type3",
Type4 "Type4",
Type5 "Type5",
Type6 "Type6"
} fields -
(
Q2 "Q2"
categorical [1..1]
{
Yes "Yes",
No "No"
};
When the respondent answers only two of the six possible iterations, or Level2, then the Level2 loop can be redefined to contain only two iterations. Making the change reduces the number of variable instances from 11,726 to 3,914.
Level2 can be redefined as a categorical loop with categorical placeholders for only the categories that are used. The method can be useful when the entire Level2 loop is asked, as you can setup the iteration labels for substitution.
Level2 "Level2" loop
{
_1 "{Iteration1}",
_2 "{Iteration2}"
} fields
(
Q2 "Q2"
categorical [1..1]
{
Yes "Yes",
No "No"
};
Level2 can also be redefined as a numeric loop:
Level2 "Level2" loop [1..2] fields
(
Q2 "Q2"
categorical [1..1]
{
Yes "Yes",
No "No"
};
In either case, add a question inside the loop that holds the iteration value:
Level2 "Level2" loop
{
_1 "{Iteration1}",
_2 "{Iteration2}"
} fields -
(
IterationID "Iteration identifier"
categorical [1..1]
{
Type1 "Type1",
Type2 "Type2",
Type3 "Type3",
Type4 "Type4",
Type5 "Type5",
Type6 "Type6"
};
Q2 "Q2"
categorical [1..1]
{
Yes "Yes",
No "No"
};
Set the IterationID response in the routing script before asking the questions in the loops.
During analysis, you will need to use the iteration categorical question. Instead of dropping the entire loop/grid onto the table, drop the IterationID question on the top, and the required questions on the side.
Remove unused questions and categories
Remove a question from the metadata when the question is removed from the routing, or a category is replaced. The question or category is still available for analysis when the superversion is used. The superversion does not impact the current version, which is loaded into memory when the project is cached. The current version is used to create the value cache.
Use nocasedata when data is not used in analysis
When there are questions in your loops that are only used for temporary logic, mark the questions as nocasedata to avoid creating variable instances for the questions.
nocasedata questions are not available during the analysis phase.
Considerations for interview restart
The interview routing script is rerun, and uses answers from the case data, when an interview is restarted (either because the respondent restarted the interview through sample management, or because the interview failed over to another interviewing engine). Rerunning the script enables the entire interview history to be reconstructed, which allows the respondent to navigate to previous questions. The behavior is important when the routing script interacts with other systems (for example, when records are inserted into databases or when sending emails). In special situations, it might be necessary to bypass the script with
If Not(IOM.Info.IsRestart) Then. For more information about what happens with quota on timeouts and interview restarts, see
Quota control in restarted interviews.
External objects
▪Ensure that external objects are created, and used between Ask() statements, when external libraries are used. On restart, the script after the previously asked question is rerun to ask the question that is restarted. Do not reuse an external object that is created before the previous Ask() statement. The object is not available when a restart occurs. In the following script, the XmlHttp object is not available when a restart happens on Q2:
Dim XmlHttp
Set XmlHttp = CreateObject("MSXML2.ServerXMLHTTP")
XmlHttp.open("GET", URL, True)
XmlHttp.send()
Q1.Ask()
ServerWaitForResponse(XmlHttp, 5)
If (XmlHttp.readyState = 4) Then
If (XmlHttp.status = 200) Then
Q2.Label.Inserts["FileData"] = XmlHttp.responseText
End If
Else
XmlHttp.Abort()
Debug.Log("Timeout waiting to retrieve '" + URL + "'. Status = " + _
XmlHttp.statusText, LogLevels.LOGLEVEL_ERROR)
End If
' NOTE. Restart on Q2 would fail as XmlHttp would be NULL
Q2.Ask()
▪Set 1-dimensional arrays onto regular variables. Arrays defined in script are not stored in the cache for restart. In the following example, if a timeout and restart occurs between Q1 and Q2, the contents of MyArray is lost. The work-around is to set the array onto a regular variable. This only applies to 1-dimensional arrays and, once set, the array on the regular variable is read-only. The array can still be read via index, for example, Var = CachedArray[1], but the array cannot be set.
Dim MyArray[], CachedArray
MyArray[0] = "AAA"
MyArray[1] = "BBB"
MyArray[2] = "CCC"
MyArray[3] = "DDD"
CachedArray = MyArray
q1.ask()
Debug.Log("MyArray = " + CText(MyArray) + ", Cached = " + CText(CachedArray))
Debug.Log("End of Q1 Post")
q2.ask()
Debug.Log("MyArray = " + CText(MyArray) + ", Cached = " + CText(CachedArray))
Debug.Log("End of Q2 Post")
If a regular variable is not used, and you move back in a survey or restart it, the following error might occur:
Execute Error(<line_number>): Subscript out of range accessing the array variable '<array_name>'
▪Use the OnInterviewStart event to create global objects for external objects that must be used across multiple survey questions. For more information, see
Objects.
▪Encapsulate creation and access to external objects in a Function or Sub. By encapsulating the creation and use of external objects in subroutines, the objects are destroyed as soon as the routine exits. Objects that are created in the main body of the script exist until the interview completes or times out. Encapsulating logic where external libraries are used also helps avoid objects that are used across multiple Ask() statements. The main routing logic is easier to understand and maintain after moving complex logic into functions.
▪Avoid creating or loading large strings in the main routing script. Temporary variables that are declared at the script level are stored in the cache file. Therefore, assigning large strings to temporary variables can result in large cache files. The following code reads a 20 MB file into the file variable, which causes the cache file size to increase by 20 MB when the temporary variable is stored.
Set fso = CreateObject("Scripting.FileSystemObject")
Set thisFile = fso.GetFile("C:\Contacts.xml") ' Get file
Set txtStream = thisFile.OpenAsTextStream() ' Open for reading
Set file = txtStream.ReadAll()
txtStream.Close() ' Close data stream
Set myXML = CreateObject("Microsoft.XMLDOM")
myXML.LoadXML(file)
Clear the variable before the next Ask() statement, or avoid creating the string, to avoid storing the temporary variable into the cache:
Set myXML = CreateObject("Microsoft.XMLDOM")
myXML.load("C:\Contacts.xml")
Alternatively, you can encapsulate the reading and processing of large strings in a Function orSub. Variables that are declared in subroutines are not stored in the cache file.
▪Use external libraries that are intended for server-side use. Microsoft Office automation must not be used from within the survey script. For example, when the XMLHttpRequest object is used, use ServerXMLHTTP instead of XMLHTTP.
In later releases, it is preferable to use the HTTP function library (pointer to documentation). shown in the following example. Requests within the same 30 second interval use the same underlying TCP connection, so caching the HttpRequest object in IOM.Objects is not required.
Function GetDataFromURL(URL)
Dim HttpRequestObject, HttpResponseObject
Set HttpRequestObject = CreateHttpRequest(URL)
HttpRequestObject.Timeout = 5 * 1000 ' Request will timeout after 5 seconds
On Error Resume Next
Set HttpResponseObject = HttpRequestObject.Send()
On Error Goto 0
If (Err.Number <> 0) Then
Debug.Log("Unable to retrieve '" + URL + "'. " + Err.Description, LogLevels.LOGLEVEL_ERROR)
Else
GetDataFromURL = HttpResponseObject.Content
End If
End Function
The following example demonstrates how to load content through URLs by using ServerXMLHTTP:
Function GetDataFromURL(IOM, URL)
Dim XmlHttp
Set XmlHttp = SetXmlHttpObject(IOM)
XmlHttp.open("GET", URL, True)
XmlHttp.send()
ServerWaitForResponse(XmlHttp, 5) ' Request will timeout after 5 seconds
If (XmlHttp.readyState = 4) Then
If (XmlHttp.status = 200) Then
GetDataFromURL = XmlHttp.responseText
End If
Else
XmlHttp.Abort()
Debug.Log("Timeout waiting to retrieve '" + URL + "'. Status = " + _
XmlHttp.statusText, LogLevels.LOGLEVEL_ERROR)
End If
End Function
Function SetXmlHttpObject(IOM)
' First, try to get the object created in OnInterviewStart
Set SetXmlHttpObject = FindItem(IOM.Objects, "XmlHttp")
If IsNullObject(SetXmlHttpObject) Then
' Create the XMLHTTP object if is hasn't already been created
Set SetXmlHttpObject = CreateObject("MSXML2.ServerXMLHTTP")
End If
End Function
Sub ServerWaitForResponse(XmlHttpRequest, TimeoutInSeconds)
' Avoid long waits in waitForResponse due possible stack overflow
Const POLL_INTERVAL = 250
Const POLLS_PER_SECOND = 4 ' with 250 ms poll, poll 4 times per second
Dim Attempts, MaxAttempts
MaxAttempts = TimeoutInSeconds * POLLS_PER_SECOND
' Call waitForResponse to force the readyState to update
XmlHttpRequest.waitForResponse(0)
While XmlHttpRequest.readyState <> 4 And Attempts < MaxAttempts
Attempts = Attempts + 1
Sleep(POLL_INTERVAL)
XmlHttpRequest.waitForResponse(0)
End While
End Sub
Sub OnInterviewStart(IOM)
Dim XmlHttp
Set XmlHttp = CreateObject("MSXML2.ServerXMLHTTP")
IOM.Objects.AddNew("XmlHttp", XmlHttp)
End Sub
▪When possible, avoid polling external objects from within the script. For example, in the previous script, the waitForResponse method might have been used instead of polling the readyState property in a loop. Unfortunately, in the case of waitForResponse, an unresolved defect in the Microsoft marshaling code can cause the interviewing engine to crash due to a stack overflow.
▪Use a Dictionary object to avoid passing arrays. For example, in a data management script you can initialize the Dictionary object in the OnJobStart event.
Event (OnJobStart, "Load ID Map")
Dim IdMap
Set IdMap = CreateObject("Scripting.Dictionary")
#include "LoadIdMap.mrs"
dmgrGlobal.Add("IdMap", IdMap)
End Event
The LoadIdMap.mrs file loads the map.
IdMap.Add("ID1", "{Age_20To29}")
IdMap.Add("ID2", "{Age_30To39}")
IdMap.Add("ID3", "{Age_20_29}")
...
It is easy to load the data from an external file when loading the dictionary instead of hardcoding the additions to the map. Then in OnNextCase, you just need to add something like the following:
Dim MapValue
MapValue = dmgrGlobal.IdMap.FindItem(Respondent.ID)
If (MapValue IS NOT NULL) Then
AgeQ = CCategorical(MapValue)
End If
Database access
Guidelines for improving the performance and maintainability of database access from the interview script are as follows:
▪Use database questions when a database is queried for a category list, for a question or loop, or for large lists that are common across projects. From a performance standpoint, the main advantages of database questions is that a single connection is established to the database for the project (instead of a separate connection for each interview). The category list can also be cached for the project (instead of being reread for each interview). Database questions can also be analyzed in UNICOM Intelligence Reporter. For more information, see
Database questions.
When other-specify categories are required, display the database question on the same page as a text question. You can also display the other-specify response text box on a subsequent page when the respondent selects the other-specify category in the database question. UNICOM Intelligence does not fully support other-specify responses in database questions.
Instead of using the following example:
resState "What state do you currently reside in (i.e., your primary residence)?"
categorical [..1]
{
_1 "Alabama" [value = 1,Region = "AL"],
_2 "Alaska" [value = 2,Region = "AK"],
_3 "Arizona" [value = 3,Region = "AZ"],
_4 "Arkansas" [value = 4,Region = "AR",Open_end = true],
_5 "California"[value = 5,Region = "CA"],
_6 "Colorado"[value = 6,Region = "CO"],
_7 "Connecticut"[value = 7,Region = "CT"],
_8 "Delaware"[value = 8,Region = "DE"],
_9 "District of Columbia"[value = 9,Region = "DC"],
etc...
Use the following example instead:
residentialState "What state do you currently reside in (i.e., your primary residence)?" text
db(
ConnectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=%PROJECT_DIRECTORY%\Location.accdb;Persist Security Info=False",
Table = "State",
MinAnswers = 1,
MaxAnswers = 1,
CacheConnection = False,
Columns(
ID = "StateID",
Label = "StateLabel",
AnalysisValue = "StateAnalysisValue"
)
);
When asking follow-on questions, such as the residential county depending on the state, use another database question for the county and the SQLFilter property to filter the responses according to the selected state.
Use a username and password in the connection string when the database is not accessible by the UNICOM Intelligence Anonymous User.
Use only OLE DB providers that are intended for multiple-user access in your production environment You can use single-user access OLE DB providers in your development environment.
▪When ADODB objects are used, follow the guidelines for external objects.
▪Unless they are added to IOM.Objects in OnInterviewStart, do not reuse ADODB objects between question asks.
▪When possible, encapsulate logic in a subroutine.
▪Call Close() on Connection and Recordset objects after database access is complete. When a database connection is opened, and not closed within the main body of the interview script, the connection remains open for the duration of the interview. Alternatively, ensure all database connections usage occurs within a Function or Sub. Doing so ensures that objects are automatically closed when the Function or Sub exits. For example, Close statements are not required in the following example, because the objects are released when the Function exits:
Function GetDbValue(dbserver, db, query)
Dim connection, recordset
Set connection = CreateObject("ADODB.Connection")
connection.Open("Provider=MSOLEDBSQL.1;Data Source=" + dbserver + _
";Initial Catalog=" + db + _
";Integrated Security=SSPI;Persist Security Info=False;")
connection.Properties["Autocommit Isolation Levels"] = 256 ' READ_UNCOMMITTED
Set recordset = connection.Execute(query)
If Not recordset.EOF Then
GetDbValue = recordset.Fields[0].Value
End If
End Function
▪When querying databases follow normal database guidelines. Index tables and profile queries to ensure the indexes are used and use stored procedures when possible.
▪Avoid OLE DB providers that are intended for single-user access. For example, avoid the Access Data Engine with Microsoft Excel files. The Access Database Engine is not intended for server-side applications. Interview routing scripts are run on a server concurrently, with a number of other routing scripts. Therefore, the external objects that are used must be designed for server side concurrent usage. The Access Data Engine is one example of an object that is not designed for server side usage. As such, Provider=Microsoft.ACE.OLEDB.12.0 must not be used in an ADO connection string.
▪Consider connecting with the “Read uncommitted” isolation level when querying against tables that are under frequent update. The method can result in a dirty read, it is preferable to blocking updates. In particular, when reading data directly from the case database or the sample table, set the isolation level after the connection is opened. For example:
connection.Properties[ "Autocommit Isolation Levels" ] = 256 ' READ_UNCOMMITTED
▪When possible, run a single query to retrieve multiple values instead of running multiple queries to retrieve a single value at a time. For example, instead of using the following script:
Function GetDbValues(dbserver, db, query, minId, maxId)
Dim connection, recordset
Set connection = CreateObject("ADODB.Connection")
connection.Open("Provider=MSOLEDBSQL.1;Data Source=" + dbserver + _
";Initial Catalog=" + db + _
";Integrated Security=SSPI;Persist Security Info=False;")
connection.Properties["Autocommit Isolation Levels"] = 256 ' READ_UNCOMMITTED
Dim id
For id = minId to maxId
Set recordset = connection.Execute(query + " = " + CText(id)) If Not recordset.EOF Then
GetDbValues = GetDbValues + CText(recordset.Fields[0].Value)
End If
GetDbValues = GetDbValues + ";"
Next
End Function
Use the following script instead:
Function GetDbValues(dbserver, db, query, minId, maxId)
Dim connection, recordset
Set connection = CreateObject("ADODB.Connection")
connection.Open("Provider=MSOLEDBSQL.1;Data Source=" + dbserver + _
";Initial Catalog=" + db + _
";Integrated Security=SSPI;Persist Security Info=False;")
connection.Properties["Autocommit Isolation Levels"] = 256 ' READ_UNCOMMITTED
Set recordset = connection.Execute(query + " BETWEEN " + _ CText(minId) + " AND " + CText(maxId)) While Not recordset.EOF
GetDbValues = GetDbValues + CText(recordset.Fields[0].Value) + ";"
recordset.MoveNext()
End While
End Function
Testing and monitoring interview performance
The guidelines for testing surveys and for monitoring interview performance are as follows:
▪Use the UNICOM Intelligence Interviewer Server Load Tool to run a test load against projects before they are put into production. Use the timing statistics to identify any survey sections that exhibit poor performance. The Load Tool produces log files. Check the logs and verify that there are no warnings or errors. The auto-answer feature in UNICOM Intelligence Professional and UNICOM Intelligence Author does not produce log files.
▪Any questions that, on average, take more than 0.5 second to answer must be investigated.
▪Set the MaxOpCodesExecuted registry setting to 10,000 in the test environment and check for max-opcodes errors. Running more than 10,000 opcodes between Ask() statements is a good indication that the script might run poorly under load.
For more information, see
MaxOpcodesExecuted.
▪Check for long-running or complex script sections by checking the number of opcodes that are run between save-points. Use the ExportOpCodeCounts.mrs script to export the number of opcodes that are run at each save-point for a completed CHJ interview cache file. The output can be used to determine the number of opcodes that the scripting engine ran to get from one Ask() statement to the next Ask() statement. The information can be used to determine the routing script complexity and provide a rough guide as to the amount of CPU load the script generates from one question to the next.
▪On the web tier, monitor the Server Failed Requests counter to detect errors when survey pages are processed. When errors occur, check the web tier's IVW logs for timeouts. Timeouts for specific projects or questions provide an indication that a survey has errors or poor performance.
▪Monitor the database tier performance counters. Look for full table scan, high percentage disk use, or a disk queue length greater than two.
▪Use the standard SQL Server query performance reports to identify specific queries that contribute load on the database tier.
Survey presentation
▪If possible, avoid major customizations for look and feel. Look and feel changes typically require extensive use of JavaScript.
▪Create subfolders for the different file types. Separate the JSP, HTML, and CSS files in different folders. When you localize templates and files, according to language, store the files in the appropriate language subfolders. For more information, see
Layout templates.
▪Use mrRef tags for all references to external files that are particular to a project. Use the
mrSharedRef tag for common files across multiple projects. For more information, see
Referring to external files.
▪When possible, use a single layout template for the entire survey. Using a single layout template standardizes the look and feel, and reduces the occurrence of missing elements (such as not inserting a named banner) or page errors. Use subtemplates and avoid the use of client-side scripting when there are question layout and look and feel variations within the survey. For more information, see
Sub-templates.
For example, instead of inserting JavaScript into the metadata:
bankLoyalty "<script>$(document).ready(function()
{$('input[type=""text""].mrEdit').after('<span class='mrRightText'> years</span>');});</script>
How long have you been a customer of ABC Bank?<I>(Please consider how long you have been with the same
financial institution, even though the name of your primary financial institution might have changed
several times over the years due to mergers or acquisitions. Enter 0 if less than one year.)</I>"
long [0 .. 90]
codes(
{
_99 "Don't Know"[value = 99]
DK
} );
Use the following script instead:
bankLoyaltyInfo "How long have you been a customer of ABC Bank?<I>
(Please consider how long you have been with the same financial institution, even though the name of
your primary financial institution might have changed several times over the years due to mergers or
acquisitions. Enter 0 if less than one year.)</I>"
info;
bankLoyalty " years" templates(Question = "BankSubTemplate.htm")
long [0 .. 90]
codes(
{
_99 "Don't Know" [value = 99]
DK
} );
bank ""
page(
bankLoyaltyInfo,
bankLoyalty
);
Where the BankSubTemplate.htm file would look like this:
<mrSubTemplate>
<table>
<tr>
<td><mrData QuestionElement="Controls"/></td>
<td valign="baseline"><mrData QuestionElement="Label"/></td>
</tr>
</table>
</mrSubTemplate>
▪When possible, include only styles in a style sheet and avoid inserting styles into the template or routing. It is a good idea to separate the business logic in the metadata and routing with the presentation of the questionnaire. The practice not only ensures the metadata document is clean and can work with many data collection modes, but it also ensures style changes are evident in regards to look and feel. When you setup templates for the questionnaire presentation, keep in mind that layout templates must be only used for layout. The survey style must be in separate files in one or more style sheets. You must ensure that all tags in the layout template, that are required for formatting, have an ID. The style sheets refer to the tag IDs.
The only time formatting is specified in the routing or metadata is when there are changes to formatting within a question (for example, bolding a certain word in text or changing the look-and-feel of a grid question’s components). When formatting is applied in the routing, the formatting must exist in its own subroutine, which allows it to be easily separated from the business logic in the rest of the code.
▪Use IOM.Info properties instead of hard-coding values. For example, when the respondent is redirected back to the survey, use IOM.Info.InterviewStartURL and do not hard-code the URL. When you refer to files, use the mrRef or mrSharedRef tags, and do not use the full URL when it is on the same server. When the survey is setup to use Secure Sockets Layer (SSL), accessing files from sources other than the current web server results in browser warnings. The rules also ensure that the survey works when it is ported to another environment.
▪Use two single quotation marks, to represent a single quotation mark, in the metadata labels For example, the word don't must be written as don''t. When axis expressions are created for questions, single quotation marks are used to delineate category labels. When there is a single quotation mark in the string, the software interprets it as the end of the string, which results in a syntax error. When any of the UNICOM Intelligence products encounter two single apostrophes in a string, they display it as one apostrophe (treating the first apostrophe as an escape character).
▪Use the FullQuestionName setting to obtain a question element in a subtemplate. For example:
var QuestionInput = document.getElementById("<mrData QuestionElement="Parameter"
Type="FullQuestionName"/>)
Sample management
▪Always activate projects with sample management when there is predefined sample (even when sample comes from a Panel Vendor and only IDs are stored). Sample management allows respondents to restart surveys when the surveys are stopped partway through. When the project accepts URL (i.usern) parameters, a new record is always created, and the interview restarts (even when a Panel Member intends to return to a partially completed survey).
▪Create a function or subroutine to centralize sample management within the survey script. The function or subroutine must account for instances where field lengths are shorter than the sample that is passed to the survey. When field lengths are shorter than the sample, sample data must be truncated to the maximum field length and an error must be written in the log. Respondents are allowed to complete the survey.
▪To make it easier to switch between Panel Providers, standard sample fields must be created for projects that work with Panel Providers. Any redirect links back to the Panel Providers must also be specified in the same location as the centralized sample management function or subroutine.
Managing survey assets
▪Use a Question Repository to store and manage survey assets. The IBM SPSS Collaboration and Deployment Services Repository integrates with UNICOM Intelligence products. The repository is useful for storing and accessing survey assets (whole surveys, parts of a survey, survey wizards, and any associated files). The repository tracks version changes, restricts access (depending on the team that works on them), and is preferable to using a shared folder in a multiple user environment.
Limits
You might want to verify the following your project's size and scope. Some projects might require extra resources, resulting in decreased performance for other projects. The following table lists some of the project size attributes that can be tracked.
Project size attributes
Attribute | Guideline | Comments |
|---|
Opcodes between questions | 10,000 | The limit can be set to a smaller value based on cache file analysis. |
Variable instances | 20,000 | It is more difficult to measure the impact of variable instances when unexpanded loops are used. |
Sample records (Web) | 1 million | Authentication fields must be indexed. |
Sample records (CATI) | 100,000 | |
Quota cells – Total | 7,500 | |
Quota cells – Per matrix | 1,000 | |
After you define your own limits, you might want to use a script to regularly check that activated projects follow the guidelines. You can also check the limits before activation. For more information, refer to the AnalyzeProjects.mrs script that is installed with the UNICOM Intelligence Developer Documentation Library at:
[INSTALL_FOLDER]\IBM\SPSS\DataCollection\7\DDL\Scripts\Interview\Utilities
The number of opcodes that are run can be tracked in UNICOM Intelligence Professional. Running opcodes can be analyzed with cache files. For more information, refer to the AnalyzeOpCodes.mrs script that is installed with the UNICOM Intelligence Developer Documentation Library at:
[INSTALL_FOLDER]\IBM\SPSS\DataCollection\7\DDL\Scripts\Interview\Utilities
See
