Final Report

Model Deployment of a Regional,
Multi-Modal 511 Traveler Information System

7.0 The Enhancement Process

This section presents the results of the enhancement process analysis. The analysis focuses on identifying management and deployment issues encountered in planning, implementing, and operating the enhanced 511 system, including lessons learned.

7.1 Overview of the Enhancement Process

The 511 Model Deployment enhancement process consisted of three phases. Figure 7-1 identifies the phases and corresponding milestones. The planning and design of the system began in August 2002, with the first meeting of the 511 project committee (511 Task Force). That committee continued to meet throughout the duration of the Model Deployment. As indicated in Figure 7-1, the design, implementation, and operation phases overlapped somewhat, as enhancements were designed, implemented, and refined over time. Although the major design document—the System Requirements Definition Report—was completed in March 2003, the design of some enhancements, including the menu, continued on for several months. Likewise, after the main roll out of the enhanced system in mid-December, 2004 (thus concluding the main implementation effort), further refinement of implemented enhancements (such as voice recognition) and implementation of additional enhancements (such as the call transfer to the Arizona Office of Tourism) continued throughout 2004. Although the formal one-year Model Deployment operations period ended in December 2004, design and implementation of some enhancements continues.

Figure 7-1. Enhancement Process Timeline

7.2 Enhancement Process Findings

This section presents the perspectives of the 511 participants, based on their interview and workshop input. These findings are not the overall conclusions of the evaluation and, in a few cases, are not supported by the usage and user survey evaluation analyses presented in Sections 4.0 and 5.0 of this report. This is not unexpected since the interviews and workshops occurred before evaluation results in these areas were available. Findings are presented in several topic areas, as listed in Table 7-1.

Table 7-1. Organization of Enhancement Process Findings

Topic Area	Topics Covered
General Management, Deployment, and Operations Issues	Schedule and phasing Budget and costs Public and private partners
Successes	Overhaul of the user interface Increased ADOT capture of arterial street incident data Increased capture of freeway incident and congestion information New data types implemented Increased call volumes Increased (statewide) 511 coverage Data quality tools and processes System performance tracking capabilities Focal point for continued regional cooperation
Challenges and Unresolved Issues	User interface New data types not implemented
Overall Lessons Learned	Management and deployment issues Technical issues
Next Steps for Arizona 511	Continued pursuit of planned Model Deployment enhancements Other planned enhancements

7.2.1 General Management, Deployment, and Operations Issues

Schedule and Phasing

1. The overall implementation schedule was tight and played a role in postponement of some of the enhancements – Design and implementation took longer than anticipated. It was not possible to design and implement all of the planned enhancements fully within the original, approximately 15-month, implementation period. ADOT always planned to phase in the enhancements. However, the schedule also played a role in delaying some enhancements, as ADOT found they were unable to devote enough manpower, simultaneously, to complete all of the enhancements on time. With a more protracted schedule, allowing for greater phasing, available implementation staff resources would have been sufficient. Apart from these obvious impacts on the timing of enhancements, project participants voiced few concerns about the overall Model Deployment timeline. The IVR consultant noted that the schedule provided very limited time for testing of the voice recognition system. In fact, considerable testing and refinement of the enhanced user interface occurred "in the field," in the first month or two after the system was deployed. Project participants' opinions vary as to the extent to which the "refinement-on-the-fly" approach was an unfortunate result of schedule constraints or an effective way to maintain momentum and get the product in the hands of users. The only other specific schedule concern was noted by the ADOT Information Technology Section Manager, the technical lead for the 511 project, during the early stages of planning and design. At that time, he was somewhat concerned that the short schedule did not allow for a competitive procurement of a traveler information voice recognition specialist, but rather necessitated ADOT's use of their existing IVR consultant, whose transportation-related IVR expertise was less certain. Later, however, this person indicated that he was very pleased with the work of their existing IVR consultant.
2. Phasing of enhancements is inevitable – As noted in the preceding item, some phasing of enhancements was always envisioned. Although a tight time frame appears to have delayed some enhancements, most of the project participants did not find schedule constraints to be a major concern. Generally, they find phasing and even some delays to be an expected, inevitable aspect of large, complex technology implementations.

Budget and Costs

3. ADOT has not identified budget as a significant concern – Although the cost of several enhancements significantly exceeded estimates, ADOT has not identified any concerns related to the overall budget. This is consistent with the findings of the cost analysis (Section 8.0) that, assuming the remaining Model Deployment enhancements can be completed on-budget, the cost of the Model Deployment will exceed the original estimate by only about 4% ($52,000).
4. Increased ADOT 511 program staffing costs not identified as a concern – ADOT has not identified 511 program staffing costs as a concern, although the staff-related operational costs of the system have clearly increased (see Section 8.0). This may be because none of the costs are "new costs" per se, but rather represent marginal increases in pre-enhanced 511-related staff costs. It may also reflect the fact that the Model Deployment has not created any new, strictly 511, staff position and therefore cost increases may not be obvious. (In assembling post-enhancement cost data, ADOT indicated that it was necessary to estimate the ADOT staff time associated with 511 because that information is not formally recorded).
5. Phone charges have not been a concern but may be an issue in the future – Average daily call volumes increased about 74% after implementation of the 511 enhancements. ADOT has not expressed any concern about the associated increase in phone charges—about 8.5%, from about $129,000 in 2003 to $140,000 in 2004. However, ADOT notes that significant increases in call volumes and phone charges may become an issue. One factor that could contribute to escalating phone charges is an increase in calls from the Tucson area, which will be long distance. ADOT indicates that if such call charges become a concern they would consider decentralizing the 511 IVR, placing one or more phone servers in Tucson to eliminate the long distance call (to Phoenix, where all of the phone servers are now centralized).

Public and Private Partners

6. Project momentum cements agency commitments and fosters participation – Several project participants—both at ADOT and at other agencies involved in the Model Deployment—noted that it is difficult to get partner agencies fully engaged before the basic user interface enhancements are completed. This includes the partners who formally committed to participate during the proposal stage. This was generally attributed to a (not unexpected) "wait and see" attitude. One ADOT staffer indicated that the obvious "holes" in the 511 system associated with any given agency, which are only apparent once the basic menu enhancements have been implemented, help stimulate participation by those agencies.
7. Potential for private partnerships for premium services is very limited – ADOT received only one responsive proposal from a private premium service provider. Upon investigation, that provider did not appear to be sufficiently financially stable and ADOT did not pursue a partnership. ADOT feels that the experience in trying to find a private sector partner indicates that there is currently little interest among private providers, reflecting the very limited perceived profit potential.
8. Leveraging existing data sharing agreements with commercial information providers is useful but a small part of the solution – Both ADOT and the City of Tucson currently partner with commercial traveler information providers. In ADOT's case, the provider is allowed to operate from the ADOT Traffic Operations Center, in exchange for access to their traveler information. In the case of Tucson, the private partner actually operates the regional traveler information system on behalf of the City. It is too early to tell in Tucson because they have not begun inputting data to the 511 system, but in Phoenix, it appears that the information from the commercial provider does not constitute a significant or comprehensive source of information.

7.2.2 Successes

This section identifies what the project participants believe are the major successes of the Model Deployment. Topmost among those successes are the major redesign of the user interface and the addition of several new types of information to the 511 system.

9. The significant overhaul of the user interface is one of the most significant accomplishments – Nearly all of the project participants point to the major redesign of the user interface as the single most important accomplishment of the Model Deployment. The redesign included conversion to voice recognition and segment- and region-based ("Quick Reports") reporting of roadway conditions. These profound changes brought the previously very simplistic and outdated ADOT 511 user interface up to the state-of-the-practice level. However, as indicated in Section 5.0, a significant number of 511 users are dissatisfied with the voice recognition system.
10. ADOT is inputting much more arterial street information – ADOT followed through with their plans to significantly increase their commitment to monitoring police scanners for arterial street incident information. The Model Deployment increased the number and quality of the scanners and, where possible, an additional HCRS operator tasked with monitoring scanners has been added to key shifts. As indicated in Section 2.2, the number of arterial street events entered by ADOT into the 511 system has increased ten fold, from 234 entries (2% of total entries) to 2,763 entries (14%) per year.
11. ADOT is inputting much more incident and congestion information – Although not an explicit Model Deployment objective, the volume of roadway congestion and incident information input to 511 by ADOT has increased significantly. This may have been stimulated by the opinions expressed by focus group participants that more of this type of information was desirable. Entries pertaining to roadway congestion (level of service) increased from 181 per year (representing 1.5% of all HCRS entries) to 3,999 per year (representing 20% of all HCRS entries.) Entries pertaining to incidents/accidents increased from 1,686 per year (representing 14% of all HCRS entries) to 5,306 entries per year (representing 35% of all HCRS entries.)
12. Several new types of data have been added to the system – Second only to the major redesign of the user interface, project participants point to the addition of new types of data to the 511 system as a major project success. These additions include Amber Alert information, a call transfer to the Arizona Office of Tourism, information on all of the regional/rural transit providers statewide, airport information, and transit information. These additions have fundamentally transformed the 511 system from its previous highway-orientation to a multi-modal information resource. (However, as noted in Section 4.0 (usage analysis) and Section 5.0 (survey analysis), the new, non-highway data are utilized by very few users, and therefore in terms of usage, the system remains very much a roadway-oriented resource.) Also, even though local agencies are not yet making much use of it, establishing the capability for local agencies to securely enter information directly into HCRS via the Internet is an important accomplishment related to new data.
13. Call volumes have increased steadily, seemingly in part due to enhanced marketing, while sufficient phone line capacity has been maintained – Total annual 511 calls increased about 95% between 2003 and 2004, from about 345,000 calls to about 670,000 calls. At the same time, as indicated in the analysis of system usage (Section 4.0), acceptable system availability has been maintained.
14. Gains have been made in ensuring data quality – ADOT feels that important actions have been taken to support continued enhancement of data quality. These actions include re-training of ADOT operators, including significant refinement in the use of landmarks in referencing roadway information, and implementation of a "preview' capability in HCRS. That feature allows HCRS operators (who are instructed to use it) to see how their event entry will be converted to an actual advisory message on the 511 system. This is critical because HCRS operators do not literally compose the advisory message, but rather the message is automatically composed drawing upon various event content input by the operator. In the future ADOT plans to enhance the preview capability so that the operator can hear what the 511 advisory will sound like.
15. System performance tracking capabilities have been greatly enhanced and an overall philosophy and process for continuous quality improvement have taken root—Prior to the Model Deployment, ADOT was able to collect only total hourly call volumes for the 511 system. There was no capability to analyze usage by menu selection and there was no mechanism for user input-ADOT conducted no surveying or focus groups and had no formal caller comment capability. The Model Deployment has greatly enhanced capabilities in these areas. The overall focus on the 511 system and user perspectives has ushered in a new philosophy of continuous quality improvement and customer service. Whereas ADOT previously only had one monthly report available (hourly call volumes), they now generate a number of reports on a monthly basis, including enhanced call volume tracking and system performance (e.g., outages). With their new capabilities, ADOT plans to monitor 511 usage by menu selection (e.g., roadways versus transit, etc.) and repeat callers in the future. The implementation of the caller comment voicemail feature in 511 has provided a powerful tool for user input and is being utilized significantly by ADOT to continuously improve the system and manage customer satisfaction. The new caller comment feature was very important in guiding the intensive refinement of the voice recognition and menu system in the couple of months following the initial roll out. Finally, the Model Deployment significantly increased the 511-related activities conducted by the Public Information Officer, including institution of the first formal 511 marketing ever carried out by ADOT.
16. The enhanced 511 system provides a working model and focal point for continued regional ITS operational coordination – The AZTech Metropolitan Model Deployment Initiative in the late 1990's was successful in bringing many agencies from throughout the Phoenix region together to coordinate in the area of ITS. However, the unifying influence of AZTech diminished over the years, in part because some of the coordinated activities were abandoned (a number of them were demonstrations not necessarily envisioned as long term activities). To some extent, regional coordination suffered from the lack of a clear, concrete, sustained application to "bring the players to the table" to partner in actual ITS operations. The 511 system has provided such a concrete, long-term coordinated application. Coordination has been enhanced through specific Model Deployment enhancements, such as the local agency Internet HCRS entry capability, as well as the general dialog on regional traveler information and agency data sharing that the 511 Model Deployment has fostered, and which will continue.
17. The 511 Model Deployment vision establishes a good target and will continue to facilitate improvement of the system – Several of the project participants feel strongly that even though not all of the planned enhancements have been implemented, the 511 Model Deployment vision is still a very viable and a very good one, noting that only one enhancement has been formally dropped (private partner premium service). They feel that the objectives of the Model Deployment will continue to provide guidance to system enhancements into the future.
18. 511 system coverage has increased substantially, to full statewide coverage – Prior to the Model Deployment there were many gaps in 511 service statewide, including many cell phone providers who did not support 511. As part of the overall 511 Model Deployment effort ADOT has greatly increased service coverage. Coverage now is essentially statewide.
19. The 511 system was never "down" for any significant period of time, despite the major changes made to the system – Despite modifying nearly every major component of the 511 system, the system remained operational. Although there were a number of complaints about the very early version of the menu system and voice recognition, the system remained fully operational and available to callers throughout the Model Deployment.

7.2.3 Challenges and Unresolved Issues

This section discusses challenges encountered in the 511 Model Deployment, approaches used to address those challenges, and unresolved issues. Although, as described in Section 7.2.2, user interface and new data type enhancements represent some of the most important successes of the Model Deployment, they also are recognized by the participants as the areas where the greatest challenges were encountered.

New Data Types

20. Collecting data from new partner agencies is challenging – The Model Deployment experience indicates that it can be difficult to collect data from agencies who do not have the overall responsibility for operating the 511 system and who have not historically contributed data to the system. These challenges are both technical and institutional in nature. Examples of technical challenges include the difficulties in establishing a secure means for local agencies to access the ADOT secure intranet to input HCRS data, and the difficulty of Valley Metro (Phoenix transit) in exporting bus schedule status information from their vendor's proprietary system. In terms of institutional challenges, the issues are primarily related to a lack of support of higher management for traveler information, which manifests itself in a lack of resources to support what is viewed as a new, unfunded, operational responsibility.
21. The relationship between Arizona 511 and established transit customer service lines is clarifying, but is not yet completely clear – No definitive relationship between the two services—which could be seen as overlapping in so much as 511 is intended as a multi-modal service—has been established, but the Model Deployment has been successful in stimulating coordination and has established some parameters for how the systems may relate. In this sense, the Model Deployment has not answered the "big questions" for transit and 511 around the country, e.g., can 511 replace traditional transit customer information lines, and how much transit information should be put directly on 511? However, the Model Deployment has brought ADOT and the transit agencies together and begun to establish a framework for how those questions may be addressed in Arizona. Currently, both Phoenix and Tucson transit agencies plan to implement their own IVR systems. Phoenix transit cites three reasons for continuing to pursue their own IVR: they (and ADOT) do not believe 511 is ready to handle the massive volume of transit information calls; many of the calls will still require a skilled transit customer service operator to address; and they have spent many years and considerable resources firmly establishing their existing phone numbers as the comprehensive source of transit information. Tucson transit has the same concern about call volume. They also need to integrate their IVR with their demand-response trip reservation system, which would be more difficult to link with via 511. Although Phoenix and Tucson transit plan to retain and even partially automate (implement IVR) their own phone services, both transit agencies are finding 511 as a useful extension and supplement to their own traveler information activities. Phoenix transit in particular has a substantial number of call transfer options in 511, corresponding to the many types of services they offer. As they implement their own IVR they plan to further integrate 511 and their systems, e.g., transfers directly from various portions of the 511 menu into various portions of their own IVR menu. Phoenix transit feels 511 may be a particularly useful resource for new residents and visitors who are more likely to come to the area knowing about 511 and less likely to know the transit customer information number.
22. Little use has been made of the ability to record information on major transit service disruptions – Although both Phoenix and Tucson transit do have general messages referring 511 users to the option to transfer to their customer service operators, they have not recorded much information related to major service disruptions. In this sense, little to no actual transit information content has been made available through 511. It is unclear whether there is simply little need or benefit to providing such information or whether the agencies have not embraced 511 as an outlet for such information dissemination. Conversations with Phoenix transit indicate the latter is more likely than the former. They note that there are not very many system-wide events and that it would be difficult to note the disruptions that impact only certain routes.
23. Significant work remains in the area of arterial street data collection, and a number of significant challenges must be overcome – As described in Section 2.2, local agencies are not yet inputting information to 511 in any significant volume. So far, only a handful of Phoenix area agencies have input a combined total of less than 50 HCRS entries pertaining to arterial streets. Nearly all of those entries have pertained to scheduled (e.g., construction and maintenance) rather than real-time events (e.g., congestion and accidents). Local agencies indicate that they lack the resources to input this information, and in the case of planned events, which most of them enter into their own internal databases (for internal schedule coordination purposes, generally), they wish to avoid having to enter the information twice; once into their own system and once into HCRS. Some local agencies feel that the lack of resources reflects a general lack of support for traveler information activities on the part of their agency leadership. Traveler information is, as they put it, "a tough sell" to senior agency and political leadership who want tangible results and infrastructure they can point to. One local traffic agency representative noted that in an environment where even something as tangible as an arterial street dynamic message sign is difficult to get funded, it is next to impossible to engender support for a relatively "invisible" regional traveler information service, especially when it is operated by the state department of transportation.
24. A lack of comprehensive and accurate roadway incident information greatly limits the ability to provide this information through 511 – One of the reasons that good real-time information on roadway incidents is not entered into 511 by local agencies is that they have very little of this information themselves. The situation is the same for ADOT when it comes to arterial streets, and only somewhat better for interstates. ADOT synthesizes interstate incident information from a variety of sources, including their own closed-circuit television camera observations, monitoring of law enforcement scanners, and reports directly from the highway patrol (Department of Public Safety). However, they know that these sources of information do not accurately capture all notable incidents.
25. Little to no information content was added for Tucson during the Model Deployment, although the stage is now set for bringing Tucson Region data into 511 – Three of the Model Deployment enhancements focused on adding Tucson region information to 511, essentially making the former Phoenix and intercity-centric 511 system truly more of a statewide resource. To this point, those enhancements are not yet bearing much fruit. Neither Tucson transit or the Tucson airport have taken much advantage of the ability to record traveler information summaries on 511, and the City of Tucson traffic department is only now poised to begin inputting data to the system. It is too early to say whether the traffic data inputting will be successful. In the case of transit, and with regard to the airport it is unclear why 511 has not yet been viewed as a significant resource.
26. Technical challenges have greatly delayed the arterial street travel time enhancement – The major challenge was related to communications. The problem has been in getting the data from the license plate readers located in the field (which gauge traffic speeds by matching individual vehicles at the beginning and end of specific arterial street corridors) back to the system server in ADOT. It took the better part of 2004 to implement the field elements and develop an effective wireless communications strategy. The system is now operating in test mode but no information is yet provided via 511.
27. Data availability and format concerns have slowed the segment weather and neighboring states' data enhancements – Neither of these enhancements were completed within the Model Deployment evaluation time frame. In the case of the weather data, much of the delay was due to the fact that the National Weather Service data product that ADOT had intended to use was not, at least early on, the product they had envisioned. To a lesser extent the postponement of this enhancement also seems related to the fact that ADOT found they lacked the staff resources to simultaneously implement all of the planned enhancements and therefore postponed some of them. In the case of other states' data, the early plan to include Utah roadway information in the 511 system was abandoned after it was determined that the Utah data was categorized at a much higher level and would be difficult to mesh with the more finely categorized Arizona data. Instead, ADOT is awaiting the development (through an effort led by other states) of an import-export utility between HCRS and the other major national commercial alternative to HCRS, the "Condition Acquisition and Reporting System," CARS. In reflecting on the experience with the "other state's data" enhancement overall, The ADOT 511 project manager noted that it is harder when, as is the case with Arizona, there are no large cities on either side of the state borders.
28. No significant gains appear to have been made in the area of Downtown Phoenix special event and parking information – The original vision was somewhat ambiguous in this area. It was not clear to what extent the enhancement would consist of merely increasing the volume of information of the sort already in the 511 system (downtown Phoenix sporting events) through traditional mechanisms (information acquired and entered into the system by ADOT personnel), and to what extent the enhancement focused on getting information directly from the new Downtown Phoenix Parking Management System, such as real-time parking occupancy information. Regardless of the vision, it does not appear that the volume of information or the mechanism for inputting this information has changed. Also, based on interviews with both ADOT and the City of Phoenix it does not appear that the City is any more engaged in this activity than they were in the past.

User Interface

29. The transition to voice recognition constitutes a major accomplishment as well as one of the major challenges, and opinions vary somewhat on the process used and final quality – Some participants feel the voice recognition deployment was somewhat rushed and could have benefited from more extensive bench testing. Others feel the tight schedule added needed urgency and that the only way to get good feedback is to let users experience the system. Similarly divergent opinions exist as to the adequacy of the final performance of the voice recognition system. Nearly everyone agrees that continued refinement will be beneficial. Where they differ is the extent to which the current system is acceptable. The same participants who feel that the roll out was not premature cite the fact that the system was greatly improved in the first couple of months of operation (largely in response to caller criticisms and suggestions) and feel that, although it could be better, it is now generally effective. Those who felt the roll out was somewhat rushed tend to acknowledge the improvements that have been made but find the voice recognition capability still in need of attention. To the extent that they agree that improvements would be desirable, both groups generally feel that background noise is the major issue. In theory, background noise causes the system to misinterpret user utterances, either attributing the background noise incorrectly as a user input or failing to recognize the user input because of the background noise.
30. An obvious and comprehensive touch tone back-up capability is important, especially to veteran 511 users – As originally deployed, the enhanced 511 system had a partial touch tone menu capability and the touch tone option was not prominent. Within the first month or so of operation there were many negative caller comments complaining that there appeared to be no touch tone option and expressing considerable frustration with the voice recognition feature. A number of the callers appeared to be experienced 511 users, who seemed surprised to find the user interface so changed and their familiar touch tone interface seemingly eliminated. In response to the complaints, the touch tone feature was enhanced to provide a fully redundant, parallel method of interface and the option was moved to a more prominent location in the menu.
31. "Power users" were a driving force in refining the voice recognition system – As noted in the preceding item, negative comments from experienced 511 users played a role in refining the touch tone menu option. These "power users"—veteran 511 users intimately familiar with the menu system and adept at quickly obtaining the information they wanted—also played a more general role in shaping the extensive refinement of the 511 system that occurred over the first couple of months of post-enhancement operation. In addition to the concern with the touch tone menu option these users were very concerned in general that the new system retain the features that allowed them to move quickly through the menu system. For example, with the old system they could quickly enter a memorized sequence of digits to access the appropriate menu and obtain information on a specific roadway, and they were inpatient with the new system which they felt did not provide the same level of short cutting. Such inputs from power users are part of the reason that ADOT has endeavored to keep the opening greeting—when no user inputs are accepted—as short as possible and to keep the number of similarly unavoidable "floodgate" messages (such as Amber Alerts) to an absolute minimum.
32. Anticipating users' preferences for roadway segmentation schemes and selection of universally meaningful roadway reference points has proven challenging – One of the major activities associated with conversion to a roadway-segment based reporting scheme was the identification of meaningful roadway segments. In an attempt to ensure that any caller looking for information on the portions of interstates within major urban areas could easily obtain such information, ADOT included them in both the preceding and following roadway segment. For example, the Phoenix area portion of I-10 was included in both the "California border to Phoenix" and the "Phoenix to Tucson" segments. This seemed to confuse some users and the segmentation has since been revised to include the Phoenix area information in a single segment. Similarly, it has been challenging for ADOT to identify meaningful roadway reference points in rural areas. No single approach—mileposts, nearby towns, or other place references (washes, canyons, etc.)—seems to resonate with a majority of users.

7.2.4 Overall Lessons Learned

Management and Deployment Issues

33. As is expected with any complex technology project, the reach of the 511 Model Deployment exceeded its grasp, and that's not necessarily a bad thing – Few of the project participants were surprised that all of the planned enhancements were not deployed on time or were being utilized by agencies entirely as planned. They generally feel that delays and even some failures are fully expected in an ambitious technology project, especially one that was explicitly intended to "push the envelope" and that relied upon many different agencies for various data. To some extent, failure to implement all enhancements is evidence that the Model Deployment truly did aim high.
34. Initial investments in a structured planning and design process will pay off – The ADOT 511 project manager noted that the time and effort spent early on to develop the Program Management Plan and System Requirements Definition document were instrumental in the project successes. He also indicated that outside consultant program management support was critical (the program management consultant led the development of the two documents, among many other support activities).
35. Partner agency relationships and participation commitments must be nurtured through the process – Having agencies sign a general memorandum of understanding as part of the Model Deployment funding proposal did not prove to be sufficient to ensure active participation by all agencies. In a few cases agencies participated in early meetings but later fell out of the process, and as a result did not actively participate in data entry. In the opinion of the evaluation team, ADOT was conscientious in laying out an inclusive plan for the Model Deployment and getting meeting invitations to all participants, in essence, erecting "the tent." However, they were not particularly proactive in reaching out to new 511 participants throughout the deployment and leading them into the process—bringing them under the tent. This is not to suggest that ADOT was in any way negligent, but merely to point out that for at least some partner agencies, simply providing the forum and game plan for coordination did not necessarily prove sufficient to stimulate meaningful participation.
36. User input during design is important – Prior to the Model Deployment ADOT had no formal user input regarding the 511 system. During the design of the enhanced system they conducted six focus groups with travelers, including both 511 users and non-users. ADOT found these sessions to be extremely useful, both in directly driving the menu design and other aspects of the Model Deployment but also in providing insights into general traveler attitudes and perspectives. These focus groups resulted in the implementation of "Regional Quick Reports," regional summaries of traffic conditions, which was an enhancement that was not part of the original Model Deployment plan. These focus groups also revealed that there was a perception that the 511 system contained little congestion and incident information, which may have helped stimulate the increase in this type of data that has been observed.

Technical Issues

37. An in-house 511 system provides certain advantages – ADOT noted that the fact that they operate the 511 system themselves, as opposed to out-sourcing it to a commercial operator, has provided them a great deal of flexibility and control in modifying the system. They also believe that although the initial costs for an in-house system are higher, the recurring costs are lower.
38. Although a potentially powerful mechanism for adding data to a 511 system, there are significant challenges and some trade-offs associated with relying on other agencies to input that data – ADOT HCRS operators indicated that, although potentially effective in getting new data into the system, reliance on other organizations (outside the ADOT TOC) sacrifices a measure of consistency and control that is possible when data is input from a central location.
39. Allow plenty of time for voice recognition fine-tuning – Implementation and refinement of the voice recognition capability required more time and effort than was anticipated. Several of the participants involved in this enhancement feel that voice recognition is as much an art as it is a science.
40. In order to ensure data quality and consistency, human data entry operators need to play a central role; a fully automated system cannot provide acceptable quality and consistency – The ADOT 511 project manager noted that the Arizona 511 system features considerable human data input. Although plans for the system entail increased reliance on automated data entry—such as inputting of estimated travel times automatically based on roadway vehicle detector data—a continuing significant level of human operator involvement is viewed as necessary in order to maintain data quality.

7.2.5 Next Steps for Arizona 511

Continued Pursuit of Planned Model Deployment Enhancements

As noted in Section 7.2.2, the overall 511 vision established through the Model Deployment is seen as a very useful framework for continued enhancement of the 511 system. Although not all of the enhancements have yet been implemented, and a few are not yet operating fully as intended (e.g., agencies are not inputting significant data) they are still viewed as good ideas, and worth further pursuit. Therefore, most of the plans for the 511 system consist of implementing the remaining Model Deployment enhancements:

• Ongoing marketing (including the 50 static road signs)
• Next bus arrival times
• Sharing data with other states
• Phoenix arterial street travel times.

In addition to these enhancements, ADOT intends to continue to refine HCRS and to continue with quality assurance activities, including system performance monitoring. Also, the Maricopa County Department of Transportation intends to increase their role in entering local street data, thus helping to address some local agencies' concerns in that area.

Other Planned Enhancements

In addition to continued pursuit of the unimplemented Model Deployment enhancements, ADOT has near-term plans to improve roadway incident data collection through the addition of a Department of Public Safety (highway patrol) computer-aided dispatch (CAD) workstation at the ADOT Traffic Operations Center. This will provide an important new source of accurate incident information. ADOT, along with the Maricopa County Department of Transportation, are also in the midst of implementing travel times for selected Phoenix area freeways. These travel times will use the Phoenix airport as a standard reference point and will be derived directly from freeway vehicle detectors (inductive loops).

7.3 Conclusions

The overall conclusions of the enhancement process analysis are as follows:

• ADOT and most of the Model Deployment participants view the project as essentially successful. Foremost among the project accomplishments are the major redesign of the user interface and the addition of several new types of data and capabilities to the system. The user interface redesign was very substantial and moved the 511 system from an antiquated, touch tone only interface to a voice recognition interface, which is the state-of-the-practice type of user interface for 511 systems. Conversion to segment and region-based roadway reporting (Quick Reports) was another major improvement to the user interface.
• In terms of successes related to new data and capabilities, project participants point to the introduction of airport and transit information and call transfer capabilities, call transfer to the Arizona Office of Tourism, a floodgate message capability that has been used for Amber Alerts, and call transfer capabilities to all of the regional/rural transit providers statewide. These new data sources have done much to move the formerly highway-oriented 511 system to a multi-modal system. Although not an explicit, planned 511 Model Deployment enhancement, the amount of information pertaining to roadway congestion and incidents has increased significantly.
• In terms of shortcomings, most project participants cite the failure to implement the next bus arrival time information as one of the more significant disappointments of the Model Deployment. The relative lack of Phoenix area arterial street data entered by local agencies is also identified as a concern by many participants. Overall, the project participants view the unimplemented enhancements as regrettable, but not entirely unexpected, given the ambitious plans for the Model Deployment and the inevitability of delays and some failures for large, multi-agency technology projects like this one.
• Several technical challenges were encountered, primarily related to availability of new data. Schedule and resource (primarily staff) constraints played a role in postponement of enhancements. However, difficulties in obtaining new data from "outside" (non-ADOT) agencies were a more significant factor. Such difficulties largely explain the delays in the segment weather, next bus arrival times, and other states' data enhancements. Other technical challenges that were encountered and which played a role in delayed enhancements include: development of a secure means to allow non-ADOT agencies to enter data into HCRS via the Internet (that is, providing them secure access to the ADOT computer network where HCRS resides); major telecommunications issues associated with getting the arterial street travel time data from the field to the ADOT Traffic Operations Center; and the magnitude of HCRS base map revisions needed to accommodate Tucson and Phoenix arterial street data entry.
• Within the formal Model Deployment period, participation by partner agencies was uneven and slow building. There are a few enhancements that the project partners generally consider "implemented," or accomplished—because the menu changes have been made and the data entry mechanisms have been established—but which the evaluation team has identified as not yet operating fully as intended. These enhancements are the downtown Phoenix parking and special event information, Phoenix arterial street information, and Grand Canyon National Park information. The Tucson airport and transit information enhancements could, arguably, be included in this group. In all of these cases, some information is being input to 511, but it is mostly by ADOT and/or it is very limited information.
• The project partners associated with these enhancements did not remain fully engaged throughout the Model Deployment and did not, at least during the evaluation period, embrace their data entry role in the 511 system. In some cases, the level of involvement of the partner agencies increased significantly as the Model Deployment progressed, and after the basic enhanced system became operational. This is the case with Phoenix transit, who became very involved during the one-year operational period in greatly expanding their portion of the 511 menu. In other cases, there are promising indications that the level of participation is improving. For example, during the post-enhancement interview with the Tucson airport conducted just after the conclusion of the one-year operational period, the airport representatives were unaware that the Model Deployment was completed and unaware that they were able to input information. They expressed great interest in doing so and will presumably now be more involved.
• Early reactions from veteran 511 users were strong, and were very influential in early, post-enhancement refinements of the user interface. Agencies that are significantly revamping long-established 511 or other telephone traveler information systems should expect that, although potentially small in number, some veteran users of the system will, at least initially, react strongly and fairly negatively. In the case of the Arizona 511 Model Deployment much of the adverse reaction can probably be attributed to the fact that there were many menu system "glitches" in the initial enhanced user interface and the voice recognition system was not performing well. However, it can be hypothesized that even when enhancements perform as intended, if they significantly alter the "feel" of the system and impact veteran users' ability to use shortcuts, they may initially be viewed critically. In the case of the Model Deployment, comments from such users, including some fairly scathing ones, were very useful in the significant refinement of the menu and voice recognition systems that occurred over the first couple of months following deployment.
• Many significant, lasting improvements were made to the 511 program that go far beyond the user interface and data enhancements. Clearly, the major redesign of the user interface and the introduction of several new types of multi-modal data are the successes that are most visible and, in concept, most immediately useful to users. However, there are many accomplishments of the Model Deployment, some explicitly planned and some not, that are as important or more important to the long-term success of the program and ongoing value to users. Foremost among these accomplishments is the institution of a continuous quality improvement philosophy for the 511 program. Under the leadership of the 511 project manager and the Traffic Operations Center Information Technologies Manager (who led much of the technical work), a pervasive, new emphasis on ongoing system performance monitoring has been instilled. This includes implementation of a number of important new data archival, analysis and reporting capabilities, and a demonstrated interest and commitment to using those capabilities to monitor and refine the system. It also includes a new focus on customer service, including the caller comment capability (and the seriousness and thoroughness with which ADOT follows up on the comments) and the many enhanced 511 activities of the Traffic Operations Center Public Information Officer (including 511 marketing and caller comment resolution).
• Finally, there are a number of "halo effects" of the Model Deployment—benefits not directly associated with the planned 511 enhancements or necessarily even restricted to the 511 program. These include the improvements in overall agency partnerships and working relationships (such as has been noted by Phoenix transit). These benefits also include, through the 511 Model Deployment and the demonstration of a long-term commitment to 511, creation of a new focal point for enhanced regional, multi-jurisdictional ITS coordination in the Phoenix region. In recent years, efforts in this area have been constrained to some extent by the absence of a robust, long-term, truly multi-jurisdictional ITS operations project or program. The 511 system is now providing a focus and a forum for addressing more fundamental regional traveler information coordination and data sharing issues.

---

ITS Home | US DOT Home | Privacy | Feedback

United States Department of Transportation - ITS