NTL > TLR Archive > Index Page

Transportation Librarians Roundtable
TRIS User Satisfaction Survey: Early Results and Next Steps

Slide 1
TRIS User Satisfaction Survey:
Early Results and Next Steps

  • Presented by Ken Winter
  • VDOT Research Library
  • An NTL Transportation Librarians Roundtable Presentation
  • Online at: http://ntl.bts.gov/networking/roundtable.html
  • April 10, 2008

Slide 2
TRIS Background…

  • July 2007 marked the 40th Anniversary of the Transportation Research Information Services (TRIS) Database…
  • The idea dates to 1923. In 1928 $8,000 was appropriated by the HRB “to collect and distribute information of completed and current research.”
  • TRIS was developed over 3 years with the sponsorship of the State Highway Departments and BPR to include citations and abstracts for published highway literature.

Slide 3
More TRIS Background…

  • In 1967, the Highway Research Information Service (HRIS) was implemented. It used computer and automated technology to expand the “clearinghouse” concept.
  • TRIS continued to improve, expanding its coverage and becoming multi-modal.
  • Partnerships developed to receive records from ITRD, Northwestern University’s transportation library and the University of California Berkeley transportation library.

Slide 4
TRIS Today…

  • A version of TRIS (TRIS Online) is available on the Web with close to 700,000 records.
  • Nearly 38,000 records in TRIS Online have links to (mostly free) full-text publications.
  • TRIS is the largest and most comprehensive database for published transportation research.
  • TRIS is available as a subscription resource from vendors Ovid/Silverplatter (TRANSPORT) and Dialog (File 63).

Slide 5
TRIS Today…Enhancements Continue

  • 19,000 EIS from Northwestern University were added to TRIS in 2007.
  • TRIS Records for articles from “Transportation Research Record” now have links to full text.
  • TRB is adding links to records of Elsevier and Taylor & Francis journal articles.
  • Coming next…theses.

Slide 6
Why a Survey? It Started with a Photo

image: group photo

Slide 7
Which Led Us to Articles in TR News

image: covers of several Transportation Research News

Slide 8
The Idea for an Updated User Survey

  • Came from discussions with Barbara Post.
  • Which led to sponsorship by TRB’s Library and Information Science for Transportation (LIST) committee.
  • Which led to conference calls and a meeting at TRB Midyear (July 2007, Chicago) and one at TRB Annual (January 2008, Washington).
  • This work has been a team effort by a committee of librarians, researchers & other transportation professionals.

Slide 9
Special Thanks to Committee Members

  1. For Helping Develop and Test the Survey
  2. For Helping Analyze Data/Presenting Findings
    Barbara Post -Transportation Research Board
    Sandy Tucker -Texas A&M University, LIST
    Roberto Sarmiento -Northwestern University
    Amanda Wilson -National Transportation Library
    Rita Evans – University of California, Berkeley
    Brie Schwartz – Transportation Research Board
    Bonnie Osif -Penn State
    Sue Sillick – Montana DOT
    Barbara Harder -BT Harder, Inc.
    Karen White – FHWA
    Mary Kay Christopher – MKC Associates
    Laura Whayne – Kentucky Transportation Center

Slide 10
Objectives of the Survey

  1. To understand who uses the TRIS database and what they use it for.
  2. To determine user satisfaction levels.
  3. To let users make suggestions for short-and long-term enhancements to TRIS.

Slide 11
Methodology

  • In November 2007 a 20-point survey was conducted using Zoomerang survey software.
  • To target core users, TRB sent out a direct e-mail to members of Division A Committees and student attendees 2006­2007. The survey was also posted to the TRANLIB and AASHTO RAC Listservs.
    • Committees & Students: 7,353 total
    • Listservs: 364 total
    • Population size: 7,717 total

Slide 12
Methodology

The actual survey questions, their wording, order and “type” were formulated by the committee from May-October 2007.

  • Significant pre testing was done.
  • Some changes were made.
  • Conducted November 7-30, 2007.

Slide 13
Methodology: Limitations

  • We let respondents self select.
  • We did not randomly select from antecedent population.
  • We let respondents self-identify “role” and did not give them role definitions.
  • There was no mechanism to prevent people from taking the survey more than once.

Slide 14
What the Survey Looked Like

images: survey screens

Slide 15
Response Rate

There were 362 responses to the survey, but 35 included only demographic data. They were disqualified, for a total of…

327 valid responses…a rate of 4%

NOTE: 16 respondents indicated they had never used TRIS. The Committee discussed this fact and decided to include those responses though they could skew results.

Slide 16
The Five “Sections” of the Survey

  • Demographics
    • Basic information about respondents.
  • Usage
    • How do you access TRIS?
  • Relevancy and Impact
    • Does content coverage (mode, function) fulfill your needs?
  • User Satisfaction
    • Overall how well does TRIS satisfy your needs?
  • The Future of TRIS
    • Suggestions for improvement.

Slide 17
Demographics

1. Select the category that best describes your place of employment.

image: pie chart of place of employment

Place of Employment (number and percentage of respondents)
Place Number of Respondents Percentage of Respondents
College or University 119 36%
State DOT 98 30%
Corp. or Private Co. 57 17%
Federal Agency 16 5%
Not-for-Profit Entity 15 5%
Other 13 4%

Slide 18
Demographics

2. Please indicate the country where you are employed:

images: pie chart of country of employment

Country of Employment (number and percentage of respondents)
Country Number of Respondents Percentage of Respondents
United States 289 88%
Canada 10 3%
Australia 4 1%
Israel 3 1%
United Kingdom 3 1%
Sweden, China, Greece, Philippines 2 Responses each  
Albania, Denmark, France, Germany, Ireland, Italy, Mexico,
Puerto Rico, Switzerland, United Arab Emirates
1 response each  

Slide 19
Demographics

3. Select the role or position that best describes your position…

images: pie chart of role or position

Position(number and percentage of respondents)
Position Number of Respondents Percentage of Respondents
Researchers 76 23%
Engineer 71 22%
Student 43 13%
Information Professional 41 13%
Faculty Member 33 10%
Administrator 27 8%
Other 30 9%
Other Support Personnel 6 2%

Slide 20
Demographics

4. Indicate your experience level:

images: pie chart of

Experience Level(number and percentage of respondents)
Experience Level Number of Respondents Percentage of Respondents
Student (Grad. or Undergrad.) 40 12%
1-5 years in transportation 51 16%
5-15 years in transportation 68 21%
15+ years in transportation 168 51%

Slide 21
Usage

5. For the version(s) of TRIS that you use, how would you rate the ease of use?

Ease-of-Use for different versions of TRIS (percentage of respondents)
Ease of Use Tris Online Transport File 63
Extremely easy 16% 5% 2%
Easy 67% 19% 8%
Difficult 7% 4% 2%
Extremely difficult 1% 0% 1%
Don't use this version 9% 72% 87%

Slide 22
Usage

6. For the version(s) of TRIS that you use, how would you rate the ease of use?

Ease-of-Use for different versions of TRIS (number of respondents)
Ease of Use Tris Online Transport File 63
Extremely easy 53 16 6
Easy 219 61 27
Difficult 24 13 7
Extremely difficult 2 1 3
Don't use this version 29 236 284

Slide 23
Respondents Preferred TRIS Online

  • 91% of respondents reported using TRIS Online.
  • About 83% of them said TRIS Online was either “Extremely Easy” or “Easy” to use.
  • Only 28% of respondents reported using TRANSPORT and 13% reported using File 63.
  • Only 24% of TRANSPORT users called it “Extremely Easy” or “Easy” to use.
  • Only 10% of File 63 users reported that way.

Slide 24
Usage

6. On average how often have you searched the TRIS database in the last 12 mo.?

Frequency of TRIS usage (number and percentage of respondents)
Frequency Number of Respondents Percentage of Respondents
Weekly 70 21%
Monthly 97 30%
A few times a year 122 37%
Never 38 12%

Slide 25
Usage

7. What types of information are you looking for when you search TRIS?

TRIS usage and type of information (number of respondents)
Type of information Number of Respondents
All research on a topic 245
Basic facts on a topic or issue 155
Current state of the practice 208
Historical materials 129
Names of persons/orgs. doing research 90
Overview of a specific topic 159
Verification of a citation 130
Other 21

Slide 26
Usage: Getting Cited Works

8. What are the MOST TYPICAL method(s) you employ to get the documents cited?

299 Respondents Answered…

  • Though not in every case, they tended to list multiple strategies, ranking their strategies in order of operation.
    • First, I do this.
    • Second, I do this.
    • Last, I do this.

Slide 27
Usage: Getting Cited Works

8. When you get results from TRIS that do not include a link to an online source, what is/are the MOST TYPICAL method(s) you employ to get the documents cited?

image: graph of result

Methods for acquiring documents cited (number and percentage of respondents)
Method Number of Respondents Percentage of Respondents
Use a Library 140 46.8%
Use a Search Engine 133 44.4%
Non-response 11 3.6%
My Collection/Local Copy/Ask a Colleague 4 1.3%
Use a Subscription Database 3 1%
Give Up 2 .06%
Buy from Bookstore/Vendor 2 .06%
Other Unspecified Method 2 .06%
Contact Publisher/Author 1 .03%
Look for Other Sources 1 .03%

Slide 28
Usage

9. What other resources do you use when searching for transportation info.?

Other Resources Use (number and percentage of respondents)
Other Resources Number of Respondents Percentage of Respondents
Free indexing/abstracting services 139 42%
General internet search engines 265 81%
My library's online catalog 180 55%
NTL catalog 102 31%
Other com. indexing/abstracting tools 89 27%
Research In Progress (RIP) Database 92 29%
Other transportation library catalogs 45 14%
TLCat 40 12%
TranStats 25 8%
Worldcat.Org 54 17%

Note: Respondents could pick as many or as few as applied, so total responses do not add up to 100%.

Slide 29
Relevancy/Impact

10. How relevant is information you find in TRIS to your discipline?

Relevance of Information in Discipline (percentage of respondents)
Relevancy Percentage of Respondents
Extremely Relevant 41%
Usually Relevant 46%
Sometimes Relevant 11%
Not Relevant At All 2%

Slide 30
Relevancy/Impact

11. Rate how well the information you find in TRIS helps you to perform your job?

Relevance of Information in Job Performance (percentage of respondents)
Relevancy Percentage of Respondents
Extremely Well 25%
Usually Well 51%
Sometimes Well 18%
Not Well At All 6%

Slide 31
Relevancy/Impact

14. Do you believe you make better decisions or are more effective because you use TRIS?

  • Yes
    • 238 Respondents
    • 84 Percent
  • No
    • 44 Respondents
    • 16 Percent

Slide 32
Relevancy/Impact

16. Would you recommend TRIS to other transportation researchers?

  • Yes
    • 263 Respondents
    • 96 Percent
  • No
    • 11 Respondents
    • 4 Percent

Slide 33
Conclusions: Frequent Requests

  • Users Want More:
    –Citations in general
    –Speed getting new citations into TRIS
    –International coverage
    –Links to full text documents
    –Topical coverage: planning, environmental
  • Users Want Less:
    –Duplicate records

Slide 34
Conclusions: Frequent Comments

  • Respondents consistently requested more online links to full text documents OR information on how to acquire documents.
  • They frequently cited libraries, librarians and library services (such as ILL) as an important part of the research process.
  • They did NOT request snazzy technological changes to TRIS…they were far more likely to request something like expanded coverage.

Slide 35
Conclusions: User Suggestions

  • Some specific suggestions for improving the technology behind TRIS included:
    “A Better search interface” that “is more user friendly” or “more intuitive.”
    • EndNote, and the ability to do more with citations
    • “User accounts” saved searches or alerts
    • Link to Worldcat or other commercial sources
    • The ability to search within results
    • The ability to “Follow references” (Google scholar)
    • Folksonomies or the ability to adding keywords

Slide 36
Conclusions: Respondents Value TRIS

  • Respondents use the “TRIS Online” interface most of all, and often stated in comments they liked the fact that was freely accessible.
  • Respondents reported using other online research tools in addition to TRIS…most notably search engines.
  • When they mentioned a free Web search engine by name...that name was Google.
  • Libraries were named more than Google.

Slide 37
Next Steps

Slide 38
And Now For Your Questions?