PROGRAMS, PROCESS, AND PRACTICES: INTELLIGENCE COLLECTION
Many MSLs rely on pharmaceutical and medical Web sites and portals to provide valuable news and information. Searching the vast Web for targeted information, however, can often remain elusive.
Cynthia C. Correia, MLS, Fuld and Company.
With the ever-increasing growth of content, technology, and other tools, business and research professionals have enjoyed improved access and availability to information. This benefit, however, signifies a greater expectation for information delivered quickly and economically (preferably free), as well as a greater reliance on information for professional efficacy, productivity, and performance. The Web has played a prominent role in this development and, holding a wealth of content and tools, it widely remains a convenient and promising source for business and competitive research and analysis.
For medical science liaisons and medical affairs professionals, the Web can offer the opinions of thought leaders, provide awareness of current therapies, and fuel insights and analyses for their company’s competitive intelligence practices. Many MSLs rely on pharmaceutical and medical Web sites and portals to provide valuable news and information. Searching the vast Web for targeted information, however, can often remain elusive.
When researching the World Wide Web, most searchers turn to one of the major search engines with the hope that the right site will appear among the top results. Unfortunately, as statistics and surveys reveal, most searchers are disappointed by this approach. Moreover, many search tools provide inadequate search features, instructions, and tips intended to enable users to uncover appropriate material on the Internet. While expert searching as practiced by information professionals involves advanced techniques and tools, some of these can enable savvy non-info pros to uncover vital resources.
When discussing advanced search techniques, one of the most essential concepts is the Invisible Web, aka Deep Web or Hidden Web. The Invisible Web is thoroughly treated by Gary Price and Chris Sherman in their definitive book, The Invisible Web-Uncovering Information Sources Search Engines Can’t See. Briefly defined, the Invisible Web describes Web-accessible resources that are not captured by search engines. This type of material includes databases, dynamic pages, non-hypertext markup language (non-html) document formats, sites requiring login, recently updated html page content, and html documents that reside deeper in a Web site – resources which often comprise the most content-rich and relevant resources for business and industry research, for example:
· Industry and Company Directories
· Industry News and Resources
· Interview Transcripts
· Conference Proceedings and Presentations
· Regulatory Filings and Reports
· Domain Name and IP Address Directories
· Patent Databases
· Court Cases
· Investment Research
· Market Research Reports
· Real-time News and Other Information
An example of an Invisible Web resource in the Pharmaceutical industry is Phrma’s New Medicines in Development Database (www.phrma.org/newmedicines). Search engines index the database Web page, but do not capture the database contents.
The increasing sophistication of search tools, however, has brought forward material previously considered belonging to Invisible Web. Google, for example, now captures non-html file types like Microsoft Word, Power Point, Excel, Adobe Portable Document File, and Rich Text Format, as well as images, telephone directories, and print catalogues. Other Web sites, online search tools, and desktop software are increasingly targeting the Invisible Web. Nevertheless, gaps exist and, as the Web universe expands exponentially, searchers need to utilize advanced tools and techniques to search for quality resources efficiently and effectively. Some tips to consider:
Choose a couple of top engines and understand their strengths, weaknesses, and search features well.
Search engines vary in the number and type of material they index, and even the best engines capture only a portion of the Web. Understanding how to use a couple of engines well broadens the number of searchable sites. Use an engine’s advanced search features to find documents more effectively. For example, find Power Point presentations that discuss clinical trials in oncology by using Google’s filetype: feature:
“clinical trials” (oncology OR cancer) recruitment filetype:ppt
Or identify Word documents containing the biographies or curriculum vitae of clinical researchers in immunology:
immunology “clinical research” (biography OR bio OR vita OR vitae) filetype:doc
The above search strings illustrate how using Google’s advance search features can target results. The phrase search “clinical trials” ensures that the two words appear together, eliminating vast number of files that contain instances of the words appearing apart. The OR operator combines synonyms, word variations, or other related concepts that documents may contain. The filetype operator specifies the type of document to retrieve. Other search features include filtering by date, searching within a domain name, and geographic searching. These features can often be found in the “Advanced Search,” “Search Tips,” or “Help” pages.
Supplement favorite engines with Invisible Web and other specialized directories and tools.
These resources vary by their scope and type of coverage, but they are aimed at collecting resources that search engines miss. Choose a couple of favorites here, too.
Some Specialized Invisible Web Resources:
· Complete Planet www.completeplanet.com
· Direct Search www.freepint.com/gary/direct.htm
· Invisible Web www.invisibleweb.com
· The Invisible Web Directory www.invisible-web.net
· Profusion www.profusion.com
· SurfWax www.surfwax.com
Medical and Pharma Sites Containing Invisible Web Resources:
· MedWeb: Pharmacy and Pharmacology www.medweb.emory.edu/MedWeb
· Link Pharm of the MSL Quarterly www.mslquarterly.com/linkpharm.php
Use search engines to identify Web pages that host the resource you want or pages linking to those resources.
While actual Invisible Web resources are not searchable via engines, their Web pages or pages linking to them are. We can take advantage of this to locate databases of Canadian physicians via the Altavista engine, for example:
database* NEAR (physician* OR doctor*) NEAR canad*
By including the term “database” in the search string, we can find sites and portals that contain the resource we want. To search for other types of resources, simply apply the appropriate terms that relate to the resource (directory, filings, list, presentations, etc.) or the search function (search, searchable, find). Also, please note the advanced search techniques used above: The asterisks expand the search terms to include plural and variants forms of the word. The NEAR operator specifies that the terms appear in proximity to one another.
Include specialized subject-specific search engines to target resources.
There are an increasing number of engines that specializes in science and medical content. While features vary, many search Invisible Web resources, peer-reviewed articles, and various file formats. For lists of specialized engines, visit Search Engine Watch, including:
Medical Search Engines www.searchenginewatch.com/links/medical.html
Science Search Engines www.searchenginewatch.com/links/science.html
Identify the likeliest sources or hosts for resources.
Industry associations, professional organizations, and government agencies, for example, often include searchable databases, conference material, and other Invisible Web resources on their Web sites. By identifying these organizations through search engines or directories, we can in turn search their Web sites for valuable material. Search engines can be helpful in identifying material within a particular site. To find abstracts or presentations only within the Web site of the Infectious Diseases Society of America, we can use the site: feature in Google, for instance:
site:www.idsociety.org “Infectious Diseases Society of America” (abstracts OR presentations)
While the Invisible Web is only one aspect of the multifaceted universe of business and competitive research, and a small component in the broader practice of competitive intelligence, it is critical to uncovering resources that are often unavailable or inconvenient through other means. It can also present an efficient and sometimes low-cost method of uncovering material that can enhance awareness, support insights, and provide a competitive edge. Access to the Invisible Web, however, requires a deeper understanding of searching and a conscious application of more advanced techniques. As we can see from the guidelines above, some of these techniques may be easily applied, and a growing collection of resources are aimed at supporting more advanced research to satisfy increasingly complex business needs.
Cynthia Cheng Correia is Director of Information Services at Fuld & Company, Inc., a research and consulting firm that has pioneered competitive and strategic intelligence services for a quarter century. She heads Fuld & Company’s literature research services, and provides consulting and training in information management and research to support competitive intelligence (CI). She appears as a speaker and has written on research and CI issues. Ms. Correia is also featured in Super Searchers on Competitive Intelligence (Information Today).