Desktop search

File:AdunaAutoFocus5.png

OSL Desktop Search engines software Aduna AutoFocus 5

Desktop search tools search within a user's own computer files as opposed to searching the Internet. These tools are designed to find information on the user's PC, including web browser history, e-mail archives, text documents, sound files, images, and video.

One of the main advantages of desktop search programs is that search results are displayed quickly due to the use of proper indexes.

A variety of desktop search programs are now available; see this list for examples.

Desktop search emerged as a concern for large firms for two main reasons: untapped productivity and security. On the one hand, users needs to be able to quickly find relevant files, but on the other hand, they shouldn't have access to restricted files. According to analyst firm Gartner, up to 80% of some companies' data is locked up inside unstructured data — the information stored on an end user's PC, the directories (folders) and files they've created on a network, documents stored in repositories such as corporate intranets and a multitude of other locations. Moreover, many companies have structured or unstructured information stored in older file formats to which they don't have ready access.

Companies doing business in the United States are frequently required under regulatory mandates like Sarbanes-Oxley, HIPAA and FERPA to make sure that access to sensitive information is 100% controlled. This creates a challenge for IT organizations, which may not have a desktop search standard, or lack strict central control over end users downloading tools from the Internet. Some consumer-oriented desktop search tools make it possible to generate indexes outside the corporate firewall and share those indexes with unauthorized users. In some cases, end users are able to index — but not preview — items they should not even know exist.

Historically, full desktop search comes from the work of Apple Computer's Advanced Technology Group, resulting in the underlying AppleSearch technology in the early 1990s. It was used to build the Sherlock search engine and then developed into Spotlight, which brought automated, non-timer-based full indexing into the operating system.

Technologies

Most desktop search engines build and maintain an index database to achieve reasonable performance when searching several gigabytes of data. Indexing usually takes place when the computer is idle and most search applications can be set to suspend indexing if a portable computer is running on batteries, in order to save power. There are notable exceptions, however: Voidtools' Everything Search Engine, which performs searches over only filenames — not the files' contents — for NTFS volumes only, is able to build its index from scratch in just a few seconds. Another exception is Vegnos Desktop Search Engine, which performs searches over filenames and files' contents without building any indices. The benefits to not having indices is that, in addition to not requiring persistent storage, more powerful queries (e.g., regular expressions) can be issued, whereas indexed search engines are limited to keyword-based queries. An index may also not be up-to-date, when a query is performed. In this case, results returned will not be accurate (that is, a hit may be shown when it is no longer there, and a file may not be shown, when in fact it is a hit). Some products have sought to remedy this disadvantage by building a real-time indexing function into the software. There are disadvantages to not indexing. Namely, the time to complete a query can be significant, and the issued query can also be resource-intensive.

Desktop search tools typically collect three types of information about files:

• file and folder names
• metadata, such as titles, authors, comments in file types such as MP3, PDF and JPEG
• file content (for supported types of documents only)

To search effectively within documents, the tools need to be able to parse many different types of documents. This is achieved by using filters that interpret selected file formats. For example, a Microsoft Office Filter might be used to search inside Microsoft Office documents.

Long-term goals for desktop search include the ability to search the contents of image files, sound files and video by context.

The sector attracted considerable attention from the struggle between Microsoft and Google. According to market analysts, both companies were attempting to leverage their monopolies (of web browsers and search engines, respectively) to strengthen their dominance. Due to Google's complaint that users of Windows Vista cannot choose any competitor's desktop search program over the built-in one, an agreement was reached between US Justice Department and Microsoft that Windows Vista Service Pack 1 would enable users to choose between the built-in and other desktop search programs, and select which one is to be the default.

As of September, 2011, Google ended life for Google Desktop, a program designed to make it easy for users to search their own PCs for emails, files, music, photos, Web pages and more.

Desktop search products are software alternatives to Windows Desktop and Outlook Search, helping business professional sift through desktop files, emails, attachments, SharePoint data, and more.,

Platforms & their histories

There are three main platforms that desktop search falls into. Windows, Mac OS & Linux. This article will focus on the history of these search platforms, the features they had, and how those features evolved.

Windows

Today's Windows Search replaced WDS (Windows Desktop Search). WDS, in turn, replaced Indexing Service. A "a base service that extracts content from files and constructs an indexed catalog to facilitate efficient and rapid searching" Indexing service was originally released in August 1996, it was built in order to speed up manually searching for files on Personal Desktops and Corporate Computer Network. Indexing service helped by using Microsoft web servers to index files on the desired hard drives. Indexing was done by file format. By using terms that users provided, a search was conducted that matched terms to the data within the file formats. The largest issue that Indexing service faced was the fact that every time a file was added, it had to be indexed. This coupled with the fact that the indexing cached the entire index in RAM, made the hardware a huge limitation. This made indexing large amounts of files require extremely powerful hardware and very long wait times.

In 2003, Windows Desktop Search (WDS) replaced Microsoft Indexing Service. Instead of only matching terms to the details of the file format and file names, WDS brings in content indexing to all Microsoft files and text-based formats such as e-mail and text files. This means, that WDS looked into the files and indexed the content. Thus, when a user searched a term, WDS no longer matched just information such as file format types and file names, but terms, and values stored within those files. WDS also brought "Instant searching" meaning the user could type a character and the query would instantly start searching and updating the query as the user typed in more characters. Windows Search apparently used up a lot of processing power, as Windows Desktop Search would only run if it was directly queried or while the PC was idle. Even only running while directly queried or while the computer was idled, indexing the entire hard drive still took hours. The index would be around 10% of the size of all the files that it indexed. For example, if the indexed files amounted to around 100GB of space, the index would, itself, be 10GB large.

With the release of Windows Vista came Windows Search 3.1. Unlike its predecessors WDS and Windows Search 3.0, 3.1 could search through both indexed and non indexed locations seamlessly. Also, the RAM and CPU requirements were greatly reduced. Cutting back indexing times immensely. This brings us to the Windows Search 4.0 which is currently running on all PCs with Windows 7 and up.

Mac OS

Mac OS was the first to fully implement Desktop Search, it allowed users to fully search all documents with in their Macintosh computer. This means file format types, meta-data on those file formats and the content within the files. Released in 1994 two years before Windows Search was released, AppleSearch already had content searching. The biggest issue that AppleSearch had large resource requirements "AppleSearch requires at least a 68040 processor and 5MB of RAM." A Macintosh computer that had these specs cost around $1400 in today's dollars that's around $2050. On top of that, the software it self cost around $1400 for a single licenses.

In 1997, Sherlock was released alongside Mac OS 8.5. Sherlock, named after the famous fictional detective Sherlock Holmes, was integrated into Mac OS's file browser: Finder. Sherlock extended the desktop search to the World Wide Web. Allowing users to now search locally and externally. Adding the web to Sherlock was relatively easy as the plugins only needed to be written in a plain text file. Sherlock was included in every single Mac OS 8, 9 and 10 until 10.5.

Spotlight was released in 2005, on Mac OSX 10.4, is a Selection-based search which means the user invokes a query using only the mouse. It allows the user to search the Internet for more information about any keyword or phrase contained within a document or webpage. Spotlight also uses a built-in Oxford American Dictionary and calculator to offer quick access to definitions and small calculations. While Spotlight had an initially long start-up time (for first time set up). The entire hard disk was indexed, and as files are added to the hard disk, the index is constantly being updated in the background. This is done using minimal CPU & RAM resources, making searching relatively easy and quick.

Linux

For Linux, we will primarily cover the Ubuntu distribution as it was and currently is still the most popular version of Linux. Strangely enough, Ubuntu didn't have desktop search until Feisty Fawn 7.04. Using Tracker desktop search, the desktop search feature was very similar to Mac OS's AppleSearch and Sherlock. Considering the fact that both are UNIX based systems. Tracker, was released in late 2007 was built to have a relatively low impact on system resources. But unfortunately occasionally had sporadic control over what resources it was using. It not only featured the basic features of file format sorting, and meta-data matching, but support for searching through emails and messages (instant messages) was added. Years later, in 2014 Recoll was added to Linux distributions, it works with other search programs such as Tracker and Beagle to provide efficient full text search. This greatly increased the types of queries that Linux desktop searches could handle as well as file types. The wonderful thing about Recoll is that it allows for greater customization of what is indexed. For example, Recoll will index the entire hard disk by default, but will and can index just a few select directories instead of wasting time indexing directories you know you will never need to look at. It also allows for more search options, you may actually narrow down what kind of query you want to ask. For example, you could search for just file types or by content.

See also

• List of desktop search engines