harvest

Harvest is a software system that collects and organizes information, making it easily searchable via a web interface. It has the ability to gather data from both inter- and intranet sources, including HTTP, FTP, and NNTP, as well as from local files such as hard drives, CD-ROMs, and file servers. In addition to HTML, Harvest supports a wide range of other file formats, including TeX, DVI, PS, mail, man pages, news, WordPerfect, RTF, Microsoft Word/Excel, SGML, C sources, and more.

Harvest's modular design makes it easy to add support for new formats, and the software includes stubs for PDF support using Xpdf or Acroread. Harvest is also a modular, distributed search system framework, with a set of components that work together to create a complete search system.

One of the key features of Harvest is its ability to distribute the load among different machines, allowing for data gathering on a number of machines simultaneously. The full text indexer, broker, and web server components can all run on separate machines. In addition, because each step of the data collection and search process is implemented as a separate program, it is easy to modify or replace parts of Harvest to customize its behavior.

Harvest also allows for complete control over the content of the search database, with the ability to customize the summarizer and filter mechanism to create summaries that meet specific needs. The search interface is written in Perl, making it easy to customize if desired.

The latest release of Harvest includes updates to the QDBM and YAZ components. Overall, Harvest is a powerful and flexible search system that can be customized to meet specific organizational needs.