Txr

Introducing Txr, the interpreter for the txr query language, perfect for text extraction tasks. With Txr, you can extract pieces of text that are bound to variables in the query and output them as raw bindings or substitute them into a template-driven report.

While tools like sed, awk, and perl support pattern matching with regular expressions, they don't implement a whole-input pattern matching paradigm like Txr. Sed is limited to basic regexp filtering and struggles with text extraction tasks that span multiple lines. Awk and Perl are programming languages used for complex text extraction, but often expressed as algorithms. A clearer, more concise Txr query can typically achieve the same result.

To develop a Txr query, simply start with sample data. The raw data is likely already a Txr query that matches itself after some character escaping. Identify the relevant variable parts and generalize the query to work for all instances of the data.

To demonstrate, let's look at a practical example. Have you ever struggled with the ps utility for listing processes? With Txr, we can easily create a ps utility relying on the /proc filesystem on Linux.

Check out the example query below for listing processes:

@(next)$/proc
@(collect) @{process /[0-9]+/}
@(next)/proc/@process/status Name:@ @name State:@ @state (@state_desc)
@(skip) Tgid:@ @tgid Pid:@ @proc_id PPid:@ @parent_id
@(bind pid proc_id) @(bind ppid parent_id)
@(skip) Uid:@ @uid@ @/.*/ Gid:@ @gid@ @/.*/
@(next)$/proc/@process/task
@(collect) @thr
@(end)
@(bind thread thr)
@(some)
@(next)/etc/passwd
@(skip) @user:@pw:@uid:@/.*/
@(or)
@(bind user uid)
@(end)
@(end)
@(output) USER PID PPID S NAME THREADS
@(repeat) @{user 8} @{proc_id -5} @{parent_id -5} @state @{name 16}
@(rep)@thr, @(first)@(last)@thr@(single)~@(end)
@(end)

The Txr query processes numeric entries under the /proc directory, reads the /proc/< pid >/status file of each process, and the list of threads under /proc/< pid >/tasks. The user IDs are resolved by matching through the /etc/passwd file.

With Txr, you can finally achieve advanced text extraction tasks and produce powerful reports with ease. Give it a try today!