Introducing Tcl 8.7 Part 2: list enhancements

Published 2020-01-23

This is the second in a series of posts about the new features in the recently released alpha version of Tcl 8.7. This post deals with the enhancements to list processing.

To take Tcl 8.7 for a spin, you can download a pre-alpha binary for your platform. Alternatively, you can build it yourself from the core-8-branch branch in the Tcl fossil repository.

None of the enhancements described here add fundamental new capabilities to Tcl that could not be accomplished at the script level in earlier versions. However, script level implementations are trickier than might seem at first glance, having to deal with non-numeric relative indexing (e.g. end-2), nested lists and unordered indices in operations that modify the list. In addition, there is the obvious benefits of "native" implementations in terms of speed.

The `lpop` command

The new lpop command extracts an element from a list stored in a variable and returns it while storing the modified list (with the element removed) back in the variable. It has the syntactic form

lpop LISTVARNAME ?INDEX ...?

As always, INDEX may be an integer, the keyword end signifying the last element or a simple expression as illustrated in the following.

% set L {a b c d e f g}
a b c d e f g
% puts [lpop L],$L
g,a b c d e f
% puts [lpop L end-2],$L
d,a b c e f
% puts [lpop L [lsearch $L b]+1],$L
c,a b e f
% puts [lpop L 0],$L
a,b e f

Notice INDEX defaults to end if unspecified.

If multiple indices are specified, they reach into nested lists.

% set L {a b {c d {e f}} g}
a b {c d {e f}} g
% puts [lpop L end-1 end 0],$L
e,a b {c d f} g

Unlike commands such as lindex, lpop will raise an error if any index is out of range.

The `lremove` command

Another new command for operating on lists is lremove which, like lpop, removes elements from a list. It differs in that, unlike lpop, it operates on list values, not variables, can remove multiple elements and returns the modified list, not the elements removed.

lremove LIST ?INDEX ...?

Unlike lpop, if multiple indices are specified, they indicate multiple elements to be removed, not a path to an element in a nested list structure. Indices may be specified in arbitrary order and may be duplicated.

% lremove {a b c d e f g} end-1 2 5
a b d e g

Note the duplicate and out of order indices.

The -stride option to `lsearch`

Lists of records are commonly represented in Tcl in one of two ways:

a nested list where the inner list is a record
a flat list where fields of the records are implicitly grouped together.

For example, a record consisting of cities and population in millions may be represented either as either of the following forms:

set nested [list {Tokyo 37} {Delhi 29} {Shanghai 26}]
set flat   [list Tokyo 37 Delhi 29 Shanghai 26]

For the most part, Tcl 8.6 provided commands for either of these styles. For example, the -index and -stride option to sort nested and flat lists respectively.

% lsort -index 0 $nested
{Delhi 29} {Shanghai 26} {Tokyo 37}
% lsort -stride 2 $flat
Delhi 29 Shanghai 26 Tokyo 37

The -stride option specifies the number of elements in the list that are considered to be one record or group.

However, lsearch in Tcl 8.6 supported the -index option to search through nested lists,

% lsearch -index 0 $nested Delhi
1
% lsearch -index 0 -inline $nested Delhi
Delhi 29

it did not provide an equivalent -stride option for searching flat lists. Tcl 8.7 fixes this deficiency.

% lsearch -stride 2 $flat Delhi
2
% lsearch -stride 2 -inline $flat Delhi
Delhi 29

By default, lsearch will compare the pattern against the first element in each stride group. The -index option may be used to change this. For example, suppose we wanted to match against the last name in a flat list of first and last name pairs.

% set names {John Lennon Elton John John Legend}
John Lennon Elton John John Legend
% lsearch -stride 2 -index 1 $names John
2
% lsearch -stride 2 -index 1 -inline $names John
Elton John

Note above that the index returned is that of the first element in the matched group, not the matched element.

The -stride option may be used with other combinations of options as well. See the manpage for more examples.