Notes on text tables and text lists
-----------------------------------------

Text tables:
	The freshmen file is a table.  Each line is a record.
	Each record consists of several fields separated by a specific
	delimiter.

	LN=smith:FN=jane:CL=92:AD=thayer

Text List:
	The fl program uses text lists for its data files.
	A record is a sequence of lines.  Each record consists of
	a fieldname and a field value.  The records are
	separated by a specific delimiter.

		LN=smith
		FN=jane
		CL=92
		AD=thayer
		==

Comparison:

	The fl program uses the list format.  It is easier
	to edit.  This format allows the data to contain
	almost any character.

	The table format is much better for Unix text-processing
	tools.  One can use grep, sort, cut, etc.  The table
	format is harder to edit.  The table format prevents you
	from using the delimiter in the data.

A solution:

	One solution is to tranform the data from list format
	to table format and back as appropriate.

	For example, if you keep the data in table format, you
	can grep the file for specific records then convert the
	table format to list format for processing by fl.

	On the other hand, if you store the data in list format
	you can convert it to table format for sorting then 
	convert it back to list format.

Tools for format conversion:

	Ok. that sounds swell.  How can I convert a table into
	a list and back?

	There are lots of ways.  Here are some;

	table to list
	-------------
	#!/bin/sh
	# t2l
	#   read a table with : delimiter , print out list with == delim
	#   idea: add a colon and == to end of line, then convert each
	#	  colon to a newline.
	#     usage: t2l data.table > data.list
	#   example: grep LN=smith freshmen | t2l | fl letter.fmt > result
	#
		sed -e 's/$/:==/' $* | tr ':' '\n'
	#
	#   note: you can do this with sed with 
	#
	#      sed -e 's/$/:==/' -e 's/:/\
	#      /g'
	#
	#   but that is sort of ugly.

	list to table
	-------------
	this requires accumulating lines into one string until
	the '==' line appears.  The versions here create tab-separated
	records.  You can change it to have colon-separated records.

	This one is a shell script:
	
		#!/bin/sh
		# l2t
		# convert list to table
		# example:  l2t data.list > data.table
		STR=""

		cat $* | while read x
		    do
			if test "$x" = "=="
			then
			    echo "$STR"
			    STR=""
			else
			    if test "$STR" != ""
			    then
				STR="$STR   $x"
			    else
				STR=$x
			    fi
			fi
		    done  
			

	This one is a sed script.  I call it l2t.sed You can use it as
	sed -f l2t.sed data.list > data.table

		#
		# convert list data to table data
		# stop at a line with ==
		# separate each line with |
		#
		/^==$/{
			s/.*//
			x
			s/^.//
			s/\n/|/g
			s/.$//
			p
			d
			n
		}
		H
		d
