|
|
|
nft_util - FTP Server Framework
The NFT_UTIL package provides a flexible framework for implementing the
server side of an FTP client-server session. The session
begins when a client establishes a TCP/IP control connection to the FTP
server. At that point, the server creates an NftSession
object, which essentially layers the FTP protocol on top of the raw
TCP/IP network connection. When the session object is created, the
server can register functions to process the different FTP commands
the client may send. The session ends when the server receives the
"QUIT" command from the client or if the control connection is broken.
You may be thinking, "What's the big deal? My computer already comes with a standard FTP server." Back in 1995 or 1996, I worked at NASA's Goddard Space Flight Center in Code 521, the Microelectronics Systems Branch, and we needed a non-standard FTP server. Our system was responsible for receiving and processing 50-Mbps data downlinks from satellites in real-time. To do so, the hardware/design engineers in Code 521 designed application-specific integrated circuits (ASICs) to receive, decode, and process this high-throughput stream of CCSDS packets. The ASICs were hosted on VME boards and controlled by software running under VxWorks. This, the first step in the processing of satellite data, is known as "Level 0 Processing" (LZP).
On one particular part of the project, the data was stored on a hard-disk farm which had a high-speed HIPPI interface. One 680x0 CPU board controlled the disk farm. Dave Kim, a good friend, wrote the software, KDOS, that ran on the CPU board and provided a file-system view of the disk farm. An FTP server was needed to allow clients on Unix workstations to retrieve data from the disk farm. Dave's CPU board apparently didn't have a network interface, so the FTP server ran on an adjacent CPU board that did have a network interface. The only way to communicate between the two boards was through shared-RAM-based mailboxes and data buffers.
Inspired by one of Dave's programs, I wrote a KDOS daemon, kdosd, which ran on the disk farm's CPU board. It would read ASCII text mailbox messages from the FTP board and perform the requested file operations on the disk farm using Dave's libkdos library. An example request might be to read so many bytes from a file and store the data in a buffer accessible to both CPU boards.
On the FTP board, we ran nafta, an FTP server I
wrote using the NFT_UTIL package. (Following in the footsteps of
YACC, NAFTA stands for "Not Another File Transfer Application"!)
When a client workstation connected to nafta,
the server would create a new NftSession object and
register KDOS-specific functions to be evaluated for the different
FTP commands. For example, to get a file from the disk farm (FTP's
"RETR" or retrieve command), nafta's "RETR"-processing
function would send an "OPEN" message to kdosd
on the disk farm board to open the requested file, a series of
"READ" messages to read the file's contents and store them in a
data buffer accessible to both boards, and a final "CLOSE" message
to close the file. Once the file's contents were available on the
board, nafta could then forward them over the
network to the FTP client to complete the "RETR" request. (To
indirectly perform the KDOS file operations using mailbox messages,
nafta, for which I no longer have the source code,
probably used the KDOS Asynchronous
I/O Utilities.)
Back in the late 1990s, I also used the Unix FTP version of
nafta, but later replaced it with
anise,
my pocket WWW/FTP server, which uses the NFT_UTIL package for FTP.
The nftPASS() function handled user authentication under
SunOS 4.1.3 and VxWorks, but I've not updated it (or researched what
needs to be updated) since the 1990s. So, the NFT_UTIL package
basically allows free access to its FTP server.
In 2016, I tried out anise's FTP server, which hadn't
been exercised in years. I noticed and fixed a bug in the "LIST"
command (incorrect file sizes and timestamps). I also changed
NFT_UTIL to use a simple array to store command-function mappings
instead of a hash table; modern compilers now complain (correctly)
that a function (code) pointer cannot be converted to and from a
(void *) data pointer, as was being done when I used hash tables.
In the process, I reacquainted myself with how the
NftSession object works and I got to thinking about
one shortcoming that became apparent: when a session object is
created, the server has to register any application-specific
command-processing functions and set other server-specific parameters.
To do so, the server has to maintain some state (whether in data or
code form) outside the NftSession object that
tells it, the server, how to initialize the session object when a
new client establishes a connection. I suppose it would be a good
idea to design an NftServer object that holds the
command-function mappings and other parameters for its impermanent,
child NftSession objects. Oh, well ...
(Dave Kim was a great person to work with. He was one of the few people I've known in a long software career—I think I can count them on the fingers of one hand—who build up their own personal software libraries; i.e., functions, tools, etc. that help them be more productive than others who depend solely on a project's official libraries or tools. In the non-technical realm, I've loved sailboats ever since I was a young kid; I've only ever been sailing once, on the Chesapeake Bay, and, like the landlubber I am at heart and stomach, I got royally seasick. Despite that, I continued to love sailboats and, in Dave Kim, I met a real, live, racing sailor—no cushy cruising for him! I learned a lot about sailing, particularly when racing, from him and I enjoyed his boating stories. He also lent me his copy of The Voyage of American Promise by Dodge Morgan, the first American to sail solo around the globe with no stops. "It takes three things to sail around the world alone. A good boat, an iron will and luck... And, my friends, here is a great boat." A fascinating book, not least because Morgan showed that you don't get sick when you're not around other people, even if you're continually drenched in ice-cold water!)
The NFT_UTIL package provides the basis for implementing a File Transfer
Protocol (FTP) server. The implementation of the NFT_UTIL package itself
and its companion package of command processing functions,
nft_proc.c, was based on the following Request for Comments
(available at a number of Internet sites):
- RFC 765 - File Transfer Protocol (obsolete)
- Although superseded by RFC 959, this RFC described the (defunct?) mail-related FTP commands: MAIL, MLFL, MSAM, MSOM, MSRQ, and MRCP.
- RFC 959 - File Transfer Protocol (FTP)
- The official FTP RFC, well-written, not dry.
- RFC 1123 - Requirements for Internet Hosts -- Application and Support
- This all-encompassing RFC clarified some remaining issues in the FTP standard and took into account existing practice.
as well as some empirical testing with the SunOS 4.1.3 and HP/UX 9.05 FTP servers.
An FTP server listens for and accepts network connection requests from clients who wish to transfer files. A session for a particular client begins when that client first connects to the FTP server and ends when the client is disconnected from the server; an FTP server with multiple clients would have multiple sessions active simultaneously. Associated with each session are two network connections:
Control - is the connection over which commands are sent to the FTP server and replies returned to the client. The control connection stays open for the life of the session.
Data - is a connection over which files and other data are sent and received. A new data connection is established for each data transfer (e.g., the sending of a single file) and closed when the transfer completes.
FTP commands are CR/LF-terminated, ASCII strings consisting of an upper case, 3- or 4-character keyword followed by zero or more, space-separated arguments; for example, the following command requests the retrieval of a file:
RETR thisFile
Some of the more common FTP commands are:
USER userName(Automatically issued by the FTP client)PASS password- log a user onto the FTP server's host.
CWD newDirectory(FTP Client Command:cd newDirectory)CDUP(FTP Client Command:cd ..)- change the current working directory. CDUP moves up to the parent of the current working directory.
PWD(FTP Client Command:pwd)- returns the current working directory.
NLST [directory](FTP Client Command:ls)- returns a list of the files in the specified directory (which defaults to the current working directory).
TYPE A|I(FTP Client Command:asciiorbinary)- specifies the type of data being transferred, "A" for ASCII and "I" (Image) for binary.
PORT hostAndPort(Automatically issued by the FTP client)- specifies a port on the client's host to which the FTP server should connect for data transfers. This command is usually issued anew prior to each file transfer or directory listing.
RETR fileName(FTP Client Command:get fileName)- retrieves the specified file from the FTP server's host.
STOR fileName(FTP Client Command:put fileName)- store the to-be-transferred data on the FTP server's host under the specified file name.
QUIT(FTP Client Command:byeorquit)- terminates the session.
FTP replies consist of a 3-digit, numeric status code followed by
descriptive text. The status codes are enumerated in RFC 959
(but see RFC 1123 for some updates). For example, the
RETR command typically results in two replies being returned
to the client over the control connection:
"150 Data connection opened: thisFile (98765 bytes)"
... the contents of "thisFile" are sent over the data connection ...
"226 Transfer complete: thisFile (98765 bytes)
RFCs 959 and 1123 specify what status codes should be used in reply to which commands. Although there are a few exceptions, the implementor is free to choose the format and contents of the reply text.
The NFT package provides a server implementation with a high-level means of conducting an FTP session. The server is responsible for listening for and answering a network connection request from a client:
TcpEndpoint client, server ;
...
tcpListen ("ftp", 99, &server) ;
tcpAnswer (server, -1.0, &client) ;
Once a client connection has been established, an FTP session can be created:
NftSession session ;
...
nftCreate (client, NULL, NULL, NULL, NULL, &session) ;
The server is now ready to read and process FTP commands from the client:
char *command ;
...
nftGetLine (session, &command) ;
nftEvaluate (session, command) ;
When the client connection is broken or an FTP QUIT command is received, the server should terminate the FTP session:
nftDestroy (session) ;
Putting all of the above together yields a simple - but working - FTP server:
#include <stdio.h> -- Standard I/O definitions.
#include "tcp_util.h" -- TCP/IP networking utilities.
#include "nft_util.h" -- FTP utilities.
int main (int argc, char *argv[])
{
char *command ;
TcpEndpoint client, server ;
NftSession session ;
- Listen at port 21.
tcpListen ((argc > 1) ? argv[1] : "21", 99, &server) ;
for ( ; ; ) { -- Answer next client.
tcpAnswer (server, -1.0, &client) ;
nftCreate (client, NULL, NULL, NULL, NULL, &session) ;
nftPutLine (session, "220 Service is ready.\n") ;
for ( ; ; ) { -- Service connected client.
if (nftGetLine (session, &command)) break ;
nftEvaluate (session, command) ;
if ((nftInfo (session))->logout) break ;
}
nftDestroy (session) ; -- Lost client.
}
}
The server's name is specified as the first argument on the command line
(i.e., argv[1]) and defaults to port 21. If a client
connection is broken, the server loops back to wait for the next client.
In event-driven applications (e.g., those based on the X Toolkit or the
IOX dispatcher), the socket connection
underlying an FTP session, returned by nftFd(), can be
monitored for input by your event dispatcher. Because input is buffered,
the input callback must repeatedly call nftGetLine() or
nftRead() while nftIsReadable() is true.
The NFT_UTIL package provides a means for modifying or extending the
functionality of an NFT-based FTP server; it does this by maintaining
a table that maps FTP command keywords to the C functions that process
those commands. When called to evaluate a command string,
nftEvaluate() parses the command line into an
argc/argv array of arguments. It then
looks up the command keyword (argv[0]) in the table and
calls the command processing function bound to that keyword, passing
the argc/argv array of arguments to the function.
nftCreate() initializes a session's keyword-function map
with default entries for the commands called for in the RFCs. These
default entries are defined in the defaultCommands[] and
defaultCallbacks[] arrays below. The default command
processing functions—except for those for PASV, PORT, and
QUIT—are found in the companion package, nft_proc.c.
An application can modify the processing of an existing command by
registering a new command processing function for the command.
The following example replaces the default nftRETR()
retrieval function with myRetrieve():
extern errno_t myRetrieve (NftSession session,
int argc, const char *argv[],
void *userData) ;
...
nftRegister (session, "RETR", myRetrieve) ;
nftRegister() can also be used to add entirely new FTP commands
to the keyword-function map. When replacing an existing command's callback,
study the RFCs and the default implementation of the command so as to
ensure that your implementation covers all the bases.
Application-specific command processing functions registered with
nftRegister() and invoked by nftEvaluate()
should be declared as follows:
errno_t myFunction (NftSession session,
int argc,
const char *argv[],
void *userData)
{
... body of function ...
}
argc is the number of arguments (plus the command keyword) in the
command string being evaluated; argv[] is an array of character
strings, each one being one of the arguments. For example, an FTP ALLO command,
"ALLO 123 456"
would be parsed into an argc/argv array as shown here:
argc = 3 argv[0] = "ALLO"
argv[1] = "123" -- File size
argv[2] = "456" -- Maximum record/page size
The command processing function is responsible for verifying the number
and validity of a command's arguments. The userData argument
is an arbitrary (void *) pointer specified when the session
was created.
A number of NFT functions are available for use in a command processing
function. A pointer to the public information in a session structure
can be obtained with a call to nftInfo():
NftSessionInfo *info = nftInfo (session) ;
nftPutLine() should be used to format and send a reply message
to the client over the session's control connection; the following example
returns a response for the PWD command:
nftPutLine (session, "257 \"%s\"\n", info->currentDirectory) ;
Commands such as RETR, STOR, and LIST must establish a separate network
connection for transferring data back and forth. A basic implementation
of the RETR command illustrates the use of the nftOpen(),
nftWrite(), and nftClose() functions to transfer data:
#include <stdio.h> -- Standard I/O definitions.
#include <string.h> -- C Library string functions.
#include "nft_util.h" -- FTP utilities.
errno_t myRetrieve (NftSession session,
int argc,
const char *argv[],
void *userData)
{
char buffer[1024], fileName[1024] ;
FILE *file ;
size_t length ;
NftSessionInfo *info = nftInfo (session) ;
-- Append the file name to the current working
-- directory and open the file for reading.
strcpy (fileName, info->currentDirectory) ;
strcat (fileName, argv[1]) ;
file = fopen (fileName, "rb") ;
-- Establish a network connection with the client
-- for the purpose of transferring data.
nftOpen (session) ;
nftPutLine (session, "150 Transferring: %s\n", fileName) ;
-- Send the contents of the file to the client.
while ((length = fread (buffer, 1, sizeof buffer, file)) > 0)
nftWrite (session, length, buffer, NULL) ;
-- End the data transfer.
nftPutLine (session, "226 Transfer complete.\n") ;
nftClose (session) ;
fclose (file) ;
return (0) ;
}
The STOR command is implemented in a similar fashion, with the calls to
fread(3) and nftWrite() replaced by calls to
nftRead() and fwrite(3), respectively. The
NFT_UTIL package handles the PORT and PASV commands that precede a data
transfer command, so nftOpen() will establish the appropriate
type of data connection with the application (or its programmer) being
none the wiser.
The myRetrieve() function above does not strictly conform to
the FTP standard in that it only supports binary transfers
("TYPE I") of arbitrary files. Text files should be
transferred according to the Telnet protocol. In particular, each
end-of-line marker (e.g., "\n") in a text file must be
transmitted as a Telnet end-of-line sequence: a carriage return followed
by a line feed ("\r\n"). The FTP client and the server are
responsible for making the appropriate conversions to and from their hosts'
end-of-line conventions when sending or receiving ASCII data.
nftRead() and nftWrite() do not perform
these conversions for you. However, the CR/LF utilities (see
crlf_util.c) simplify the handling of ASCII text. Using these
conversion utilities, the myRetrieve() function needs to be
modified in only two places in order to support ASCII text transfers:
#include "crlf_util.h" -- CR/LF utilities.
errno_t myRetrieve (...)
{
...
-- Open the file for reading.
if (info->representation[0] == 'A') -- ASCII transfer?
file = fopen (fileName, "r") ; -- Open ASCII file.
else
file = fopen (fileName, "rb") ; -- Open binary file.
...
-- Send the contents of the file to the client.
while ((length = fread (buffer, 1, (sizeof buffer)/2, file)) > 0) {
if (info->representation[0] == 'A') {
nl2crlf (buffer, length, sizeof buffer) ;
length = strlen (buffer) ;
}
if (nftWrite (session, length, buffer, NULL)) break ;
}
...
}
Within the send loop, nl2crlf() is called to convert newline
characters in the file to the carriage return/line feed sequence. Note that
only half of the buffer is used for reading, thus allowing for the worst case
scenario of a string consisting entirely of newlines being expanded to
twice its size.
nftClose() - closes a session's data connection.
nftCreate() - creates an FTP session.
nftDestroy() - deletes an FTP session.
nftEvaluate() - evaluates an FTP command.
nftFd() - returns a session's control or data socket number.
nftGetLine() - reads the next line from a session's control connection.
nftIgnoreCmd() - ignores an FTP command.
nftInfo() - returns a pointer to the session's public information.
nftIsReadable() - checks if input is waiting to be read from a session's control or data connection.
nftIsUp() - checks if a session's control or data connection is up.
nftIsWriteable() - checks if data can be written to a session's control or data connection.
nftLookup() - looks up the function to process an FTP command.
nftName() - returns the name of a session's control or data connection.
nftNextCommand() - reads and evaluates the next FTP command.
nftOpen() - opens a session's data connection.
nftPASV() - processes the FTP PASV command.
nftPeer() - returns the name of the session's peer.
nftPORT() - processes the FTP PORT command.
nftPutLine() - writes a line of output to a session's control connection.
nftQUIT() - processes the FTP QUIT command.
nftRead() - reads input from a session's data connection.
nftRegister() - maps an FTP command to a user-supplied function.
nftSyntax() - returns the syntax of an FTP command.
nftWrite() - writes output to a session's data connection.
nft_util.c
nft_util.h
nftAccessCmds() - processes FTP access control commands.
nftCWD() - processes the FTP CWD command.
nftFileCmds() - processes the FTP file management commands.
nftHELP() - processes the FTP HELP command.
nftListCmds() - processes the FTP LIST and NLST commands.
nftMODE() - processes the FTP MODE command.
nftPASS() - processes the FTP PASS command.
nftRETR() - processes the FTP RETR command.
nftServiceCmds() - processes FTP service commands.
nftSTAT() - processes the FTP STAT command.
nftStoreCmds() - processes the FTP APPE and STOR commands.
nftSTRU() - processes the FTP STRU command.
nftTYPE() - processes the FTP TYPE command.
nftUSER() - processes the FTP USER command.
nft_proc.c
nft_proc.h