All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
$Id: fetch.3,v 1.2 1998/09/26 20:42:44 des Exp $
.Dd July 1, 1998 .Dt FETCH 3 .Os .Sh NAME .Nm fetchGetURL , .Nm fetchPutURL , .Nm fetchParseURL , .Nm fetchGet , .Nm fetchPut , .Nm fetchGetFile , .Nm fetchPutFile , .Nm fetchGetHTTP , .Nm fetchPutHTTP , .Nm fetchGetFTP , .Nm fetchPutFTP .Nd file transfer library .Sh SYNOPSIS .Fd #include <fetch.h> .Ft FILE * .Fn fetchGetURL "char *URL" "char *flags" .Ft FILE * .Fn fetchPutURL "char *URL" "char *flags" .Ft url_t * .Fn fetchParseURL "char *URL" "char *flags" .Ft FILE * .Fn fetchGet "url_t *URL" "char *flags" .Ft FILE * .Fn fetchPut "url_t *URL" "char *flags" .Ft FILE * .Fn fetchGetFile "url_t *u" "char *flags" .Ft FILE * .Fn fetchPutFile "url_t *u" "char *flags" .Ft FILE * .Fn fetchGetHTTP "url_t *u" "char *flags" .Ft FILE * .Fn fetchPutHTTP "url_t *u" "char *flags" .Ft FILE * .Fn fetchGetFTP "url_t *u" "char *flags" .Ft FILE * .Fn fetchPutFTP "url_t *u" "char *flags" .Vt extern int fetchLastErrCode; .Vt extern const char *fetchLastErrText; .Sh DESCRIPTION These functions implement a high-level library for retrieving and uploading files using Uniform Resource Locators (URLs).
p .Fn fetchGetURL and .Fn fetchPutURL constitute the recommended interface to the .Nm fetch library. They examine the URL passed to them to determine the transfer method, and call the appropriate lower-level functions to perform the actual transfer. The .Fa flags argument is a string of characters which specify transfer options. The meaning of the individual flags is scheme-dependent, and is detailed in the appropriate section below.
p .Fn fetchParseURL takes a URL in the form of a null-terminated string and splits it into its components function according to the Common Internet Scheme Syntax detailed in RFC1738. A regular expression which produces this syntax is: d -literal <scheme>:(//(<user>(:<pwd>)?@)?<host>(:<port>)?)?/(<document>)? .Ed
p Note that some components of the URL are not necessarily relevant to all URL schemes. For instance, the file scheme only needs the <scheme> and <document> components.
p The pointer returned by .Fn fetchParseURL should be freed using .Fn free .
p .Fn fetchGet and .Fn fetchPut are similar to .Fn fetchGetURL and .Fn fetchPutURL , except that they expect a pre-parsed URL in the form of a pointer to an .Fa url_t structure rather than a string.
p All of the .Fn fetchGetXXX and .Fn fetchPutXXX functions return a pointer to a stream which can be used to read or write data from or to the requested document, respectively. Note that although the implementation details of the individual access methods vary, it can generally be assumed that a stream returned by one of the .Fn fetchGetXXX functions is read-only, and that a stream returned by one of the .Fn fetchPutXXX functions is write-only. .Sh FILE SCHEME .Fn fetchGetFile and .Fn fetchPutFile provide access to documents which are files in a locally mounted file system. Only the <document> component of the URL is used.
p .Fn fetchGetFile does not accept any flags.
p .Fn fetchPutFile accepts the .Fa a (append to file) flag. If that flag is specified, the data written to the stream returned by .Fn fetchPutFile will be appended to the previous contents of the file, instead of replacing them. .Sh FTP SCHEME .Fn fetchGetFTP and .Fn fetchPutFTP implement the FTP protocol as described in RFC959.
p If the .Fa p (passive) flag is specified, a passive (rather than active) connection will be attempted.
p If no user name or password is given, the .Nm fetch library will attempt an anonymous login, with user name "ftp" and password "ftp". .Sh HTTP SCHEME The .Fn fetchGetHTTP and .Fn fetchPutHTTP functions implement the HTTP/1.1 protocol. With a little luck, there's even a chance that they comply with RFC2068.
p Since there seems to be no good way of implementing the HTTP PUT method in a manner consistent with the rest of the .Nm fetch library, .Fn fetchPutHTTP is currently unimplemented. .Sh RETURN VALUES .Fn fetchParseURL returns a pointer to a .Fa url_t structure containing the individual components of the URL. If it is unable to allocate memory, or the URL is syntactically incorrect, .Fn fetchParseURL returns a NULL pointer.
p All other functions return a stream pointer which may be used to access the requested document, or NULL if an error occurred. In the latter case, the variables .Va fetchLastErrCode and .Va fetchLastErrText are set to appropriately descriptive values. .Sh ENVIRONMENT The FTP and HTTP related functions use the .Ev HTTP_PROXY and .Ev FTP_PROXY environment variables, respectively, as the address of a proxy server to use for transferring files. .Sh SEE ALSO .Xr fetch 1 , .Xr ftpio 3 .Rs .%A T. Berners-Lee, L. Masinter & M. McCahill .%D December 1994 .%T Uniform Resource Locators (URL) .%O RFC1738 .Re .Rs .%A R. Fielding, J. Gettys, J. Mogul, H. Frystyk, T. Berners-Lee .%D Januray 1997 .%B Hypertext Transfer Protocol -- HTTP/1.1 .%O RFC2068 .Re .Rs .%A J. Postel, J. K. Reynolds .%D October 1985 .%B File Transfer Protocol .%O RFC959 .Re .Sh DIAGNOSTICS Add later. .Sh NOTES Some parts of the library are not yet implemented. The most notable examples of this are .Fn fetchPutHTTP and proxy support for the FTP access method. .Sh HISTORY The .Nm fetch library first appeared in .Fx 3.0 . .Sh AUTHORS The .Nm fetch library was mostly written by .An Dag-Erling Co�dan Sm�rgrav Aq des@FreeBSD.org with numerous suggestions from .An Jordan K. Hubbard Aq jkh@FreeBSD.org and other FreeBSD developers. It incorporates the older .Nm ftpio library, which was originally written by .Nm Poul-Henning Kamp Aq pkh@FreeBSD.org and later turned inside out by .An Jordan K. Hubbard Aq jkh@FreeBSD.org .
p This manual page was written by .An Dag-Erling Co�dan Sm�rgrav Aq des@FreeBSD.org .Sh BUGS There's no way to select a proxy at run-time other than setting the .Ev HTTP_PROXY or .Ev FTP_PROXY environment variables as appropriate. There is also no way to stop the FTP and HTTP functions from trying to use a proxy if these variables are set.
p HTTP authentication doesn't work. I'm not sure that's a bug in my code; as far as I can determine, .Nm libfetch handles HTTP/1.1 basic authentication correctly as outlined in RFC2068, but I haven't been able to find an HTTP server that honors the Authentication: header field. Also, .Nm libfetch does not attempt to interpret and respond to authentication requests from the HTTP server.
p No attempt is made to encode spaces etc. within URLs. Spaces in the document part of an URLshould be replaced with "%20" in HTTP URLs and "\ " in FTP URLs.
p Error numbers are unique only within a certain context; the error codes used for FTP and HTTP overlap, as do those used for resolver and system errors. For instance, error code 202 means "Command not implemented, superfluous at this site" in an FTP context and "Accepted" in an HTTP context.
p The man page is poorly written and produces badly formatted text.
p Tons of other stuff.