#
339250 |
|
09-Oct-2018 |
des |
MFH (r314778): use reallocarray(3) for extra bounds checks MFH (r333306): fix typo in man page MFH (r333571, r333572): preserve if-modified-since across redirects MFH (r334317): simplify the DEBUG macro MFH (r334319): style bug roundup MFH (r334326): fix netrc file location logic, improve netrcfd handling MFH (r338572): fix end-of-transfer statistics, improve no-tty display
PR: 202424, 224426, 228017
|
#
331722 |
|
29-Mar-2018 |
eadler |
Revert r330897:
This was intended to be a non-functional change. It wasn't. The commit message was thus wrong. In addition it broke arm, and merged crypto related code.
Revert with prejudice.
This revert skips files touched in r316370 since that commit was since MFCed. This revert also skips files that require $FreeBSD$ property changes.
Thank you to those who helped me get out of this mess including but not limited to gonzo, kevans, rgrimes.
Requested by: gjb (re)
|
#
330897 |
|
14-Mar-2018 |
eadler |
Partial merge of the SPDX changes
These changes are incomplete but are making it difficult to determine what other changes can/should be merged.
No objections from: pfg
|
#
323659 |
|
17-Sep-2017 |
marius |
MFC: r322669
In fetch_resolve() if the port number or service name is included in the host argument (e. g. "www.freebsd.org:443"), correctly set the service pointer accordingly. Previously, the service pointer was set to the separator instead, causing getaddrinfo(3) to fail.
|
#
315902 |
|
24-Mar-2017 |
des |
MFH (r313974,r314596): open .netrc early in case we want to drop privs MFH (r314396,r315143): fix a crash caused by an incorrect format string MFH (r314701): fix handling of 416 errors when requesting a range MFH (r315455): fix parsing of IP literals (square brackets)
PR: 212065, 217723
|
#
310059 |
|
14-Dec-2016 |
des |
MFH (r308996, r309051, r309738): refactor, avoid repeating DNS requests
|
#
302408 |
|
07-Jul-2016 |
gjb |
Copy head@r302406 to stable/11 as part of the 11.0-RELEASE cycle. Prune svn:mergeinfo from the new branch, as nothing has been merged here.
Additional commits post-branch will follow.
Approved by: re (implicit) Sponsored by: The FreeBSD Foundation |
#
300665 |
|
25-May-2016 |
truckman |
Don't leak addrinfo in fetch_bind()
Reported by: Coverity CID: 1225038 MFC after: 1 week
|
#
298896 |
|
01-May-2016 |
pfg |
lib: minor spelling fixes in comments.
No functional change.
|
#
294326 |
|
19-Jan-2016 |
bapt |
Test for /etc/ssl/cert.pem existence to avoid masking SSL_CA_CERT_PATH
Prior to this patch, unless SSL_CA_CERT_FILE is set in the environment, libfetch will set the CA file to "/usr/local/etc/cert.pem" if it exists, and to "/etc/ssl/cert.pem" otherwise. This has the consequence of masking SSL_CA_CERT_PATH, because OpenSSL will ignore the CA path if a CA file is set but fails to load (see X509_STORE_load_locations()).
While here, fall back to OpenSSL defaults if neither SSL_CA_CERT_FILE nor SSL_CA_CERT_PATH are set in the environment, and if neither of the libfetch default CA files exists.
PR: 193871 Submitted by: John W. O'Brien <john@saltant.com> Approved by: des MFC after: 1 week
|
#
288217 |
|
25-Sep-2015 |
grembo |
Fix non-POSIX-compliant use of getaddrinfo in libfetch
Submitted by: Boris Kolpackov <boris@codesynthesis.com> Reviewed by: bapt Approved by: bapt MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D3724
|
#
280630 |
|
25-Mar-2015 |
jkim |
Remove defunct SSLv2 support from fetch(1) and fetch(3).
|
#
273124 |
|
15-Oct-2014 |
des |
As pointed out by several people, r273114 was incorrect: it unconditionally disabled everything except TLS 1.0. Replace it with a more carefully wrought patch:
- Switch the default for SSLv3 from on to off - Add environment variables to control TLS 1.1 and 1.2 - In verbose mode, report which version is used - Update the man page to reflect these changes.
MFC after: 1 week
|
#
273114 |
|
15-Oct-2014 |
des |
Drop support for SSLv3.
MFC after: 1 week
|
#
268671 |
|
15-Jul-2014 |
bapt |
Add a comment to explain the EAGAIN is only there for POSIX compliance
Resquested by: kib Reviewed by: des
|
#
268493 |
|
10-Jul-2014 |
bapt |
Support EAGAIN in fetch_writev
Reviewed by: des Approved by: des
|
#
266291 |
|
17-May-2014 |
des |
Look for root certificates in /usr/local/etc/ssl before /etc/ssl.
MFH: 1 week
|
#
261284 |
|
30-Jan-2014 |
des |
Bump copyright dates
|
#
261263 |
|
29-Jan-2014 |
des |
r261230 broke the cases where the amount of data to be read is not known in advance, or where the caller doesn't care and just keeps reading until it hits EOF.
In fetch_read(): the socket is non-blocking, so read() will return 0 on EOF, and -1 (errno == EAGAIN) when the connection is still open but there is no data waiting. In the first case, we should immediately return 0. The EINTR case was also broken, although not in a way that matters.
In fetch_writev(): use timersub() and timercmp() as in fetch_read().
In http_fillbuf(): set errno to a sensible value when an invalid chunk header is encountered.
In http_readfn(): as in fetch_read(), a zero return from down the stack indicates EOF, not an error. Furthermore, when io->error is EINTR, clear it (but no errno) before returning so the caller can retry after dealing with the interrupt.
MFC after: 3 days
|
#
261230 |
|
28-Jan-2014 |
des |
Solve http buffering issues and hangs once and for all (hopefully!) by simply not trying to return exactly what the caller asked for - just return whatever we got and let the caller be the judge of whether it was enough. If an error occurs or the connection times out after we already received some data, return a short read, under the assumption that the next call will fail or time out before we read anything.
As it turns out, none of the code that calls fetch_read() assumes an all-or-nothing result anyway, except for a couple of lines where we read the CR LF at the end of a hunk in HTTP hunked encoding, so the changes outside of fetch_read() and http_readfn() are minimal.
While there, replace select(2) with poll(2).
MFC after: 3 days
|
#
260904 |
|
20-Jan-2014 |
des |
Fix format string.
Submitted by: Jörg Sonnenberger <joerg@NetBSD.org> MFC after: 1 week
|
#
258349 |
|
19-Nov-2013 |
bdrewery |
Fix build with GCC
SSL_set_tlsext_host_name(3) internally does not modify the host buffer pased to it. So it is safe to DECONST the struct url* here.
Reported by: gjb Approved by: bapt (implicit) MFC after: 1 week X-MFC-With: r258347
|
#
258347 |
|
19-Nov-2013 |
bdrewery |
Support SNI in libfetch
SNI is Server Name Indentification which is a protocol for TLS that indicates the host that is being connected to at the start of the handshake. It allows to use Virtual Hosts on HTTPS.
Submitted by: sbz Submitted by: Michael Gmelin <freebsd@grem.de> [1] PR: kern/183583 [1] Reviewed by: des Approved by: bapt MFC after: 1 week
|
#
253680 |
|
26-Jul-2013 |
des |
Implement certificate verification, and many other SSL-related imrovements; complete details in the PR.
PR: kern/175514 Submitted by: Michael Gmelin <freebsd@grem.de> MFC after: 1 week
|
#
236193 |
|
28-May-2012 |
jilles |
libfetch: Avoid SIGPIPE on network connections.
To avoid unexpected process termination from SIGPIPE when writing to a closed network connection, enable SO_NOSIGPIPE on all network connections.
The POSIX standard MSG_NOSIGNAL is not used since it requires modifying all send calls to add this flag. This is particularly nasty for SSL connections.
Reviewed by: des Tested by: bapt MFC after: 5 days
|
#
234837 |
|
30-Apr-2012 |
des |
Since the socket is non-blocking, it is necessary to use select(2) even when there is no timeout, because read(2) will return immediately if there is no data waiting in the TCP buffer, causing fetch_read() to busy-loop on slow connections.
MFC after: 3 weeks Noticed by: Yanhui Shen <shen.elf@gmail.com>
|
#
230478 |
|
23-Jan-2012 |
des |
Fix two nits in previous commit pointed out by pjd@.
MFC after: 3 weeks
|
#
230307 |
|
18-Jan-2012 |
des |
Fix two issues related to the use of SIGINFO in fetch(1) to display progress information. The first is that fetch_read() (used in the HTTP code but not the FTP code) can enter an infinite loop if it has previously been interrupted by a signal. The second is that when it is interrupted, fetch_read() will discard any data it may have read up to that point. Luckily, both bugs are extremely timing-sensitive and therefore difficult to trigger.
PR: bin/153240 Submitted by: Mark <markjdb@gmail.com> MFC after: 3 weeks
|
#
226537 |
|
19-Oct-2011 |
des |
latin1 -> utf8
|
#
225814 |
|
27-Sep-2011 |
des |
Update copyright dates and strip my middle name.
|
#
221830 |
|
13-May-2011 |
des |
Mark all socket and file descriptors close-on-exec.
PR: bin/151866 MFC after: 3 weeks
|
#
214256 |
|
23-Oct-2010 |
emaste |
Move variable declarations into the conditional block where they are used, to fix warning if WITH_SSL is not set.
Submitted by: Sean Bruno MFC after: 1 week
|
#
210568 |
|
28-Jul-2010 |
des |
Redo fetch_read() using non-blocking sockets. This is necessary to avoid a hang in the SSL case if the server sends a close notification before we are done reading. In the non-SSL case, it can provide a minor (but probably not noticeable) performance improvement for small transfers.
MFC after: 3 weeks
|
#
198339 |
|
21-Oct-2009 |
fabient |
PR: 139751 Approved by: des Obtained from: Xavier Heiny <xavier.heiny@netasq.com> MFC after: 3 weeks
|
#
178234 |
|
15-Apr-2008 |
cperciva |
Fix one-byte buffer overflow: NUL gets written to the buffer, but isn't counted in the width specification in scanf.
This is not a security problem, since this function is only used to parse a user's configuration file.
Submitted by: Joerg Sonnenberger Obtained from: dragonflybsd MFC after: 1 week
|
#
177447 |
|
20-Mar-2008 |
des |
s/wait/delta/ to avoid namespace collision.
MFC after: 2 weeks
|
#
176105 |
|
08-Feb-2008 |
des |
Use memcpy(3) instead of the BSD-specific bcopy(3).
Submitted by: Joerg Sonnenberger <joerg@britannica.bec.de> MFC after: 2 weeks
|
#
174761 |
|
18-Dec-2007 |
des |
As several people pointed out, I did all the ctype casts the wrong way (not for the first time...)
Noticed by: bde, ru ++ MFC after: 1 week
|
#
174752 |
|
18-Dec-2007 |
des |
Add support for the NO_PROXY / no_proxy environment variable as used by lynx, curl etc. Note that this patch differs significantly from that in the PR, as the submitter refined it after submitting the PR.
PR: 110388 Submitted by: Alexander Pohoyda <alexander.pohoyda@gmx.net> MFC after: 3 weeks
|
#
174588 |
|
14-Dec-2007 |
des |
Clean up namespace violations.
MFC after: 1 week
|
#
141970 |
|
16-Feb-2005 |
des |
Fix a memory leak: when freeing the connection structure, don't forget to free the connection buffer as well.
PR: bin/76153 MFC after: 1 week
|
#
135546 |
|
21-Sep-2004 |
des |
Update copyright years.
|
#
121423 |
|
23-Oct-2003 |
ume |
preparation for RFC3493. EAI_NODATA was deprecated.
|
#
111816 |
|
03-Mar-2003 |
des |
Add and document support for a FETCH_BIND_ADDRESS environment variable specifying a local address to bind sockets to. Caveat: lightly tested.
PR: bin/37572
|
#
109967 |
|
28-Jan-2003 |
des |
style(9): add parentheses to sizeof even when not strictly required.
MFC after: 3 days
|
#
109960 |
|
27-Jan-2003 |
jwd |
Fix signed/unsigned comparison warning/error from 'make release'
|
#
109695 |
|
22-Jan-2003 |
des |
Experimental support for .netrc.
|
#
108579 |
|
03-Jan-2003 |
des |
Set auto-retry mode to avoid some spurious errors.
Submitted by: Andre Albsmeier <andre.albsmeier@siemens.com> MFC after: 7 days
|
#
106205 |
|
30-Oct-2002 |
des |
Fix a bug in fenner's _fetch_writev() patch (rev 1.36)
Submitted by: fenner
|
#
106186 |
|
30-Oct-2002 |
des |
Recommit the non-broken parts of 1.34 and 1.37. Change the type and name of a variable introduced in 1.33.
|
#
106175 |
|
29-Oct-2002 |
imp |
Reinstate revs 1.35-36 and 1.38. Revisions 1.34 and 1.37 were specifically the root cause of the bus errors I was experiencing.
Submitted by: fenner Tested by: obrien Prompted by: peter
|
#
106137 |
|
29-Oct-2002 |
obrien |
Fix `pkg_add -r' by backing out revs 1.34-1.38. Revs 1.37-8 produce a bus error in some environments. Revs 1.34-6 do not bus error, but write corrupted files.
|
#
106081 |
|
28-Oct-2002 |
des |
Fix an off-by-one error (> where >= should have been used) which caused _fetch_writev() to incorrectly report EPIPE in certain cases.
Also fix a number of const warnings by using __DECONST(), plus a signed / unsigned comparison by casting the rhs to ssize_t.
Submitted by: fenner, Craig Rodrigues <rodrigc@attbi.com>
|
#
106049 |
|
27-Oct-2002 |
des |
Slight amendment to rev 1.34: instead of considering any short read an error, only report an error if no data was read at all (unless len was 0 to start with). Otherwise, the final read of practically any transfer will end in a fatal error.
|
#
106046 |
|
27-Oct-2002 |
des |
Introduce _fetch_writev(), which is the conn_t version of writev(2). In the SSL case, it is no different from the old _fetch_write(), but in the non-SSL case it uses writev(2) to send the entire vector as a single packet (provided it can fit in one packet). Implement _fetch_write() and _fetch_putln() in terms of _fetch_writev().
This should improve performance in the non-SSL case (by reducing protocol overhead) and solve the problem where too-smart-for-their-own-good firewalls reject FTP packets that do not end in CRLF.
PR: bin/44123 Submitted by: fenner
|
#
106044 |
|
27-Oct-2002 |
des |
Eliminate two cases of undefined behaviour: total in _fetch_write() was not initialized before use, and _http_growbuf() did not return a value on success.
Reported by: Peter Edwards <pmedwards@eircom.net> MFC after: 2 weeks
|
#
106040 |
|
27-Oct-2002 |
des |
Back out the previous commit, and fix the bug rather than try to hide its symptoms: make timeouts and short transfers fatal, and set errno to an appropriate value (ETIMEDOUT for a timeout, EPIPE for a short transfer).
MFC after: 2 weeks
|
#
103718 |
|
20-Sep-2002 |
alfred |
Fix an infinite loop when _fetch_read() can return 0 (if the connection is broken), take this into account and return at this point.
|
#
103459 |
|
17-Sep-2002 |
fenner |
Make _fetch_connect() always set the error code. Tell ftp that _fetch_connect() always sets the error code (http already knew)
|
#
98748 |
|
24-Jun-2002 |
des |
Reintroduce debugging code that somehow got lost in a previous revision.
|
#
98117 |
|
11-Jun-2002 |
des |
Add a reference count to struct fetchconn so we don't prematurely close and free a cached FTP connection.
|
#
97891 |
|
05-Jun-2002 |
des |
Make SSL support conditional on NOCRYPT.
|
#
97868 |
|
05-Jun-2002 |
des |
Add SSL support + slight cleanup.
Submitted by: Henry Whincup <henry@techiebod.com> (in principle)
|
#
97866 |
|
05-Jun-2002 |
des |
Wrap everything in struct connection, and enforce timeouts everywhere (except for DNS operations). Always use funopen() for HTTP, to support both timeouts and SSL.
|
#
97856 |
|
05-Jun-2002 |
des |
First step towards SSL support: wrap connections in a 'struct connection' which contains the socket descriptor, the input buffer and (yet unused) SSL state variables. This has the neat side effect of greatly improving reentrance (though we're not *quite* there yet) and opening the door to HTTP connection caching.
This commit is inspired by email conversations with and patches from Henry Whincup <henry@techiebod.com> last fall.
|
#
90268 |
|
05-Feb-2002 |
des |
Switch to a self-starting allocation scheme.
|
#
90267 |
|
05-Feb-2002 |
des |
Reindent, and add parentheses to return statements. Some functions in ftp.c and http.c now have exceedingly long lines due to deep nesting; this will be corrected by reorganizing the code in a later revision.
|
#
88769 |
|
01-Jan-2002 |
des |
Remove VT100 escapes from debugging messages now that they're enabled by default.
PR: 32988 MFC after: 3 days
|
#
85143 |
|
19-Oct-2001 |
des |
Back out part of previous commit which was gcc-centric
|
#
85093 |
|
18-Oct-2001 |
des |
Tons of type, style and warning fixes that have been rotting in my tree for ages - some of which wouldn't be necessary if gcc wasn't broken or TPTB were willing to do something (-fno-builtin) about it.
|
#
84203 |
|
30-Sep-2001 |
dillon |
Add __FBSDID()s to libfetch
|
#
77237 |
|
26-May-2001 |
des |
MAXPATHLEN -> PATH_MAX
|
#
75891 |
|
23-Apr-2001 |
archie |
Apply 'const' liberally.
Fix some other minor glitches.
|
#
69271 |
|
27-Nov-2000 |
des |
Don't try to get the proxy port number from /etc/services.
|
#
68551 |
|
10-Nov-2000 |
des |
Use the documented (and historical) defaults. Centralize the decision logic in order to avoid this bug in the future.
Submitted by: se
|
#
63334 |
|
17-Jul-2000 |
des |
Introduce the (undocumented) variable fetchRestartCalls, which controls whether or not interrupted system calls will be restarted. This fixes a bug where fetch(1) would hang (potentially forever) if a server stopped responding, because the signal handler would absorb the user's efforts to interrupt the transfer.
|
#
62981 |
|
11-Jul-2000 |
des |
Add _fetch_putln()
|
#
62964 |
|
11-Jul-2000 |
des |
Clean up the debugging output
|
#
62911 |
|
10-Jul-2000 |
ume |
Free chunks obtained by getaddrinfo(3).
Approved by: des
|
#
60928 |
|
25-May-2000 |
des |
Output newline after debugging message
|
#
60924 |
|
25-May-2000 |
des |
Dump com_err, it's a stinking crock of shit.
|
#
60737 |
|
20-May-2000 |
ume |
IPv6 support.
|
#
60190 |
|
07-May-2000 |
des |
Use sizeof more consistently.
|
#
55557 |
|
07-Jan-2000 |
des |
More old uncommitted patches: implement timeouts at the protocol level. Currently only supported for ftp connections.
|
#
50476 |
|
27-Aug-1999 |
peter |
$Id$ -> $FreeBSD$
|
#
41989 |
|
21-Dec-1998 |
des |
Implement and document file list retrieval.
|
#
41923 |
|
18-Dec-1998 |
des |
Don't specify a language to compile_et. Rename fetchConnect to _fetch_connect since it's internal.
|
#
41862 |
|
16-Dec-1998 |
des |
Add verbose flag, and support functions. Brucify the Makefile. Differentiate atime and mtime in fetch*Stat(). Fix a few pointer bugs. Tweak some error messages. Don't #include sys/param.h and stdio.h in fetch.h. Document that sys/param.h and stdio.h must be #included before fetch.h.
|
#
40975 |
|
06-Nov-1998 |
des |
Second of a series of cleanups to libfetch.
This commit introduces the following features:
a) the fetchStat*() functions, which return meta-information for a document, such as size, modification time, etc.
b) the use of the com_err(3) facilities to report errors.
It also fixes a bunch of style bugs and a few logic bugs and somewhat improves the man page.
Changed files, in alphabetical order:
Makefile: Don't generate macros in {ftp,http}err.c.
Generate category fields for the error message lists.
Compile the error table.
Install fetch_err.h along with fetch.h.
common.c: Remove the _netdb_errstring() macro, and add FETCH_ERR_NETDB to the error code in the _netdb_seterr() macro.
Add categories to the _netdb_errlist table.
Report errors through the Common Error library.
common.h: Add the DEBUG macros.
Add prototype for fetchConnect().
Remove the prototype for _fetch_errstring(), which is local to common.c
Add a categroy field to struct fetcherr, and define constants for error categories.
Define macros for _{url,netdb,ftp,http}_seterr().
errors.et: (new file) List error categories.
fetch.3: Document the fetchStat*() functions.
Move the "unimplemented functionality" comments from NOTES to BUGS.
Document that applications which use libfetch must also use libcom_err, and list existing error codes.
Undocument fetchLastErr{Code,String}.
Remove the (empty) DIAGNOSTICS section.
Mention Eugene Skepner in the AUTHORS section.
fetch.c: Move the DEBUG macros to common.c
Add fetchStat() and fetchStatURL().
Generate error messages for URL parser errors, and fix a minor bug in the parser.
Use 'struct url' instead of 'url_t'.
Remove fetchLastErr{Code,String}.
fetch.h: Use 'struct url' instead of 'url_t', and remove the typedef.
Define struct url_stat (used by fetchStat()).
Add prototypes for fetchStat*().
Remove the declarations for fetchLastErr{Code,String}.
Include fetch_err.h.
fetch_err.et: (new file) Error table for libfetch.
file.c: Add fetchStatFile().
Use 'struct url' instead of 'url_t'.
ftp.c: Add fetchStatFTP().
Use 'struct url' instead of 'url_t'.
Don't use fetchLastErrCode.
ftp.errors: Add categories to all error messages.
http.c: Add fetchStatHTTP().
Use 'struct url' instead of 'url_t'.
Don't use fetchLastErr{Code,Text}.
http.errors: Add categories to all error messages.
Prompted by: jkh and Eugene Skepner Numerous sugestions from: Garett Wollman and Eugene Skepner
|
#
40939 |
|
05-Nov-1998 |
des |
First of a series of cleanups to libfetch. Changed files, in alphabetical order:
Makefile: Add common.c to SRCS.
Make debugging easier by making 'CFLAGS += -DNDEBUG' conditional on DEBUG
Don't declare struct {ftp,http}err in {ftp,http}err.c; use struct fetcherr instead.
README: Remove the todo list, which is out of date anyway.
common.c: (new file) Gather utility functions in this file.
Merge the error reporting functions intp _fetch_errstring(), _fetch_seterr() and _fetch_syserr().
Set fetchLastErrCode and fetchLastErrText appropriately when fetchConnect fails.
common.h: (new file) Gather internal prototypes and structures in this files.
fetch.3: Undocument fetchFreeURL().
Document a few more known bugs.
Document fetchLastErrCode and fetchLastErrText.
fetch.c: Add descriptive comments to all functions that lacked them.
Move fetchConnect() to common.c.
Obviate the need for fetchFreeURL(), and remove it.
fetch.h: Modify struct url_t so the document part is at the end.
ftp.c: Remove code that is duplicated elsewhere.
http.c: Remove code that is duplicated elsewhere.
Prompted by: jkh
|