#
267654 |
|
19-Jun-2014 |
gjb |
Copy stable/9 to releng/9.3 as part of the 9.3-RELEASE cycle.
Approved by: re (implicit) Sponsored by: The FreeBSD Foundation |
#
266655 |
|
25-May-2014 |
des |
MFH (r241737): staticize option variables MFH (r253804): credit Michael Gmelin for SSL code MFH (r251262): fix inode check for file change detection MFH (r261233): bump copyright dates MFH (r261234): increase minimum buffer size
Approved by: re (gjb)
|
#
256267 |
|
10-Oct-2013 |
glebius |
Merge r253680,253805 from head: ---------------------------------------------------------------------- r253680 | des | 2013-07-26 19:53:43 +0400 (Fri, 26 Jul 2013) | 7 lines
Implement certificate verification, and many other SSL-related imrovements; complete details in the PR.
PR: kern/175514 Submitted by: Michael Gmelin <freebsd@grem.de> MFC after: 1 week
---------------------------------------------------------------------- r253805 | des | 2013-07-30 17:07:55 +0400 (Tue, 30 Jul 2013) | 5 lines
Include an Accept header in requests.
PR: kern/180917 MFC after: 1 week
Reviewed by: des
|
#
252791 |
|
05-Jul-2013 |
andre |
MFC r243147:
Change fetch(1) to: o Report the instantaneous bandwidth instead of an average since the beginning of the download. o At the finish of the download report the average bandwidth and also the total time it took instead of 00m00s.
Reviewed by: des
MFC r244058:
Fix bandwidth reporting when doing a restarted download with "-r". The offset is already accounted for in xs->lastrcvd and doesn't have to be subtracted again.
Reported by: Florian Smeets <flo@smeets.im> Submitted by: Mateusz Guzik <mjguzik@gmail.com> Tested by: Florian Smeets <flo@smeets.im>
|
#
244499 |
|
20-Dec-2012 |
eadler |
MFC r244037: Add check for failure of mkstemp and setenv.
Approved by: cperciva (implicit)
|
#
231247 |
|
09-Feb-2012 |
bapt |
MFH r230307: fix SIGINFO-related data corruption bug
Approved by: des (mentor)
|
#
226016 |
|
04-Oct-2011 |
des |
MFH r225815: document that passive mode is the default MFH r225982: convert to UTF-8
Approved by: re (kib)
|
#
225816 |
|
27-Sep-2011 |
des |
MFH r225805: use fseeko(2) instead of fseek(2).
Approved by: re (kib)
|
#
225803 |
|
27-Sep-2011 |
des |
MFH r225800: finally fix resume mode with non-cooperating server.
Approved by: re (kib)
|
#
225736 |
|
22-Sep-2011 |
kensmith |
Copy head to stable/9 as part of 9.0-RELEASE release cycle.
Approved by: re (implicit)
|
#
225599 |
|
15-Sep-2011 |
des |
When resuming an HTTP download, we failed to verify that the range returned by the server matched what we requested, and blindly appended what we received to what we already had. This could go two ways: if the delivered offset was higher than expected, the local file would contain duplicate data, while if it was lower than expected, there would be data missing from the middle of the file. Furthermore, if the transfer was interrupted again, each subsequent attempt would compound the error. Fix the first problem by restarting the transfer from scratch if there is a gap, and the second by explicitly seeking to the correct location in the local file so as to overwrite any duplicated data.
PR: bin/117277 Approved by: re (kib) MFC after: 3 weeks
|
#
201290 |
|
30-Dec-2009 |
ru |
Treat an empty argument as an error, instead of fetching the contents of the root directory.
MFC after: 1 week
|
#
200462 |
|
13-Dec-2009 |
delphij |
Revert most part of 200420 as requested, as more review and polish is needed.
|
#
200420 |
|
11-Dec-2009 |
delphij |
Remove unneeded header includes from usr.bin/ except contributed code.
Tested with: make universe
|
#
187361 |
|
17-Jan-2009 |
des |
The FTP_TIMEOUT and HTTP_TIMEOUT environment variables were ignored because T_secs already had a non-zero default. Unbreak by moving the default to ftp_timeout / http_timeout.
Submitted by: Christoph Mallon <christoph.mallon@gmx.de> MFC after: 2 weeks
|
#
186241 |
|
17-Dec-2008 |
murray |
1. Update fetch to consistently return 1 on error, as the man page states, rather than usually returning 1 but in a few instances using a sysexits(3) return value.
2. Remove a few unused variables from libfetch.
PR: docs/122470 (1, only) Reviewed by: des > Description of fields to fill in above: 76 columns --| > PR: If a GNATS PR is affected by the change. > Submitted by: If someone else sent in the change. > Reviewed by: If someone else reviewed your modification. > Approved by: If you needed approval for this commit. > Obtained from: If the change is from a third party. > MFC after: N [day[s]|week[s]|month[s]]. Request a reminder email. > Security: Vulnerability reference (one per line) or description. > Empty fields above will be automatically removed.
M usr.bin/fetch/fetch.c M lib/libfetch/fetch.c
|
#
186124 |
|
15-Dec-2008 |
murray |
Add support for HTTP 1.1 If-Modified-Since behavior.
fetch(1) accepts a new argument -i <file> that if specified will cause the file to be downloaded only if it is more recent than the mtime of <file>.
libfetch(3) accepts the mtime in the url structure and a flag to indicate when this behavior is desired.
PR: bin/87841 Submitted by: Jukka A. Ukkonen <jau@iki.fi> (partially) Reviewed by: des, ru MFC after: 3 weeks
|
#
186043 |
|
13-Dec-2008 |
ru |
Improve usage and sync SYNOPSIS with usage.
|
#
185912 |
|
11-Dec-2008 |
des |
Apply the same defaults to https as to http.
Submitted by: Mike Tancsa <mike@sentex.net> MFC after: 1 week
|
#
181962 |
|
21-Aug-2008 |
obrien |
Use the SCHEME_ knobs rather than knowing what they expand to.
Reviewed by: des
|
#
164152 |
|
10-Nov-2006 |
des |
Try to preserve ownership and permissions when replacing an existing file.
PR: bin/104702 Submitted by: Kevin Day <toasty@dragondata.com> MFC after: 1 week
|
#
153919 |
|
30-Dec-2005 |
des |
Only clear sb.st_size if it is clearly wrong or meaningless. This fixes mirror mode.
PR: bin/86940 MFC after: 2 weeks
|
#
153894 |
|
30-Dec-2005 |
des |
Reflect progress in process title.
Submitted by: flz MFC after: 2 weeks
|
#
137854 |
|
18-Nov-2004 |
cperciva |
Fix buffer overflow. This is FreeBSD-SA-04:16.fetch.
Approved by: des
|
#
135546 |
|
21-Sep-2004 |
des |
Update copyright years.
|
#
134350 |
|
26-Aug-2004 |
des |
The check for r_flag was accidentally removed in the previous commit.
Submitted by: SANETO Takanori <sanewo@ba2.so-net.ne.jp> MFC after: 3 days
|
#
133779 |
|
15-Aug-2004 |
des |
Fix a couple of edge cases in which sb.st_size may be incorrect or meaningless. In particular, don't assume that it is left untouched if stat(2) fails; that assumption happens to fail at high optimization levels on some platforms.
MFC after: 1 week
|
#
132696 |
|
27-Jul-2004 |
des |
Don't strip trailing linear whitespace from passwords.
MFC after: 2 weeks
|
#
132695 |
|
27-Jul-2004 |
des |
Style nits.
|
#
131615 |
|
05-Jul-2004 |
des |
Truncate long file names in stat_display(), as was originally intended.
MFC after: 1 week
|
#
131052 |
|
24-Jun-2004 |
des |
The description of the -S option in the man page says we won't fail if the remote size is unknown, but we do. Resolve this in the man page's favor.
Requested by: Andre Albsmeier <andre.albsmeier@siemens.com> MFC after: 1 week
|
#
129440 |
|
19-May-2004 |
le |
Fix integer overflow in the file size output when dealing with large files (i.e. DVD images).
Reviewed by: des@
|
#
127941 |
|
06-Apr-2004 |
des |
Workaround for servers that ignore byte ranges when using chunked encoding.
MFC after: 3 days
|
#
125977 |
|
18-Feb-2004 |
des |
Use %zu for size_t, like God intended.
|
#
125976 |
|
18-Feb-2004 |
des |
Fix printf() format bug in previous commit (size_t != int). Use %ju and an (uintmax_t) cast to print size_t values. Use %jd and an (intmax_t) cast to print off_t values. Use off_t variables to hold the difference between two off_t values. Don't bother with unsigned char where char will do nicely.
|
#
125965 |
|
18-Feb-2004 |
des |
Revamp the statistics code, and switch to a much more compact display format. The old code tried to produce the exact same output as the pre-libfetch implementation, but I no longer see any value in this.
|
#
112114 |
|
11-Mar-2003 |
des |
Use floating point arithmetic to compute the ETA to avoid integer overflow during slow transfers of large files.
|
#
112083 |
|
11-Mar-2003 |
des |
Clean up the ETA logic a bit and make sure it works for restarted transfers.
|
#
109735 |
|
23-Jan-2003 |
des |
Slight cleanup of the ETA / BPS code. Avoid a division by zero.
|
#
109702 |
|
22-Jan-2003 |
des |
Add -N option to specify a netrc file. Fix handling of -v option. Don't treat negative offsets as valid positive ones. Clean up the ETA and transfer rate code. Show transfer rate along with ETA if the verbose level is higher than 1.
|
#
107353 |
|
27-Nov-2002 |
des |
Use the URL instead of the output file name when reporting errors relating to the remote document.
PR: 33856 MFC after: 1 week Approved by: re
|
#
106857 |
|
13-Nov-2002 |
des |
Don't set atime / mtime if we don't have an output file, which can happen in certain failure scenarii (mostly DNS trouble).
|
#
106796 |
|
12-Nov-2002 |
fenner |
Don't divide by zero to calculate the ETA, even if we haven't gotten any data in 30 seconds.
|
#
106586 |
|
07-Nov-2002 |
fenner |
Don't exit with success after timeout during authentication.
|
#
106051 |
|
27-Oct-2002 |
des |
Don't complain about not knowing the remote file size when working in quiet mode.
PR: bin/37079 Submitted by: Nicolas Rachinsky <list@rachinsky.de>
|
#
106043 |
|
27-Oct-2002 |
des |
Whitespace and indentation cleanup.
|
#
106042 |
|
27-Oct-2002 |
des |
Forgot to disable alarms after fetchXGet() in previous commit.
|
#
106041 |
|
27-Oct-2002 |
des |
Add an ETA timer that kicks in after 30 seconds.
Re-add alarm(2) calls around the calls to fetchStat(3) and fetchXGet(3), since these calls can still time out on DNS lookups or TCP connect(2).
Remove the alarm(2) calls in the main loop, since all methods properly handle transfer timeouts (as opposed to connection timeouts).
Set the sigalrm flag if a timeout occurs in the main loop.
Move the signal: label up a little so we still set the atime and mtime when the transfer times out or is interrupted, so that restarted transfers will work as expected (as long as the file still exists).
MFC after: 2 weeks
|
#
102478 |
|
27-Aug-2002 |
des |
s/optarg/s/ (cut'n'paste braino)
Spotted by: kris
|
#
100834 |
|
28-Jul-2002 |
des |
Use strtol(3) instead of a home-grown version, and fix a "possibly unitialized variable" warning.
|
#
95258 |
|
22-Apr-2002 |
des |
Usage style sweep: spell "usage" with a small 'u'. Also change one case of blatant __progname abuse (several more remain) This commit does not touch anything in src/{contrib,crypto,gnu}/.
|
#
93257 |
|
27-Mar-2002 |
bde |
Restored the part of 1.45 that was clobbered in rev.1.46: #include <sys/time.h> instead of depending on namespace pollution in <sys/stat.h> for the declaration of struct timeval.
|
#
93213 |
|
26-Mar-2002 |
charnier |
Add FBSDID. Remove unused include. Getopt returns -1 not EOF.
|
#
91225 |
|
25-Feb-2002 |
bde |
#include <sys/time.h> instead of depending on namespace pollution in <sys/stat.h> for the declaration of struct timeval.
Removed unused includes (<time.h> doesn't declare anything of interest; only <sys/time.h> does).
Sorted includes a bit.
|
#
90729 |
|
16-Feb-2002 |
des |
If downloading to a temporary file, set the timestamp on the temp file, not on the file we're replacing.
PR: bin/34992 Submitted by: Shunichiro Ariura <syun1rou@blackshell.org> MFC after: 1 week
|
#
87563 |
|
09-Dec-2001 |
des |
Set fetchDebug if v_level is 3 or more.
PR: bin/32615 MFC after: 1 week
|
#
86242 |
|
10-Nov-2001 |
iedowse |
Remove an extraneous argument to fprintf that -Wformat noticed.
Reviewed by: des
|
#
85933 |
|
02-Nov-2001 |
des |
Remove a debugging warnx() that got committed by mistake.
|
#
83863 |
|
23-Sep-2001 |
des |
Don't print running stats unless we're the foreground process.
PR: 30764
|
#
83307 |
|
10-Sep-2001 |
mike |
Add a cast to silence a warning (error because of WARNS=2) on Alpha.
Discussed with: des
|
#
83217 |
|
08-Sep-2001 |
des |
If the local file does not exist, or is a regular file, and we're not trying to resume a transfer, download the requested document into a temporary file which we later rename. This avoids leaving half-completed files around in case of a crash (it'll still leave a half-completed file, but with a hope- fully non-conflicting name), and should reduce the need for human inter- vention on ports-building machines.
The temporary file name for "foo/bar" is constructed by invoking mkstemps() with the pattern "foo/.fetch.XXXXXX.bar"
Requested by: obrien
|
#
81884 |
|
18-Aug-2001 |
des |
Remove a blank line that snuck in with the previous commit. Set the default timeout to 120 seconds instead of 0 (no timeout).
|
#
81863 |
|
17-Aug-2001 |
des |
Add some comments.
|
#
80521 |
|
28-Jul-2001 |
se |
Include value of command line argument that causes an error message or warning in said message, since fetch may be run from a makefile or script which does not print the command line.
Approved by: des
|
#
79838 |
|
17-Jul-2001 |
des |
If we're writing to stdout, we don't care about the size.
Submitted by: wollman
|
#
79837 |
|
17-Jul-2001 |
des |
WARNS cleanup + remove an unused macro and an unused function argument.
|
#
79422 |
|
08-Jul-2001 |
des |
Fix an embarassing if-test-reversal bug that broke scheme guessing.
|
#
77576 |
|
01-Jun-2001 |
des |
Don't try to print a completion percentage for zero-size files.
PR: 27780 Submitted by: Craig Leres <leres@ee.lbl.gov>
|
#
77260 |
|
27-May-2001 |
des |
Damn. Fix typo in previous commit.
|
#
77241 |
|
26-May-2001 |
des |
When running on a tty, install an authentication callback.
|
#
74957 |
|
28-Mar-2001 |
des |
Bring the usage message in synch with reality.
PR: bin/26160
|
#
74717 |
|
23-Mar-2001 |
des |
Now that libfetch uses the high port range by default, add a -U option to make it use the low (default) port range instead.
|
#
73937 |
|
07-Mar-2001 |
des |
If SIGINFO is received during the transfer, print a status line similar to the summary shown at the end of the transfer.
Some days, I really hate Unix...
Suggested by: cracauer
|
#
70275 |
|
22-Dec-2000 |
des |
Warn if the size of the remote file isn't known. If the -R option was specified, don't truncate the local file even if its mtime is incorrect.
PR: bin/23719
|
#
69976 |
|
13-Dec-2000 |
des |
If the URL did not specify a scheme, try to guess it from the host name.
|
#
69383 |
|
30-Nov-2000 |
des |
If the transfer timed out, but we don't know how large the file is supposed to be, assume it was truncated.
|
#
69256 |
|
27-Nov-2000 |
kris |
Format string auditing
|
#
67326 |
|
19-Oct-2000 |
des |
Understand the difference between an empty file and a non-existent file. This has been sitting in my tree for ages...
|
#
65362 |
|
02-Sep-2000 |
des |
Don't try to set the mtime of the output file if it's not a regular file.
Pointed out by: cwt
|
#
65295 |
|
31-Aug-2000 |
des |
Forced commit to note that the previous commit closed a PR.
PR: 19748
|
#
65277 |
|
31-Aug-2000 |
des |
Don't unlink the target file if it's not a regular file.
|
#
63877 |
|
26-Jul-2000 |
des |
Restore the old semantics of deleting the output file when interrupted. Some people liked this and some didn't, so POLA won.
|
#
63720 |
|
21-Jul-2000 |
des |
Don't print summary in quiet mode.
|
#
63717 |
|
21-Jul-2000 |
des |
Don't display a running count in quiet mode. Don't add the offset to the size; it's libfetch's job to report the correct size of the requested file.
|
#
63568 |
|
19-Jul-2000 |
des |
Fix two bugs related to resumed transfers:
- if the dates didn't match, fetch would append the received file to the existing file instead of replacing it.
- if the local file was complete and up-to-date, fetch would miscalculate the expected size and report a failure instead of a success, because it had no way of knowing that the server was actually resending the entire file since the requested offset was invalid.
|
#
63501 |
|
19-Jul-2000 |
des |
Braino: don't use passive mode unless the user requests it.
PR: bin/20025 Submitted by: HIYAMA Takeshi <th@cis.ibaraki.ac.jp>
|
#
63353 |
|
17-Jul-2000 |
des |
While I'm at it, break a line that was too long, remove a pointless diagnostic and adjust the verbosity level of another.
|
#
63345 |
|
17-Jul-2000 |
des |
Rearrange / rewrite large portions of fetch() to take advantage of new libfetch features (fetchRestartCalls, fetchXGet()).
Since it doesn't make much sense to have m_flag and r_flag set at the same time, and it can actually cause trouble in some cases, die if they're both set.
Set the SA_RESETHAND flag for SIGINT so that when we've caught one, we can kill ourselves with a second SIGINT (thus notifying our parent of our tragic fate) instead of just exiting.
These changes fix several problems that would show up when fetching ports, as well as speeding up HTTP transfers quite a bit (at least for relatively small files).
Most of these changes were prompted by an interaction problem with an HTTP server called SWS-1.0, which exhibited two bugs, the first of which prevented fetch from working around the second (the first was not sending content-type in reply to HEAD requests, the second was sending garbage after the end of the requested file).
|
#
63235 |
|
15-Jul-2000 |
des |
Better signal handling.
Submitted by: green
|
#
63067 |
|
13-Jul-2000 |
des |
Fix a bug in the stats display that showed up when resuming aborted transfers.
Submitted by: green
|
#
63046 |
|
12-Jul-2000 |
des |
Rework the stats code for the nth time. Much cleaner now. Always display the completion percentage if stderr is a tty. Drop the char-by-char transfer mode, it was based on an incorrect assumption regarding the semantics of fread(). Finally (I hope) straighten out the business of setting the mtime, as well as when to remove the output file and when not to. Thanks are owed to the many who have provided nearly instantaneous and highly constructive feedback and suggestions about these matters.
|
#
63015 |
|
12-Jul-2000 |
des |
Now that the HTTP code supports timeouts, we don't need to use alarm() around the call to fetchStat(). Catch SIGINT, and rework the signal handling so it doesn't skimp on the cleanup after a timeout or interrupt. Also, don't just bail out after a timeout; there may be more files to fetch.
|
#
63005 |
|
12-Jul-2000 |
des |
Rework the stats code to avoid code duplication. Fix a bug where the stats code would print the expected size instead of the number of bytes received. Fix the reading code so it'll support partial reads.
|
#
62837 |
|
09-Jul-2000 |
des |
Don't segfault if given an invalid URL (doh!)
Submitted by: green
|
#
62815 |
|
08-Jul-2000 |
des |
Straighten out the behvaiour of -m and -n (thanks to bde). Check that stderr, not stdout, is a tty (thanks to green).
|
#
62254 |
|
29-Jun-2000 |
des |
Implement the -c option
|
#
62245 |
|
29-Jun-2000 |
des |
Don't forget to delete the output file if the request fails. Don't delete the output file if -r was specified.
|
#
62216 |
|
28-Jun-2000 |
des |
New libfetch-based fetch.
|