1-
21:mod: `urllib ` --- Open arbitrary resources by URL
32=================================================
43
@@ -17,8 +16,8 @@ built-in function :func:`open`, but accepts Universal Resource Locators (URLs)
1716instead of filenames. Some restrictions apply --- it can only open URLs for
1817reading, and no seek operations are available.
1918
20- It defines the following public functions:
21-
19+ High-level interface
20+ --------------------
2221
2322.. function :: urlopen(url[, data[, proxies]])
2423
@@ -174,6 +173,9 @@ It defines the following public functions:
174173 :func: `urlretrieve `.
175174
176175
176+ Utility functions
177+ -----------------
178+
177179.. function :: quote(string[, safe])
178180
179181 Replace special characters in *string * using the ``%xx `` escape. Letters,
@@ -235,6 +237,9 @@ It defines the following public functions:
235237 to decode *path *.
236238
237239
240+ URL Opener objects
241+ ------------------
242+
238243.. class :: URLopener([proxies[, **x509]])
239244
240245 Base class for opening and reading URLs. Unless you need to support opening
@@ -260,6 +265,48 @@ It defines the following public functions:
260265 :class: `URLopener ` objects will raise an :exc: `IOError ` exception if the server
261266 returns an error code.
262267
268+ .. method :: open(fullurl[, data])
269+
270+ Open *fullurl * using the appropriate protocol. This method sets up cache and
271+ proxy information, then calls the appropriate open method with its input
272+ arguments. If the scheme is not recognized, :meth: `open_unknown ` is called.
273+ The *data * argument has the same meaning as the *data * argument of
274+ :func: `urlopen `.
275+
276+
277+ .. method :: open_unknown(fullurl[, data])
278+
279+ Overridable interface to open unknown URL types.
280+
281+
282+ .. method :: retrieve(url[, filename[, reporthook[, data]]])
283+
284+ Retrieves the contents of *url * and places it in *filename *. The return value
285+ is a tuple consisting of a local filename and either a
286+ :class: `mimetools.Message ` object containing the response headers (for remote
287+ URLs) or ``None `` (for local URLs). The caller must then open and read the
288+ contents of *filename *. If *filename * is not given and the URL refers to a
289+ local file, the input filename is returned. If the URL is non-local and
290+ *filename * is not given, the filename is the output of :func: `tempfile.mktemp `
291+ with a suffix that matches the suffix of the last path component of the input
292+ URL. If *reporthook * is given, it must be a function accepting three numeric
293+ parameters. It will be called after each chunk of data is read from the
294+ network. *reporthook * is ignored for local URLs.
295+
296+ If the *url * uses the :file: `http: ` scheme identifier, the optional *data *
297+ argument may be given to specify a ``POST `` request (normally the request type
298+ is ``GET ``). The *data * argument must in standard
299+ :mimetype: `application/x-www-form-urlencoded ` format; see the :func: `urlencode `
300+ function below.
301+
302+
303+ .. attribute :: version
304+
305+ Variable that specifies the user agent of the opener object. To get
306+ :mod: `urllib ` to tell servers that it is a particular user agent, set this in a
307+ subclass as a class variable or in the constructor before calling the base
308+ constructor.
309+
263310
264311.. class :: FancyURLopener(...)
265312
@@ -289,6 +336,18 @@ It defines the following public functions:
289336 users for the required information on the controlling terminal. A subclass may
290337 override this method to support more appropriate behavior if needed.
291338
339+ The :class: `FancyURLopener ` class offers one additional method that should be
340+ overloaded to provide the appropriate behavior:
341+
342+ .. method :: prompt_user_passwd(host, realm)
343+
344+ Return information needed to authenticate the user at the given host in the
345+ specified security realm. The return value should be a tuple, ``(user,
346+ password) ``, which can be used for basic authentication.
347+
348+ The implementation prompts for this information on the terminal; an application
349+ should override this method to use an appropriate interaction model in the local
350+ environment.
292351
293352.. exception :: ContentTooShortError(msg[, content])
294353
@@ -297,7 +356,9 @@ It defines the following public functions:
297356 *Content-Length * header). The :attr: `content ` attribute stores the downloaded
298357 (and supposedly truncated) data.
299358
300- Restrictions:
359+
360+ :mod: `urllib ` Restrictions
361+ --------------------------
301362
302363 .. index ::
303364 pair: HTTP; protocol
@@ -358,75 +419,6 @@ Restrictions:
358419 module :mod: `urlparse `.
359420
360421
361- .. _urlopener-objs :
362-
363- URLopener Objects
364- -----------------
365-
366- .. sectionauthor :: Skip Montanaro <skip@pobox.com>
367-
368-
369- :class: `URLopener ` and :class: `FancyURLopener ` objects have the following
370- attributes.
371-
372-
373- .. method :: URLopener.open(fullurl[, data])
374-
375- Open *fullurl * using the appropriate protocol. This method sets up cache and
376- proxy information, then calls the appropriate open method with its input
377- arguments. If the scheme is not recognized, :meth: `open_unknown ` is called.
378- The *data * argument has the same meaning as the *data * argument of
379- :func: `urlopen `.
380-
381-
382- .. method :: URLopener.open_unknown(fullurl[, data])
383-
384- Overridable interface to open unknown URL types.
385-
386-
387- .. method :: URLopener.retrieve(url[, filename[, reporthook[, data]]])
388-
389- Retrieves the contents of *url * and places it in *filename *. The return value
390- is a tuple consisting of a local filename and either a
391- :class: `mimetools.Message ` object containing the response headers (for remote
392- URLs) or ``None `` (for local URLs). The caller must then open and read the
393- contents of *filename *. If *filename * is not given and the URL refers to a
394- local file, the input filename is returned. If the URL is non-local and
395- *filename * is not given, the filename is the output of :func: `tempfile.mktemp `
396- with a suffix that matches the suffix of the last path component of the input
397- URL. If *reporthook * is given, it must be a function accepting three numeric
398- parameters. It will be called after each chunk of data is read from the
399- network. *reporthook * is ignored for local URLs.
400-
401- If the *url * uses the :file: `http: ` scheme identifier, the optional *data *
402- argument may be given to specify a ``POST `` request (normally the request type
403- is ``GET ``). The *data * argument must in standard
404- :mimetype: `application/x-www-form-urlencoded ` format; see the :func: `urlencode `
405- function below.
406-
407-
408- .. attribute :: URLopener.version
409-
410- Variable that specifies the user agent of the opener object. To get
411- :mod: `urllib ` to tell servers that it is a particular user agent, set this in a
412- subclass as a class variable or in the constructor before calling the base
413- constructor.
414-
415- The :class: `FancyURLopener ` class offers one additional method that should be
416- overloaded to provide the appropriate behavior:
417-
418-
419- .. method :: FancyURLopener.prompt_user_passwd(host, realm)
420-
421- Return information needed to authenticate the user at the given host in the
422- specified security realm. The return value should be a tuple, ``(user,
423- password) ``, which can be used for basic authentication.
424-
425- The implementation prompts for this information on the terminal; an application
426- should override this method to use an appropriate interaction model in the local
427- environment.
428-
429-
430422.. _urllib-examples :
431423
432424Examples
0 commit comments