urlgrabber

urlgrabber is a pure python package that drastically simplifies the fetching of files. It is designed to be used in programs that need common (but not necessarily simple) url-fetching features. It is extremely simple to drop into an existing program and provides a clean interface to protocol-independant file-access. Best of all, urlgrabber takes care of all those pesky file-fetching details, and lets you focus on whatever it is that your program is written to do!

urlgrabber came into existence as the part of yum that downloads rpms and header files, but it quickly became clear that this is a general problem that many applications must deal with.

Features

Using urlgrabber, data can be fetched in three basic ways:
urlgrab(url) copy the file to the local filesystem
urlopen(url) open the remote file and return a file object
urlread(url) return the contents of the file as a string
When using these functions (or methods), urlgrabber supports the following features:

Not sure if urlgrabber is the tool for you? Check out our comparison of the major options.

Documentation, Examples, and Help

There are many sources of urlgrabber-related assistance and information

Authors and Credits

urlgrabber is written and maintained by Michael Stenner and Ryan Tomayko. We would like to thank Seth Vidal for many valuable ideas and suggestions, and also for the very earliest version of the code that became urlgrabber. We would also like to thank Linux@DUKE and Duke University for the resources they have provided.

All urlgrabber-related mail (questions, comments, requests, bug reports, praise) should be directed to the yum-devel mailing list. Please indicate that it is a urlgrabber-related post by beginning the subject with [UG].

Bugs, Patches, and Contributions

We appreciate contributions in all forms. You can submit bugs and patches either to the Red Hat bugzilla or the the mailing list as described above. We ask that patches adhere to the following guidelines.

License and Copyright

urlgrabber is © 2002-2006 Michael D. Stenner and Ryan Tomayko.
Portions are © 2009 Red Hat, Inc

This software is licensed under the GNU LGPL and comes without any warranty, written or implied. For more information about GNU LGPL please see http://www.gnu.org/licenses/lgpl.html.

Download

Release information:

urlgrabber follows kernel-style version numbering. As such, the 3.0.x series is the current "stable" branch, and 3.9.x is considered development.

We are no longer providing RPMs for urlgrabber for two reasons: